Skip to content

Add local caching#24

Open
vavalomi wants to merge 5 commits intoworldbank:masterfrom
vavalomi:master
Open

Add local caching#24
vavalomi wants to merge 5 commits intoworldbank:masterfrom
vavalomi:master

Conversation

@vavalomi
Copy link
Collaborator

@vavalomi vavalomi commented Jun 5, 2024

!!! WORK-IN-PROGRESS

These changes add support for local caching for the datasets as well as catalogs.

  • new, general-purpose, and transparent caching mechanism added via -datachache- command.
  • some of the existing approaches of locally stored data (specifically for catalogs) were switched to this new approach.
  • TODO: price data as well as latest catalogs are stored in mata matrices, so could be also transitioned to this.
  • TODO: no cache expiration procedures have been implemented yet. The only way to refresh currently is to manually delete local files.

TODO: there are parts of the system (within main datalibweb, as well as separate ado files) that may still make direct API calls and therefore not go through caching.

TESTING: as large parts of the code affecting logic were modified, testing needs to be done to identify broken workflows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments