Make dependency storage.get() more efficient

Issue #51 new
Gilmore Davidson created an issue

I'm writing a job that relies on stored data for past weeks.

To get that data now, using the storage dependency, I need to loop over each week and try to get the data for that week. Each storage.get() loads and parses a JSON file every time.

I suggest two things, either of which could be implemented on its own.

  1. Once the JSON file has been loaded from the first .get() call, keep the result in memory in the StorageFS instance. Subsequent gets can just look at that cached data. Any call to .set() can update the in-memory object as well as write the new JSON file.
  2. Add a new method to get all defined values out of the JSON at once, so the job could just call storage.getAll() or something similar.

Comments (2)

  1. Iván Loire

    Another option is to implement a more powerful data store like redis (a blazing fast in-memory database).

  2. Log in to comment