Some experience when trying to improve the performance.

[零欸特]

Recently, I found that the popup item list often resets after I modifying the search query, so I decided to refactor the popup with more reactive design to improve the performance.

The result is pretty good:
https://bitbucket.org/eight04/in-my-pocket/branches/compare/dev-refactor-popup%0Ddev-fast-open2#diff

Everything is reactive hence we can remove lots of duplicated logic.

However, I also noticed a performance spike when modifying the items array (e.g. when archiving an item in the popup), and the issue is that we store a very large string (I have 15k items) in browser.storage: When an item is modified, the browser has to send the entire string via message passing to other processes to notify the change i.e. browser.storage.onChanged, which is extremely slow, and we don't need the full array for a single modification.

I think we have following options:

1. Drop browser.storage.onChanged completely, and develop our own message system from scratch using browser.runtime.sendMessage. It seems that browser doesn't broadcast onChanged event when there is no listener, like the current master branch. Note that we won't be able to use onChanged event on settings either i.e. we can't use browser.storage.sync in the future.
2. Separate the large array into small pieces. For example, we can use an array to store item id (e.g. items/index), and store item data to a different key (e.g. items/data/${item.id}). Therefore when modifying an item, the browser will only pass a single item data; when archiving an item, the browser will only pass the array including item id.
3. Store items into IndexedDB instead of browser.storage, then use our own message system to make it reactive. This will be very fast since we know exactly what data should be broadcast. We also keep the possibility to use onChanged event in the future.

Personally, I think (2) should be enough to improve the performance. By only storing the item id, the size of the large items string reduces to ~6% (according to my pocket item list).

WDYT? I may start working on (2) these weeks if there is no response.

[零欸特]

Some investigation about storage/message passing performance (with 15k items):

1. If items are stored separately in browser.storage, it takes ~5 seconds to read all of them.
2. If items are stored in a single array in localStorage, it takes ~150ms to read all of them, including JSON.parse.
3. Passing all items via browser.runtime.sendMessage takes ~1.5 seconds.
4. Passing 50 items via browser.runtime.sendMessage takes 50ms.

Conclusion: the fastest way is to sort/filter/paginate items in the background, so the popup only needs 50ms to setup.

[零欸特]

Here is a working branch. It stores items separately in browser.storage, cache all items in the background. The popup also caches items in localStorage:
https://bitbucket.org/eight04/in-my-pocket/branches/compare/dev-refactor-db%0Ddev-fast-open2#diff

[零欸特]

With dev-refactor-db branch, there is almost no lag spike after archiving an item:

[零欸特]

Passing all items via browser.runtime.sendMessage takes ~1.5 seconds.

Another thing that may worth investigating is to JSON.stringify the data before sending, so the browser will clone a single string instead of a complex object.

[Pierre-Adrien Buisson]

Hey, wow thanks for the investigation. I’m curious as to how you managed to profile the addon like this?

I gave your branch a look but to be 100% honest, I’m not sure I understand what event-lite is doing, and most of all I’m not fond of onboarding a dependency I’m not really understanding and not familiar with (even less so far a lib like this with 2 contribs and 40 commits) (same for npm-run-all2 even though it’s not core to the branch you’ve worked on, as far as I understand).

I’ll give this another look someday though!

[零欸特]

You have to use Firefox Profiler: https://profiler.firefox.com/

Document: https://profiler.firefox.com/docs/#/./guide-getting-started

what event-lite is doing

It is an event emitter library. Event emitter is a design pattern which is often used in JavaScript. For example:

// items.js
import {updateCount} from "./badge.js"
import {updateList} from "./item-list.js";
function addItem() {
  // ... add an item ...
  updateCount();
  updateList();
}
function archiveItem() {
  // ... archiving an item ...
  updateCount();
  updateList();
}

With event emitter:

// items.js
export const events = new EventEmitter();

function addItem() {
  // ... add an item ...
  events.emit("change")
}
function archiveItem() {
  // ... archiving an item ...
  events.emit("change")
}

// badge.js
import {events} from "./items.js";
events.on("change", updateCount);

// items-list.js
import {events} from "./items.js";
events.on("change", updateList);

The main advantage is that items.js doesn't have to import implementations from other modules and can easily cooperate with them by broadcasting a special message. Event pattern is useful when multiple modules have to react to a specific source.

[零欸特]

Here is the branch that I use currently:
https://bitbucket.org/eight04/in-my-pocket/branches/compare/dev-store%0Ddev-fast-open2#diff

With more bugfixes and performance improvement.