faster initial page loads for Browse or Review

[Craig Berry]

It can take 30 to 60 seconds to get the initial page of results loaded whether browsing texts or reviewing annotations. This is about to get worse as we more than double the 12,000+ texts available, and it's already a problem with 300,000+ annotations. If the search that creates the list could be fast enough that would be the best solution. Barring that, perhaps we could force the user to choose a filter before seeing any results.

We could give an initial indication of how many texts there are with something like:

xquery version "3.1";
declare namespace tei="http://www.tei-c.org/ns/1.0";

let $col := collection('/db/apps/EarlyPrintTexts/texts')
let $numdocs := count($col//tei:TEI)
return $numdocs

which runs in the tens of milliseconds. So we could say something like:

"This collection includes 12,523 documents; please make selections in the form to the left and click 'Filter' to view your subset of interest."

[Craig Berry]

memsort may be something that could help with this:

https://github.com/wolfgangmm/memsort

It has to be generated at run time and is not automatically updated when the underlying data change, so it may be more suitable to the texts collection than the annotations collection.

[Craig Berry]

Memsort has been implemented for Browse and has the desired effect. It can't have the same benefits for the Review page because annotations are dynamic (can be added, changed, or removed at run time) so the memsort indexes would have to be recreated for each query, thus losing any benefit they could provide.

[Craig Berry]

The problem with Review is not initial page load, but every page load. Initial query time is usually quite fast, but fetching the context for each hit is quite slow due to inherent limitations in eXist that we can't fix.