notes about search indexing
speedy indexing / avoiding troubles
it shouldn't be needed to do a 100% full index build, doing and continuing a partial index run must be possible (STILL MISSING)
one idea to implement partial indexing / resumable indexing could be to have different indexing phases:
- index all items' metadata, but not the content - this should be rather quick.
- index the current revision of all items, but not the content.
- index content of current revisions
- index content of all other revisions
Problems that can happen:
- we have to expect crashes from malfunctioning filters
- we have to expect misbehaved converters/filters (like consuming and not freeing lots of memory)