Need guidance for Manticore reindexing process.

[B.JEGADEESAN]

After migrating from Sphinx 3.3.1 to Manticore 6.2.12 with real-time (RT) configurations, we initiated the reindexing process using multiple processes. Presently, both real-time mails and backend reindexing mails are visible in the web UI. With approximately 18 million mails, we are seeking guidance on determining the optimal number of IDs for reindexing in one process. Moreover, we want to ensure that this reindexing process does not adversely impact or cause crashes in real-time data or the ongoing backend reindexing process.

Regarding Manticore RT indexing, please provide information on the cron jobs that need to be enabled and those that should not be enabled.

We are currently using Piler version 1.4.4 and Manticore version 6.2.12.

[Janos SUTO]

Indexing 18 million emails is no small feat. Assuming that you run several reindex processes in parallel, I’d say a 10k batch sounds reasonable to me, ie. reindex -f 1 -t 10000, reindex -f 10001 -t 20000, etc. The number of reindex processes should depend on your system resources, eg. cpu, memory and i/o.

Regarding the cron jobs disable the indexer.*.sh scripts, and keep everything else.