kbatch not working in Sphinx version 3.1.1 / upgrading sphinx 2.3.2 to 3.x.x

[Valerio W.]

Hi Janos, I’ve just upgraded my piler installation to the latest from the master branch. Everything seems to work fine, thanks for that!

I’ve also tried to upgrade sphinx from version 2.3.2 to version 3.1.1 but I’ve run into a bunch of problems:

First, it seems that “sql_query_kbatch” is only working since sphinx version 3.1.2:

see https://sphinxsearch.com/forum/view.html?id=15995

so your commit 35d3685: https://bitbucket.org/jsuto/piler/commits/35d3685360a98a4f1f332ebb194f4a485404d5d2 should check a version higher than 311 (otherwise I get the error message: ERROR: unknown key name 'sql_query_kbatch'…).

Despite of that I’ve tried to upgrade with the latest available sphinx version 3.2.1 according to the following scheme:

1. used new sphinx.conf,filled out the necessary variables, made it executable
2. checked if searchd is happy with it
3. got now sphinx binaries and put them to /usr/bin
4. stopped searchd
5. removed everything from /var/piler/sphinx/
6. su -c “indexer --all --config /usr/local/etc/piler/sphinx.conf” piler
7. started searchd
8. reindex -f 1 -t 100
9. /usr/local/libexec/piler/indexer.delta.sh

But at point 9 I always get the following error message:

FATAL: failed to merge index 'delta1' into index 'dailydelta1': kbatch
target 'main1' from merge source index must also be in destination index

I’ve mainly used the information/hints from your conversation with BKH: https://www.mail-archive.com/[e-mail]/msg01418.html

At this point I got stuck, and because none of the my old emails appeard in the webinterface I’ve reverted the sphinx related changes and stayed at version 2.3.2 for now.

Do you may have some tips for me, how I can upgrade sphinx successfully ?

Thanks in advance!

Valerio

‌

[Janos SUTO]

Hello Valerio, firstly I suggest to upgrade to sphinx 3.2.1. Then check if you have SPHINX_VERSION 321 defined in sphinx.conf file. Your steps look reasonable.

[Valerio W.]

Hello Janos, I’ve already done that. Unfortunately, I got stuck with the error message: “FATAL…”, mentioned earlier.

[Janos SUTO]

I’ve just kicked in my pipeline.

I have Sphinx 3.2.1 (commit f152e0b)

sphinx.conf has “define('SPHINX_VERSION', 321);”

There was no issue when it created the index files (6). It was running indexer.delta.sh just fine. Then I’ve reindexed 100 emails, and ran the delta indexer without any issue. Sorry, I can’t reproduce this error.

[Janos SUTO]

No news is good news.