1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

After Update from 1.3.9 to 1.3.10 empty search results

Issue #1146 closed
Jan Ebinger created an issue

Hello,

after updating my mailpiler from 1.3.9 to 1.3.10 i get only emty search results.

My system runs on Debian 10.

‌

I did the following steps for updating:

sudo sudo
cd

/etc/init.d/rc.searchd stop
/etc/init.d/rc.piler stop

cd build
wget https://bitbucket.org/jsuto/piler/downloads/piler-1.3.10.tar.gz
tar xfv piler-1.3.10.tar.gz
cd piler-1.3.10
./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var --with-database=mysql --enable-memcached
make
make install
ldconfig

/etc/init.d/rc.searchd start
/etc/init.d/rc.piler start

If i run /etc/init.d/rc.searchd start i get the following output:

root@archiv:~/build/piler-1.3.10# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
WARNING: index 'main1': preload: failed to open /var/piler/sphinx/main1.sph: No such file or directory; NOT SERVING
precaching index 'main2'
WARNING: index 'main2': preload: failed to open /var/piler/sphinx/main2.sph: No such file or directory; NOT SERVING
precaching index 'main3'
WARNING: index 'main3': preload: failed to open /var/piler/sphinx/main3.sph: No such file or directory; NOT SERVING
precaching index 'main4'
WARNING: index 'main4': preload: failed to open /var/piler/sphinx/main4.sph: No such file or directory; NOT SERVING
precaching index 'dailydelta1'
precaching index 'delta1'
precaching index 'tag1'
precaching index 'note1'
precached 8 indexes in 0.029 sec
root@archiv:~/build/piler-1.3.10#

‌

Does anyone have an idea?

Best regards

Jan

‌

Comments (20)

  1. Janos SUTO repo owner

    It looks like an issue with the sphinx database. Check if you have main*.* files in /var/piler/sphinx and they are owned by piler:piler

  2. Jan Ebinger reporter

    I have checked it and got the folowing output:

    root@archiv:/var/piler/sphinx# ls -la
insgesamt 593472
drwx------ 2 piler piler      4096 Feb  8 15:14 .
drwxr-xr-x 9 root  root       4096 Jul 28  2020 ..
-rw-r--r-- 1 piler piler   2763820 Feb  8 15:05 dailydelta1.spa
-rw-r--r-- 1 piler piler 276763068 Feb  8 15:05 dailydelta1.spd
-rw-r--r-- 1 piler piler   2128227 Feb  8 15:05 dailydelta1.spe
-rw-r--r-- 1 piler piler      1164 Feb  8 15:05 dailydelta1.sph
-rw-r--r-- 1 piler piler   7282590 Feb  8 15:05 dailydelta1.spi
-rw-r--r-- 1 piler piler         0 Feb  8 15:05 dailydelta1.spk
-rw-r--r-- 1 piler piler         0 Feb  8 15:05 dailydelta1.spm
-rw-r--r-- 1 piler piler 318417375 Feb  8 15:05 dailydelta1.spp
-rw-r--r-- 1 piler piler         1 Feb  8 15:05 dailydelta1.sps
-rw-r--r-- 1 root  root          1 Jul 23  2020 dailydelta1.tmp.tmps
-rw-r--r-- 1 piler piler       740 Feb  8 15:05 delta1.spa
-rw-r--r-- 1 piler piler     43995 Feb  8 15:05 delta1.spd
-rw-r--r-- 1 piler piler         1 Feb  8 15:05 delta1.spe
-rw-r--r-- 1 piler piler      1164 Feb  8 15:05 delta1.sph
-rw-r--r-- 1 piler piler     29068 Feb  8 15:05 delta1.spi
-rw-r--r-- 1 piler piler      7216 Feb  8 15:05 delta1.spk
-rw-r--r-- 1 piler piler         0 Feb  8 15:05 delta1.spm
-rw-r--r-- 1 piler piler     62612 Feb  8 15:05 delta1.spp
-rw-r--r-- 1 piler piler         1 Feb  8 15:05 delta1.sps
-rw-r--r-- 1 root  root          0 Jul 23  2020 main1.spl
-rw-r--r-- 1 piler piler         0 Feb  8 15:00 note1.spa
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 note1.spd
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 note1.spe
-rw-r--r-- 1 piler piler       872 Feb  8 15:00 note1.sph
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 note1.spi
-rw-r--r-- 1 piler piler         0 Feb  8 15:00 note1.spk
-rw-r--r-- 1 piler piler         0 Feb  8 15:00 note1.spm
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 note1.spp
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 note1.sps
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 note1.tmp.tmps
-rw-r--r-- 1 piler piler         0 Feb  8 15:00 tag1.spa
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 tag1.spd
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 tag1.spe
-rw-r--r-- 1 piler piler       871 Feb  8 15:00 tag1.sph
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 tag1.spi
-rw-r--r-- 1 piler piler         0 Feb  8 15:00 tag1.spk
-rw-r--r-- 1 piler piler         0 Feb  8 15:00 tag1.spm
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 tag1.spp
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 tag1.sps
-rw-r--r-- 1 piler piler         1 Feb  8 15:00 tag1.tmp.tmps

    So I changed the two files with:

    chown piler:piler main1.spl
chown piler:piler dailydelta1.tmp.tmps

    But the error is still there.

  3. Janos SUTO repo owner

    Notice that a given index, eg. delta1 has many more files, not only .spl like main1.spl. Based on the wrong ownership I suspect that you ran the indexer script as root. Try the following: run indexer --config /usr/local/etc/piler/sphinx.conf main1 as user piler, then verify that all main1 files exist.

    However, since your index data is practically gone, I suggest you to upgrade sphinx to 3.3.1, stop searchd, reset the index database with indexer --config /usr/local/etc/piler/sphinx.conf as user piler, then reindex everything.

    Also I suggest to do a daily backup of the main1.* files after the main indexer finished.

  4. Jan Ebinger reporter

    Okay so I got the following output:

    piler@archiv:~$ indexer --config /usr/local/etc/piler/sphinx.conf main1
FATAL: config file '/usr/local/etc/piler/sphinx.conf' does not exist or is not readable
piler@archiv:~$ cat /usr/local/etc/piler/sphinx.conf
cat: /usr/local/etc/piler/sphinx.conf: Datei oder Verzeichnis nicht gefunden

    ‌

  6. Jan Ebinger reporter

    Okay thank you. That works but after these steps the GUI is still showing empty search results.

    piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main1

root@archiv:~# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
precaching index 'main2'
WARNING: index 'main2': preload: failed to open /var/piler/sphinx/main2.sph: No such file or directory; NOT SERVING
precaching index 'main3'
WARNING: index 'main3': preload: failed to open /var/piler/sphinx/main3.sph: No such file or directory; NOT SERVING
precaching index 'main4'
WARNING: index 'main4': preload: failed to open /var/piler/sphinx/main4.sph: No such file or directory; NOT SERVING
precaching index 'dailydelta1'
precaching index 'delta1'
precaching index 'tag1'
precaching index 'note1'
precached 8 indexes in 0.029 sec
root@archiv:~# /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .

root@archiv:~# /etc/init.d/rc.searchd stop
stopping searchd
root@archiv:~# /etc/init.d/rc.piler stop
stopping piler-smtp

su piler

piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main2
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
indexing index 'main2'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 7 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg
piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main3
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
indexing index 'main3'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 7 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg
piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main4
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
indexing index 'main4'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 7 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg


root@archiv:~# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
precaching index 'main2'
precaching index 'main3'
precaching index 'main4'
precaching index 'dailydelta1'
precaching index 'delta1'
precaching index 'tag1'
precaching index 'note1'
precached 8 indexes in 0.039 sec
root@archiv:~# /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .

    ‌

  7. Janos SUTO repo owner

    OK, looks better. I can see you didn’t want to upgrade sphinx. It’s fine. Now you need to reindex everything, and your search results will come back eventually.

  8. Jan Ebinger reporter

    Unfortunately it doesn't work yet. I got some errors.

    piler@archiv:~/tmp$ reindex -a
Syntax Error: Couldn't find trailer dictionary
Syntax Error: Couldn't find trailer dictionary
Syntax Error: Couldn't read xref table
Syntax Error: Couldn't find trailer dictionary
Syntax Error: Couldn't find trailer dictionary
Syntax Error: Couldn't read xref table
zpipe: invalid or incomplete deflate data
zpipe: invalid or incomplete deflate data
zpipe: invalid or incomplete deflate data
Syntax Error: Document stream is empty
zpipe: invalid or incomplete deflate data

    What is the correct way to upgrade sphinx? https://www.mailpiler.org/wiki/current:sphinx3

    1. sudo su
    2. cd /usr/bin
    3. wget https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz
    4. Unpack the .tar.gt-file
    5. Edit the sphinx.conf?

    ‌

  9. Janos SUTO repo owner

    Those messages are errors from the extractor utilities when they find some issue. Anyway, piler can’t do much about it, the message is still archived.

    Unpack the sphinx tar.gz file in / dir because it’s already contains usr/bin path. See if for yourself with tar tzvf sphinx-3.3.1.tar.gz. And yes, edit sphinx.conf to match sphinx version.

  10. Jan Ebinger reporter

    Okay I have edit the/etc/piler/sphinx.conf

    But if I start searchd it shows me the old version 2.2.11.

    root@archiv:~# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 20.
ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 1.
FATAL: failed to parse config file '/etc/piler/sphinx.conf'
root@archiv:~# /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .

    ‌

  12. Jan Ebinger reporter

    I have checked it but the error is still existing.

    root@archiv:~# cd /
root@archiv:/# wget https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz
--2021-02-09 14:33:26--  https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz
Auflösen des Hostnamens download.mailpiler.com (download.mailpiler.com)… 195.201.39.187
Verbindungsaufbau zu download.mailpiler.com (download.mailpiler.com)|195.201.39.187|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: 21232012 (20M) [application/octet-stream]
Wird in »sphinx-3.3.1-bin.tar.gz.1« gespeichert.

sphinx-3.3.1-bin.ta 100%[===================>]  20,25M  74,6MB/s    in 0,3s

2021-02-09 14:33:26 (74,6 MB/s) - »sphinx-3.3.1-bin.tar.gz.1« gespeichert [21232012/21232012]

root@archiv:/# tar tzvf sphinx-3.3.1-bin.tar.gz
drwxr-xr-x root/root         0 2020-07-11 08:06 usr/
drwxr-xr-x root/root         0 2020-07-11 08:06 usr/bin/
-rwxr-xr-x root/root  19925768 2020-07-11 08:06 usr/bin/searchd
-rwxr-xr-x root/root  16636312 2020-07-11 08:06 usr/bin/indextool
-rwxr-xr-x root/root  17666680 2020-07-11 08:06 usr/bin/indexer
-rwxr-xr-x root/root  16612024 2020-07-11 08:06 usr/bin/wordbreaker
root@archiv:/# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 20.
ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 1.
FATAL: failed to parse config file '/etc/piler/sphinx.conf'
root@archiv:/# /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .

    ‌

  14. Jan Ebinger reporter

    That was the problem. tar -xvzf sphinx-3.3.1-bin.tar.gz was good.

    Now i got:

    root@archiv:~# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 3.3.1 (commit b72d67b)
Copyright (c) 2001-2020, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
WARNING: index 'main1': prealloc: /var/piler/sphinx/main1.sph is v.42 (from Sphi                                                                                                             nx 2.x), binary is v.50; NOT SERVING
precaching index 'main2'
WARNING: index 'main2': prealloc: /var/piler/sphinx/main2.sph is v.42 (from Sphi                                                                                                             nx 2.x), binary is v.50; NOT SERVING
precaching index 'main3'
WARNING: index 'main3': prealloc: /var/piler/sphinx/main3.sph is v.42 (from Sphi                                                                                                             nx 2.x), binary is v.50; NOT SERVING
precaching index 'main4'
WARNING: index 'main4': prealloc: /var/piler/sphinx/main4.sph is v.42 (from Sphi                                                                                                             nx 2.x), binary is v.50; NOT SERVING
precaching index 'dailydelta1'
WARNING: index 'dailydelta1': prealloc: /var/piler/sphinx/dailydelta1.sph is v.4                                                                                                             2 (from Sphinx 2.x), binary is v.50; NOT SERVING
precaching index 'delta1'
WARNING: index 'delta1': prealloc: /var/piler/sphinx/delta1.sph is v.42 (from Sp                                                                                                             hinx 2.x), binary is v.50; NOT SERVING
precaching index 'tag1'
WARNING: index 'tag1': prealloc: /var/piler/sphinx/tag1.sph is v.42 (from Sphinx                                                                                                              2.x), binary is v.50; NOT SERVING
precaching index 'note1'
WARNING: index 'note1': prealloc: /var/piler/sphinx/note1.sph is v.42 (from Sphi                                                                                                             nx 2.x), binary is v.50; NOT SERVING
FATAL: no valid indexes to serve
shutdown complete
root@archiv:~# /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .

    ‌

  15. Janos SUTO repo owner

    OK. Then stop searchd, run rm -f /var/piler/sphinx/* then indexer --all --config /etc/piler/sphinx.conf, and finally start searchd. It should start cleanly this time.

  16. Jan Ebinger reporter

    I have done it but the error is still exisitng. 

    piler@archiv:~/sphinx$ /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 3.3.1 (commit b72d67b)
Copyright (c) 2001-2020, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

no config file, using 'sphinxdata' folder...
listening on all interfaces, port=9312
listening on all interfaces, port=9306
piler@archiv:~/sphinx$ /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .
piler@archiv:~/sphinx$ ls
note1.spa  note1.sph  note1.spk   tag1.spa  tag1.sph  tag1.spk
note1.spd  note1.spi  note1.spp   tag1.spd  tag1.spi  tag1.spp
note1.spe  note1.spj  sphinxdata  tag1.spe  tag1.spj

    If I run the main and daily crontab manuel I get these output:

    root@archiv:~# /usr/local/libexec/piler/indexer.delta.sh
ERROR: index 'delta1': reserved field name 'from'.
FATAL: failed to merge index 'delta1' into index 'dailydelta1': failed to open /var/piler/sphinx/dailydelta1.sph: No such file or directory
root@archiv:~# /usr/local/libexec/piler/indexer.main.sh
FATAL: failed to merge index 'dailydelta1' into index 'main1': failed to open /var/piler/sphinx/main1.sph: No such file or directory

    ‌

  17. Janos SUTO repo owner

    Something interferes with the piler shipped sphinx config. The “sphinxdata” dir should not exist. After you ran the commands I showed above, have you checked that all required files are present in /var/piler/sphinx? You need to find what else is running that messed with piler sphinx files.

  18. Jan Ebinger reporter

    Other question but what is when I get the following output in piler-1.3.9? What is to do?

    ‌

    root@archiv:~# /etc/init.d/rc.searchd stop
stopping searchd
root@archiv:~# /etc/init.d/rc.piler stop
stopping piler
stopping piler-smtp

root@archiv:~# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
WARNING: index 'main1': preload: failed to open /var/piler/sphinx/main1.sph: No such file or directory; NOT SERVING
precaching index 'main2'
WARNING: index 'main2': preload: failed to open /var/piler/sphinx/main2.sph: No such file or directory; NOT SERVING
precaching index 'main3'
WARNING: index 'main3': preload: failed to open /var/piler/sphinx/main3.sph: No such file or directory; NOT SERVING
precaching index 'main4'
WARNING: index 'main4': preload: failed to open /var/piler/sphinx/main4.sph: No such file or directory; NOT SERVING
precaching index 'dailydelta1'
precaching index 'delta1'
precaching index 'tag1'
precaching index 'note1'
precached 8 indexes in 0.031 sec
root@archiv:~# /etc/init.d/rc.piler start
starting piler-smtp . . .
starting piler . . .

    ‌

    root@archiv:~# su - piler
piler@archiv:~$ indexer -all
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

ERROR: malformed or unknown option near '-all'.

    ‌

  19. Janos SUTO repo owner

    I think we are back to square #1. You don’t have the main index files. Also pay more attention to what I wrote above: -all vs. --all

  21. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
closed
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!