After Update from 1.3.9 to 1.3.10 empty search results
Hello,
after updating my mailpiler from 1.3.9 to 1.3.10 i get only emty search results.
My system runs on Debian 10.
I did the following steps for updating:
sudo sudo
cd
/etc/init.d/rc.searchd stop
/etc/init.d/rc.piler stop
cd build
wget https://bitbucket.org/jsuto/piler/downloads/piler-1.3.10.tar.gz
tar xfv piler-1.3.10.tar.gz
cd piler-1.3.10
./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var --with-database=mysql --enable-memcached
make
make install
ldconfig
/etc/init.d/rc.searchd start
/etc/init.d/rc.piler start
If i run /etc/init.d/rc.searchd start
i get the following output:
root@archiv:~/build/piler-1.3.10# /etc/init.d/rc.searchd start
starting searchd . . .
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/etc/piler/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
WARNING: index 'main1': preload: failed to open /var/piler/sphinx/main1.sph: No such file or directory; NOT SERVING
precaching index 'main2'
WARNING: index 'main2': preload: failed to open /var/piler/sphinx/main2.sph: No such file or directory; NOT SERVING
precaching index 'main3'
WARNING: index 'main3': preload: failed to open /var/piler/sphinx/main3.sph: No such file or directory; NOT SERVING
precaching index 'main4'
WARNING: index 'main4': preload: failed to open /var/piler/sphinx/main4.sph: No such file or directory; NOT SERVING
precaching index 'dailydelta1'
precaching index 'delta1'
precaching index 'tag1'
precaching index 'note1'
precached 8 indexes in 0.029 sec
root@archiv:~/build/piler-1.3.10#
Does anyone have an idea?
Best regards
Jan
Comments (20)
-
repo owner -
reporter I have checked it and got the folowing output:
root@archiv:/var/piler/sphinx# ls -la insgesamt 593472 drwx------ 2 piler piler 4096 Feb 8 15:14 . drwxr-xr-x 9 root root 4096 Jul 28 2020 .. -rw-r--r-- 1 piler piler 2763820 Feb 8 15:05 dailydelta1.spa -rw-r--r-- 1 piler piler 276763068 Feb 8 15:05 dailydelta1.spd -rw-r--r-- 1 piler piler 2128227 Feb 8 15:05 dailydelta1.spe -rw-r--r-- 1 piler piler 1164 Feb 8 15:05 dailydelta1.sph -rw-r--r-- 1 piler piler 7282590 Feb 8 15:05 dailydelta1.spi -rw-r--r-- 1 piler piler 0 Feb 8 15:05 dailydelta1.spk -rw-r--r-- 1 piler piler 0 Feb 8 15:05 dailydelta1.spm -rw-r--r-- 1 piler piler 318417375 Feb 8 15:05 dailydelta1.spp -rw-r--r-- 1 piler piler 1 Feb 8 15:05 dailydelta1.sps -rw-r--r-- 1 root root 1 Jul 23 2020 dailydelta1.tmp.tmps -rw-r--r-- 1 piler piler 740 Feb 8 15:05 delta1.spa -rw-r--r-- 1 piler piler 43995 Feb 8 15:05 delta1.spd -rw-r--r-- 1 piler piler 1 Feb 8 15:05 delta1.spe -rw-r--r-- 1 piler piler 1164 Feb 8 15:05 delta1.sph -rw-r--r-- 1 piler piler 29068 Feb 8 15:05 delta1.spi -rw-r--r-- 1 piler piler 7216 Feb 8 15:05 delta1.spk -rw-r--r-- 1 piler piler 0 Feb 8 15:05 delta1.spm -rw-r--r-- 1 piler piler 62612 Feb 8 15:05 delta1.spp -rw-r--r-- 1 piler piler 1 Feb 8 15:05 delta1.sps -rw-r--r-- 1 root root 0 Jul 23 2020 main1.spl -rw-r--r-- 1 piler piler 0 Feb 8 15:00 note1.spa -rw-r--r-- 1 piler piler 1 Feb 8 15:00 note1.spd -rw-r--r-- 1 piler piler 1 Feb 8 15:00 note1.spe -rw-r--r-- 1 piler piler 872 Feb 8 15:00 note1.sph -rw-r--r-- 1 piler piler 1 Feb 8 15:00 note1.spi -rw-r--r-- 1 piler piler 0 Feb 8 15:00 note1.spk -rw-r--r-- 1 piler piler 0 Feb 8 15:00 note1.spm -rw-r--r-- 1 piler piler 1 Feb 8 15:00 note1.spp -rw-r--r-- 1 piler piler 1 Feb 8 15:00 note1.sps -rw-r--r-- 1 piler piler 1 Feb 8 15:00 note1.tmp.tmps -rw-r--r-- 1 piler piler 0 Feb 8 15:00 tag1.spa -rw-r--r-- 1 piler piler 1 Feb 8 15:00 tag1.spd -rw-r--r-- 1 piler piler 1 Feb 8 15:00 tag1.spe -rw-r--r-- 1 piler piler 871 Feb 8 15:00 tag1.sph -rw-r--r-- 1 piler piler 1 Feb 8 15:00 tag1.spi -rw-r--r-- 1 piler piler 0 Feb 8 15:00 tag1.spk -rw-r--r-- 1 piler piler 0 Feb 8 15:00 tag1.spm -rw-r--r-- 1 piler piler 1 Feb 8 15:00 tag1.spp -rw-r--r-- 1 piler piler 1 Feb 8 15:00 tag1.sps -rw-r--r-- 1 piler piler 1 Feb 8 15:00 tag1.tmp.tmps
So I changed the two files with:
chown piler:piler main1.spl chown piler:piler dailydelta1.tmp.tmps
But the error is still there.
-
repo owner Notice that a given index, eg. delta1 has many more files, not only .spl like main1.spl. Based on the wrong ownership I suspect that you ran the indexer script as root. Try the following: run indexer --config /usr/local/etc/piler/sphinx.conf main1 as user piler, then verify that all main1 files exist.
However, since your index data is practically gone, I suggest you to upgrade sphinx to 3.3.1, stop searchd, reset the index database with indexer --config /usr/local/etc/piler/sphinx.conf as user piler, then reindex everything.
Also I suggest to do a daily backup of the main1.* files after the main indexer finished.
-
reporter Okay so I got the following output:
piler@archiv:~$ indexer --config /usr/local/etc/piler/sphinx.conf main1 FATAL: config file '/usr/local/etc/piler/sphinx.conf' does not exist or is not readable piler@archiv:~$ cat /usr/local/etc/piler/sphinx.conf cat: /usr/local/etc/piler/sphinx.conf: Datei oder Verzeichnis nicht gefunden
-
repo owner Try /etc/piler/sphinx.conf
-
reporter Okay thank you. That works but after these steps the GUI is still showing empty search results.
piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main1 root@archiv:~# /etc/init.d/rc.searchd start starting searchd . . . Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... listening on 127.0.0.1:9312 listening on 127.0.0.1:9306 precaching index 'main1' precaching index 'main2' WARNING: index 'main2': preload: failed to open /var/piler/sphinx/main2.sph: No such file or directory; NOT SERVING precaching index 'main3' WARNING: index 'main3': preload: failed to open /var/piler/sphinx/main3.sph: No such file or directory; NOT SERVING precaching index 'main4' WARNING: index 'main4': preload: failed to open /var/piler/sphinx/main4.sph: No such file or directory; NOT SERVING precaching index 'dailydelta1' precaching index 'delta1' precaching index 'tag1' precaching index 'note1' precached 8 indexes in 0.029 sec root@archiv:~# /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . . root@archiv:~# /etc/init.d/rc.searchd stop stopping searchd root@archiv:~# /etc/init.d/rc.piler stop stopping piler-smtp su piler piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main2 Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... indexing index 'main2'... collected 0 docs, 0.0 MB total 0 docs, 0 bytes total 0.002 sec, 0 bytes/sec, 0.00 docs/sec total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg total 7 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main3 Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... indexing index 'main3'... collected 0 docs, 0.0 MB total 0 docs, 0 bytes total 0.001 sec, 0 bytes/sec, 0.00 docs/sec total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg total 7 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg piler@archiv:~$ indexer --config /etc/piler/sphinx.conf main4 Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... indexing index 'main4'... collected 0 docs, 0.0 MB total 0 docs, 0 bytes total 0.002 sec, 0 bytes/sec, 0.00 docs/sec total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg total 7 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg root@archiv:~# /etc/init.d/rc.searchd start starting searchd . . . Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... listening on 127.0.0.1:9312 listening on 127.0.0.1:9306 precaching index 'main1' precaching index 'main2' precaching index 'main3' precaching index 'main4' precaching index 'dailydelta1' precaching index 'delta1' precaching index 'tag1' precaching index 'note1' precached 8 indexes in 0.039 sec root@archiv:~# /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . .
-
repo owner OK, looks better. I can see you didn’t want to upgrade sphinx. It’s fine. Now you need to reindex everything, and your search results will come back eventually.
-
reporter Unfortunately it doesn't work yet. I got some errors.
piler@archiv:~/tmp$ reindex -a Syntax Error: Couldn't find trailer dictionary Syntax Error: Couldn't find trailer dictionary Syntax Error: Couldn't read xref table Syntax Error: Couldn't find trailer dictionary Syntax Error: Couldn't find trailer dictionary Syntax Error: Couldn't read xref table zpipe: invalid or incomplete deflate data zpipe: invalid or incomplete deflate data zpipe: invalid or incomplete deflate data Syntax Error: Document stream is empty zpipe: invalid or incomplete deflate data
What is the correct way to upgrade sphinx? https://www.mailpiler.org/wiki/current:sphinx3
- sudo su
- cd /usr/bin
- wget https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz
- Unpack the .tar.gt-file
- Edit the sphinx.conf?
-
repo owner Those messages are errors from the extractor utilities when they find some issue. Anyway, piler can’t do much about it, the message is still archived.
Unpack the sphinx tar.gz file in / dir because it’s already contains usr/bin path. See if for yourself with tar tzvf sphinx-3.3.1.tar.gz. And yes, edit sphinx.conf to match sphinx version.
-
reporter Okay I have edit the
/etc/piler/sphinx.conf
But if I start searchd it shows me the old version 2.2.11.
root@archiv:~# /etc/init.d/rc.searchd start starting searchd . . . Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 20. ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 1. FATAL: failed to parse config file '/etc/piler/sphinx.conf' root@archiv:~# /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . .
-
repo owner Double check that /usr/bin/searchd is the new 3.3.1 binary.
-
reporter I have checked it but the error is still existing.
root@archiv:~# cd / root@archiv:/# wget https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz --2021-02-09 14:33:26-- https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz Auflösen des Hostnamens download.mailpiler.com (download.mailpiler.com)… 195.201.39.187 Verbindungsaufbau zu download.mailpiler.com (download.mailpiler.com)|195.201.39.187|:443 … verbunden. HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK Länge: 21232012 (20M) [application/octet-stream] Wird in »sphinx-3.3.1-bin.tar.gz.1« gespeichert. sphinx-3.3.1-bin.ta 100%[===================>] 20,25M 74,6MB/s in 0,3s 2021-02-09 14:33:26 (74,6 MB/s) - »sphinx-3.3.1-bin.tar.gz.1« gespeichert [21232012/21232012] root@archiv:/# tar tzvf sphinx-3.3.1-bin.tar.gz drwxr-xr-x root/root 0 2020-07-11 08:06 usr/ drwxr-xr-x root/root 0 2020-07-11 08:06 usr/bin/ -rwxr-xr-x root/root 19925768 2020-07-11 08:06 usr/bin/searchd -rwxr-xr-x root/root 16636312 2020-07-11 08:06 usr/bin/indextool -rwxr-xr-x root/root 17666680 2020-07-11 08:06 usr/bin/indexer -rwxr-xr-x root/root 16612024 2020-07-11 08:06 usr/bin/wordbreaker root@archiv:/# /etc/init.d/rc.searchd start starting searchd . . . Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 20. ERROR: unknown key name 'sql_query_kbatch' in /etc/piler/sphinx.conf line 27 col 1. FATAL: failed to parse config file '/etc/piler/sphinx.conf' root@archiv:/# /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . .
-
repo owner Sigh. Have you actually extracted the files? Have you checked searchd --version?
-
reporter That was the problem.
tar -xvzf sphinx-3.3.1-bin.tar.gz
was good.Now i got:
root@archiv:~# /etc/init.d/rc.searchd start starting searchd . . . Sphinx 3.3.1 (commit b72d67b) Copyright (c) 2001-2020, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... listening on 127.0.0.1:9312 listening on 127.0.0.1:9306 precaching index 'main1' WARNING: index 'main1': prealloc: /var/piler/sphinx/main1.sph is v.42 (from Sphi nx 2.x), binary is v.50; NOT SERVING precaching index 'main2' WARNING: index 'main2': prealloc: /var/piler/sphinx/main2.sph is v.42 (from Sphi nx 2.x), binary is v.50; NOT SERVING precaching index 'main3' WARNING: index 'main3': prealloc: /var/piler/sphinx/main3.sph is v.42 (from Sphi nx 2.x), binary is v.50; NOT SERVING precaching index 'main4' WARNING: index 'main4': prealloc: /var/piler/sphinx/main4.sph is v.42 (from Sphi nx 2.x), binary is v.50; NOT SERVING precaching index 'dailydelta1' WARNING: index 'dailydelta1': prealloc: /var/piler/sphinx/dailydelta1.sph is v.4 2 (from Sphinx 2.x), binary is v.50; NOT SERVING precaching index 'delta1' WARNING: index 'delta1': prealloc: /var/piler/sphinx/delta1.sph is v.42 (from Sp hinx 2.x), binary is v.50; NOT SERVING precaching index 'tag1' WARNING: index 'tag1': prealloc: /var/piler/sphinx/tag1.sph is v.42 (from Sphinx 2.x), binary is v.50; NOT SERVING precaching index 'note1' WARNING: index 'note1': prealloc: /var/piler/sphinx/note1.sph is v.42 (from Sphi nx 2.x), binary is v.50; NOT SERVING FATAL: no valid indexes to serve shutdown complete root@archiv:~# /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . .
-
repo owner OK. Then stop searchd, run rm -f /var/piler/sphinx/* then indexer --all --config /etc/piler/sphinx.conf, and finally start searchd. It should start cleanly this time.
-
reporter I have done it but the error is still exisitng.
piler@archiv:~/sphinx$ /etc/init.d/rc.searchd start starting searchd . . . Sphinx 3.3.1 (commit b72d67b) Copyright (c) 2001-2020, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) no config file, using 'sphinxdata' folder... listening on all interfaces, port=9312 listening on all interfaces, port=9306 piler@archiv:~/sphinx$ /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . . piler@archiv:~/sphinx$ ls note1.spa note1.sph note1.spk tag1.spa tag1.sph tag1.spk note1.spd note1.spi note1.spp tag1.spd tag1.spi tag1.spp note1.spe note1.spj sphinxdata tag1.spe tag1.spj
If I run the main and daily crontab manuel I get these output:
root@archiv:~# /usr/local/libexec/piler/indexer.delta.sh ERROR: index 'delta1': reserved field name 'from'. FATAL: failed to merge index 'delta1' into index 'dailydelta1': failed to open /var/piler/sphinx/dailydelta1.sph: No such file or directory root@archiv:~# /usr/local/libexec/piler/indexer.main.sh FATAL: failed to merge index 'dailydelta1' into index 'main1': failed to open /var/piler/sphinx/main1.sph: No such file or directory
-
repo owner Something interferes with the piler shipped sphinx config. The “sphinxdata” dir should not exist. After you ran the commands I showed above, have you checked that all required files are present in /var/piler/sphinx? You need to find what else is running that messed with piler sphinx files.
-
reporter Other question but what is when I get the following output in piler-1.3.9? What is to do?
root@archiv:~# /etc/init.d/rc.searchd stop stopping searchd root@archiv:~# /etc/init.d/rc.piler stop stopping piler stopping piler-smtp root@archiv:~# /etc/init.d/rc.searchd start starting searchd . . . Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/piler/sphinx.conf'... listening on 127.0.0.1:9312 listening on 127.0.0.1:9306 precaching index 'main1' WARNING: index 'main1': preload: failed to open /var/piler/sphinx/main1.sph: No such file or directory; NOT SERVING precaching index 'main2' WARNING: index 'main2': preload: failed to open /var/piler/sphinx/main2.sph: No such file or directory; NOT SERVING precaching index 'main3' WARNING: index 'main3': preload: failed to open /var/piler/sphinx/main3.sph: No such file or directory; NOT SERVING precaching index 'main4' WARNING: index 'main4': preload: failed to open /var/piler/sphinx/main4.sph: No such file or directory; NOT SERVING precaching index 'dailydelta1' precaching index 'delta1' precaching index 'tag1' precaching index 'note1' precached 8 indexes in 0.031 sec root@archiv:~# /etc/init.d/rc.piler start starting piler-smtp . . . starting piler . . .
root@archiv:~# su - piler piler@archiv:~$ indexer -all Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) ERROR: malformed or unknown option near '-all'.
-
repo owner I think we are back to square
#1. You don’t have the main index files. Also pay more attention to what I wrote above: -all vs. --all -
repo owner - changed status to closed
No news is good news.
- Log in to comment
It looks like an issue with the sphinx database. Check if you have main*.* files in /var/piler/sphinx and they are owned by piler:piler