1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

problem with sphinx

Issue #1212 resolved
Adrian Osica created an issue

Hello, I have a problem with indexing emails. Sphinx can't see the emails?

‌

piler@piler1:~$  indexer --all --config /etc/piler/sphinx.conf --rotate
Sphinx 3.4.1 (commit efbcc65)
Copyright (c) 2001-2021, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
indexing index 'main1'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
indexing index 'main2'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
indexing index 'main3'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
indexing index 'main4'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
indexing index 'dailydelta1'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
indexing index 'delta1'...
collected 6 docs, 0.0 MB
sorted 0.0 Mhits, 100.0% done
total 6 docs, 18.9 Kb
total 0.0 sec, 3.648 Mb/sec, 1158 docs/sec
indexing index 'tag1'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
indexing index 'note1'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
rotating indices: successfully sent SIGHUP to searchd (pid=645).

piler@piler1:~$ /usr/local/bin/indexer --config /etc/piler/sphinx.conf  tag1 --rotate
Sphinx 3.4.1 (commit efbcc65)
Copyright (c) 2001-2021, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
indexing index 'tag1'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
rotating indices: successfully sent SIGHUP to searchd (pid=645).

piler@piler1:~$ /usr/local/bin/indexer --config /etc/piler/sphinx.conf  note1 --rotate
Sphinx 3.4.1 (commit efbcc65)
Copyright (c) 2001-2021, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/piler/sphinx.conf'...
indexing index 'note1'...
collected 0 docs, 0.0 MB
total 0 docs, 0.0 Kb
total 0.0 sec, 0.0 Kb/sec, 0 docs/sec
rotating indices: successfully sent SIGHUP to searchd (pid=645).

‌

So webui is empty 😞

‌

Comments (9)

  1. Janos SUTO repo owner

    Check if the indexer.delta.sh script is running from piler’s crontab. Also verify that the sphinx index files (delta1.* and dailydelta1.*) are updated regularly.

  2. Adrian Osica reporter

    Thank you for reply, it seems to be working fine, but search still not working. Log below

    ‌

    Nov 18 23:01:44 piler1 piler-smtp[304]: info: cannot open /etc/piler/smtp.acl, piler-smtp accepts smtp connections from everywhere
Nov 18 23:01:44 piler1 piler-smtp[304]: reloaded config: /etc/piler/piler.conf
Nov 18 23:01:44 piler1 piler-smtp[304]: piler-smtp 1.3.11, build 1001 starting
Nov 18 23:01:44 piler1 piler[320]: reloaded config: /etc/piler/piler.conf
Nov 18 23:01:44 piler1 piler[320]: piler 1.3.11, build 1001 starting
Nov 18 23:05:01 piler1 piler: INDEXER INFO: indexing delta1 started
Nov 18 23:05:01 piler1 piler: INDEXER INFO: indexing delta1 finished
Nov 18 23:05:06 piler1 piler: INDEXER INFO: merging delta to dailydelta started
Nov 18 23:05:07 piler1 piler: INDEXER INFO: merging delta to dailydelta finished
Nov 18 23:16:00 piler1 piler-webui[340]: username=auditor@local, event='logged in', ipaddr=10.5.4.8
Nov 18 23:16:01 piler1 piler-webui[339]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE        MATCH('') ORDER BY `sent` DESC LIMIT 0,20 OPTION max_matches=1000' in 0.00 s, 0 hits, 0 total found

    ‌

    -rw-r--r-- 1 piler piler    0 Nov 18 23:05 dailydelta1.spa
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 dailydelta1.spd
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 dailydelta1.spe
-rw-r--r-- 1 piler piler 1257 Nov 18 23:05 dailydelta1.sph
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 dailydelta1.spi
-rw-r--r-- 1 piler piler    0 Nov 18 23:05 dailydelta1.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:05 dailydelta1.spl
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 dailydelta1.spp
-rw-r--r-- 1 piler piler    0 Nov 18 23:05 delta1.spa
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 delta1.spd
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 delta1.spe
-rw-r--r-- 1 piler piler 1257 Nov 18 23:05 delta1.sph
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 delta1.spi
-rw-r--r-- 1 piler piler   47 Nov 18 23:05 delta1.spj
-rw-r--r-- 1 piler piler    0 Nov 18 23:05 delta1.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:05 delta1.spl
-rw-r--r-- 1 piler piler    1 Nov 18 23:05 delta1.spp
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main1.spa
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main1.spd
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main1.spe
-rw-r--r-- 1 piler piler 1257 Nov 18 22:48 main1.sph
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main1.spi
-rw-r--r-- 1 piler piler    8 Nov 18 22:48 main1.spj
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main1.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:01 main1.spl
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main1.spp
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main2.spa
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main2.spd
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main2.spe
-rw-r--r-- 1 piler piler 1257 Nov 18 22:48 main2.sph
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main2.spi
-rw-r--r-- 1 piler piler    8 Nov 18 22:48 main2.spj
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main2.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:01 main2.spl
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main2.spp
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main3.spa
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main3.spd
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main3.spe
-rw-r--r-- 1 piler piler 1257 Nov 18 22:48 main3.sph
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main3.spi
-rw-r--r-- 1 piler piler    8 Nov 18 22:48 main3.spj
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main3.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:01 main3.spl
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main3.spp
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main4.spa
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main4.spd
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main4.spe
-rw-r--r-- 1 piler piler 1257 Nov 18 22:48 main4.sph
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main4.spi
-rw-r--r-- 1 piler piler    8 Nov 18 22:48 main4.spj
-rw-r--r-- 1 piler piler    0 Nov 18 22:48 main4.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:01 main4.spl
-rw-r--r-- 1 piler piler    1 Nov 18 22:48 main4.spp
-rw-r--r-- 1 piler piler    0 Nov 18 23:15 note1.spa
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 note1.spd
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 note1.spe
-rw-r--r-- 1 piler piler  921 Nov 18 23:15 note1.sph
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 note1.spi
-rw-r--r-- 1 piler piler    8 Nov 18 23:15 note1.spj
-rw-r--r-- 1 piler piler    0 Nov 18 23:15 note1.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:15 note1.spl
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 note1.spp
-rw-r--r-- 1 piler piler    0 Nov 18 23:15 tag1.spa
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 tag1.spd
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 tag1.spe
-rw-r--r-- 1 piler piler  920 Nov 18 23:15 tag1.sph
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 tag1.spi
-rw-r--r-- 1 piler piler    8 Nov 18 23:15 tag1.spj
-rw-r--r-- 1 piler piler    0 Nov 18 23:15 tag1.spk
-rw-r--r-- 1 piler piler    0 Nov 18 23:15 tag1.spl
-rw-r--r-- 1 piler piler    1 Nov 18 23:15 tag1.spp

    ‌

  3. Janos SUTO repo owner

    It seems that the index files are indeed updated from piler’s crontab, however, they are all empty.

    Try the following. Send an email to the archive. Verify that the sph_index table is populated with the data to be indexed. Then run the delta indexer as user piler. Finally connect to sphinx with the mysql cli utility, and check its data, ie.

    mysql -h 127.0.0.1 -P9306
select * from dailydelta1,delta1;

    You should to see some rows returned.

  4. Adrian Osica reporter

    Ok, i send one mail and returning one row. After run delta indexer sph_index table is empty

    MySQL [(none)]> select * from dailydelta1,delta1;
+------+------------+------+-------------+
| id   | sent       | size | attachments |
+------+------------+------+-------------+
|   60 | 1637511233 | 2338 |           0 |
+------+------------+------+-------------+
1 row in set (0.00 sec)

    ‌

    log when searching

    Nov 21 17:37:14 piler1 piler-webui[342]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE        MATCH('') ORDER BY `sent` DESC LIMIT 0,20 OPTION max_matches=1000' in 0.00 s, 0 hits, 60 total found

    ‌

    -rw-r--r-- 1 piler piler    100 Nov 21 17:35 dailydelta1.spa
-rw-r--r-- 1 piler piler     62 Nov 21 17:35 dailydelta1.spd
-rw-r--r-- 1 piler piler      1 Nov 21 17:35 dailydelta1.spe
-rw-r--r-- 1 piler piler   1257 Nov 21 17:35 dailydelta1.sph
-rw-r--r-- 1 piler piler    169 Nov 21 17:35 dailydelta1.spi
-rw-r--r-- 1 piler piler      4 Nov 21 17:35 dailydelta1.spk
-rw-r--r-- 1 piler piler      0 Nov 21 17:35 dailydelta1.spl
-rw-r--r-- 1 piler piler     19 Nov 21 17:35 dailydelta1.spp
-rw-r--r-- 1 piler piler      0 Nov 21 17:35 delta1.spa
-rw-r--r-- 1 piler piler      1 Nov 21 17:35 delta1.spd
-rw-r--r-- 1 piler piler      1 Nov 21 17:35 delta1.spe
-rw-r--r-- 1 piler piler   1257 Nov 21 17:35 delta1.sph
-rw-r--r-- 1 piler piler      1 Nov 21 17:35 delta1.spi
-rw-r--r-- 1 piler piler     47 Nov 21 17:35 delta1.spj
-rw-r--r-- 1 piler piler      0 Nov 21 17:35 delta1.spk
-rw-r--r-- 1 piler piler      0 Nov 21 17:35 delta1.spl
-rw-r--r-- 1 piler piler      1 Nov 21 17:35 delta1.spp
-rw-r--r-- 1 piler piler   1260 Nov 21 02:30 main1.spa
-rw-r--r-- 1 piler piler  96793 Nov 21 02:30 main1.spd
-rw-r--r-- 1 piler piler      1 Nov 21 02:30 main1.spe
-rw-r--r-- 1 piler piler   1257 Nov 21 02:30 main1.sph
-rw-r--r-- 1 piler piler  57449 Nov 21 02:30 main1.spi
-rw-r--r-- 1 piler piler      8 Nov 21 02:30 main1.spk
-rw-r--r-- 1 piler piler      0 Nov 21 02:30 main1.spl
-rw-r--r-- 1 piler piler 165145 Nov 21 02:30 main1.spp
-rw-r--r-- 1 piler piler      0 Nov 19 11:45 main2.spa
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main2.spd
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main2.spe
-rw-r--r-- 1 piler piler   1257 Nov 19 11:45 main2.sph
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main2.spi
-rw-r--r-- 1 piler piler      8 Nov 19 11:45 main2.spj
-rw-r--r-- 1 piler piler      0 Nov 19 11:45 main2.spk
-rw-r--r-- 1 piler piler      0 Nov 19 12:48 main2.spl
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main2.spp
-rw-r--r-- 1 piler piler      0 Nov 19 11:45 main3.spa
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main3.spd
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main3.spe
-rw-r--r-- 1 piler piler   1257 Nov 19 11:45 main3.sph
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main3.spi
-rw-r--r-- 1 piler piler      8 Nov 19 11:45 main3.spj
-rw-r--r-- 1 piler piler      0 Nov 19 11:45 main3.spk
-rw-r--r-- 1 piler piler      0 Nov 19 12:48 main3.spl
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main3.spp
-rw-r--r-- 1 piler piler      0 Nov 19 11:45 main4.spa
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main4.spd
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main4.spe
-rw-r--r-- 1 piler piler   1257 Nov 19 11:45 main4.sph
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main4.spi
-rw-r--r-- 1 piler piler      8 Nov 19 11:45 main4.spj
-rw-r--r-- 1 piler piler      0 Nov 19 11:45 main4.spk
-rw-r--r-- 1 piler piler      0 Nov 19 12:48 main4.spl
-rw-r--r-- 1 piler piler      1 Nov 19 11:45 main4.spp
-rw-r--r-- 1 piler piler      0 Nov 21 17:45 note1.spa
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 note1.spd
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 note1.spe
-rw-r--r-- 1 piler piler    921 Nov 21 17:45 note1.sph
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 note1.spi
-rw-r--r-- 1 piler piler      8 Nov 21 17:45 note1.spj
-rw-r--r-- 1 piler piler      0 Nov 21 17:45 note1.spk
-rw-r--r-- 1 piler piler      0 Nov 21 17:45 note1.spl
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 note1.spp
-rw-r--r-- 1 piler piler      0 Nov 21 17:45 tag1.spa
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 tag1.spd
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 tag1.spe
-rw-r--r-- 1 piler piler    920 Nov 21 17:45 tag1.sph
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 tag1.spi
-rw-r--r-- 1 piler piler      8 Nov 21 17:45 tag1.spj
-rw-r--r-- 1 piler piler      0 Nov 21 17:45 tag1.spk
-rw-r--r-- 1 piler piler      0 Nov 21 17:45 tag1.spl
-rw-r--r-- 1 piler piler      1 Nov 21 17:45 tag1.spp

    ‌

  5. Janos SUTO repo owner

    Can you try sphinx 3.3.1? You need to re-initialize the sphinx db, then run reindex -a.

  6. Adrian Osica reporter

    Ok, now the record is duplicated and search still not working

    Nov 22 09:56:03 piler1 piler-webui[342]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE        MATCH('') ORDER BY `sent` DESC LIMIT 0,20 OPTION max_matches=1000' in 0.00 s, 0 hits, 120 total found

    ‌

  8. Adrian Osica reporter

    I deleted the database, all the files and installed everything from scratch this time with sphinx 3.3.1
    Now the search works correctly

  10. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!