1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

mails disappear on new Server-Clone

Issue #1304 closed
Philipp von Strobl-Albeg created an issue

Hi JSuto,

first let me say, i really like piler and use it for years now without any problems. Thanks for youor work there !

When i cloned my working piler server to do some tests i left out all “bigger” folders and setup up with new domain/tenant like in post intall manual mentioned. Now mails get in to the archive (“stored”) and found for some time in the webui, but - probably because of one of the cron-jobs - suddently disapprear.

Using piler-14.1 newly compiled.

When i check piler-db table metadat, i get 209 rows (seems to be all mails in there).

Reindex and indexer bring 10 mails back:

reindex -c /usr/local/etc/piler/piler.conf -a -p
processed: 209 [100%]
put 209 messages to sph_index table for reindexing

‌

su piler -c "indexer --all --config /usr/local/etc/piler/sphinx.conf --rotate"
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com))

using config file '/usr/local/etc/piler/sphinx.conf'...
indexing index 'main1'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.004 sec, 0 bytes/sec, 0.00 docs/sec
indexing index 'main2'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
indexing index 'main3'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
indexing index 'main4'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
indexing index 'dailydelta1'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
indexing index 'delta1'...
collected 209 docs, 0.9 MB
sorted 0.2 Mhits, 100.0% done
total 209 docs, 863266 bytes
total 0.113 sec, 7609286 bytes/sec, 1842.23 docs/sec
indexing index 'tag1'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
indexing index 'note1'...
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.002 sec, 0 bytes/sec, 0.00 docs/sec
total 11 reads, 0.000 sec, 53.6 kb/call avg, 0.0 msec/call avg
total 61 writes, 0.003 sec, 18.1 kb/call avg, 0.0 msec/call avg
rotating indices: successfully sent SIGHUP to searchd (pid=9702).

‌

For Sphinx i just see less:

MySQL [(none)]> show tables;
+-------------+-------+
| Index | Type |
+-------------+-------+
| dailydelta1 | local |
| delta1 | local |
| main1 | local |
| main2 | local |
| main3 | local |
| main4 | local |
| note1 | local |
| tag1 | local |
+-------------+-------+
8 rows in set (0,000 sec)

‌

MySQL [(none)]> select * from main1,delta1;
+------+------------+-------+-------------+
| id | sent | size | attachments |
+------+------------+-------+-------------+
| 1 | 1688482023 | 719 | 0 |
| 2 | 1688482023 | 13294 | 0 |
| 3 | 1688482023 | 671 | 0 |
| 4 | 1688482023 | 30374 | 0 |
| 5 | 1688482023 | 3186 | 0 |
| 6 | 1688482023 | 2552 | 0 |
| 7 | 1688482023 | 308 | 0 |
| 8 | 1688482023 | 353 | 0 |
| 9 | 1688482023 | 7010 | 0 |
| 10 | 1688482023 | 3626 | 0 |
| 11 | 1688482023 | 972 | 0 |
| 12 | 1688482023 | 815 | 0 |
| 13 | 1688482023 | 2200 | 0 |
| 14 | 1688482023 | 1205 | 0 |
| 15 | 1688482023 | 338 | 0 |
| 16 | 1688482023 | 4746 | 0 |
| 17 | 1688482023 | 6740 | 0 |
| 18 | 1688482023 | 4971 | 0 |
| 19 | 1688482023 | 2483 | 0 |
| 20 | 1688482023 | 239 | 0 |
+------+------------+-------+-------------+
20 rows in set (0,001 sec)

‌

MySQL [(none)]> select * from dailydelta1;
Empty set (0,000 sec)

Sphinx Version 2.2.11-r5(13:36:10 02.07.2023)( Use-Flags: id64 mariadb syslog)

Sadly i’m using a gentoo and standalone so the manticore option seems not to be handy as an alternative.

Could you please give me a hint / point the rioght direction ?

‌

‌

Comments (15)

  2. Janos SUTO repo owner

    indexer --all should be run only when you create the index files. For adding new documents, ie. emails to the index you should (read: must) use the indexer.*.sh scripts shipped by piler. I’m not sure what you meant by “skipping the bigger folders”, but see to it, that the cronjobs are working.

  4. Philipp von Strobl-Albeg reporter

    Hi,

    thanks for the fast feedback - very appreciate it !

    I think here are eventually two problems. Lets start with the initial one:

    To start from “scratch” (because it is only a test system) i did:

    /etc/init.d/rc.searchd stop

    reindex -c /usr/local/etc/piler/piler.conf -a -p
    processed: 210 [100%]
    put 210 messages to sph_index table for reindexing

    su piler -c "indexer --all --config /usr/local/etc/piler/sphinx.conf"
    Sphinx 2.2.11-id64-release (95ae9a6)
    Copyright (c) 2001-2016, Andrew Aksyonoff
    Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

    using config file '/usr/local/etc/piler/sphinx.conf'...
    indexing index 'main1'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.004 sec, 0 bytes/sec, 0.00 docs/sec
    indexing index 'main2'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
    indexing index 'main3'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
    indexing index 'main4'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
    indexing index 'dailydelta1'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
    indexing index 'delta1'...
    collected 210 docs, 0.9 MB
    sorted 0.2 Mhits, 100.0% done
    total 210 docs, 872916 bytes
    total 0.108 sec, 8051022 bytes/sec, 1936.85 docs/sec
    indexing index 'tag1'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
    indexing index 'note1'...
    collected 0 docs, 0.0 MB
    total 0 docs, 0 bytes
    total 0.001 sec, 0 bytes/sec, 0.00 docs/sec
    total 11 reads, 0.000 sec, 54.9 kb/call avg, 0.0 msec/call avg
    total 61 writes, 0.001 sec, 18.5 kb/call avg, 0.0 msec/call avg

    /etc/init.d/rc.searchd start
    starting searchd . . .
    Sphinx 2.2.11-id64-release (95ae9a6)
    Copyright (c) 2001-2016, Andrew Aksyonoff
    Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

    using config file '/usr/local/etc/piler/sphinx.conf'...
    listening on 127.0.0.1:9312
    listening on 127.0.0.1:9306
    precaching index 'main1'
    precaching index 'main2'
    precaching index 'main3'
    precaching index 'main4'
    precaching index 'dailydelta1'
    precaching index 'delta1'
    precaching index 'tag1'
    precaching index 'note1'
    precached 8 indexes in 0.012 sec

    ‌

    After that i login with my user in webui and only see 10 messages.

  5. Janos SUTO repo owner

    Show me the logged sphinx query from the mail log. Also connect to the 9306 port with mysql client, and check how many documents are in each index.

  6. Philipp von Strobl-Albeg reporter

    Hope this is the right one:

    sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' ( (@from psXfateasyXcom) | (@to psXfateasyXcom) ) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=1000' in 0.00 s, 10 hits, 10 total found

    ‌

    MySQL [(none)]> count * from dailydelta1;
    ERROR 1064 (42000): sphinxql: syntax error, unexpected COUNT near 'count * from dailydelta1'
    MySQL [(none)]> select * from dailydelta1;
    +------+------------+-------+-------------+
    | id | sent | size | attachments |
    +------+------------+-------+-------------+
    | 211 | 1688655600 | 97955 | 0 |
    +------+------------+-------+-------------+
    1 row in set (0,001 sec)

    MySQL [(none)]> select * from delta1;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from delta1;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from main1;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from main2;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from main3;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from main4;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from note1;
    Empty set (0,000 sec)

    MySQL [(none)]> select * from tag1;
    Empty set (0,000 sec)

    ‌

    indexer.*.sh runs a recommended by the post install:

    PILERSTART

    5,35 * * * * piler /usr/local/libexec/piler/indexer.delta.sh
    30 2 * * * piler /usr/local/libexec/piler/indexer.main.sh
    */15 * * * * piler /usr/bin/indexer --config /usr/local/etc/piler/sphinx.conf --quiet tag1 --rotate
    */15 * * * * piler /usr/bin/indexer --config /usr/local/etc/piler/sphinx.conf --quiet note1 --rotate

    ‌

    so i tried to run indexer.main.sh command by hand to process dailydelta:

    su piler -c "indexer --config /usr/local/etc/piler/sphinx.conf dailydelta1 --rotate"
    Sphinx 2.2.11-id64-release (95ae9a6)
    Copyright (c) 2001-2016, Andrew Aksyonoff
    Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

    using config file '/usr/local/etc/piler/sphinx.conf'...
    indexing index 'dailydelta1'...
    ERROR: index 'dailydelta1': failed to create /var/piler/sphinx/dailydelta1.tmp.tmps: Permission denied.
    total 0 docs, 0 bytes
    total 0.003 sec, 0 bytes/sec, 0.00 docs/sec
    total 0 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
    total 0 writes, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg

    So i checked the permission:
    ls -l /var/piler/
    insgesamt 40
    drwx--x--x 2 piler piler 4096 1. Jul 09:18 error
    drwx------ 2 piler piler 4096 1. Jul 09:18 imap
    drwx------ 2 piler piler 4096 3. Jul 18:24 manticore
    drwx------ 2 piler piler 12288 6. Jul 18:03 sphinx
    drwxr-xr-x 2 piler piler 4096 1. Jul 09:22 stat
    drwx------ 3 piler piler 4096 22. Jun 07:56 store
    drwx--x--x 12 piler piler 4096 6. Jul 17:00 tmp
    drwxr-xr-x 15 root root 4096 3. Jul 18:26 www

    ‌

    and Checkced against the other server and the permissions are the same. Are they wrong ?

  7. Janos SUTO repo owner

    No, the permissions on the index files are not the same, otherwise you shouldn’t get the permission denied issue.

    The indexes contain 210 documents, a user is able to see some of them. Try using an auditor account which should be able to see all of them.

  8. Philipp von Strobl-Albeg reporter

    Hi,

    i did the check as auditor but no mails are found.

    After that i emptied the db and import 213 mails. When i no check the auditor and run

    su piler -c "indexer --all -c /usr/local/etc/piler/sphinx.conf --rotate"

    Then every 213 mails were found.
    After that i run
    su piler -c /usr/local/libexec/piler/indexer.main.sh

    • set -o nounset
    • set -o errexit
    • set -o pipefail
    • MAINTMPFILE=/var/run/piler/main.indexer.tmp
    • CONFIG_FILE=/usr/local/etc/piler/sphinx.conf
    • INDEXDIR=sphinx
    • PRIORITY=mail.info
    • TOUCHFILE=/var/piler/stat/indexer
    • MAIN_INDEX=main1
    • [[ -f /usr/local/etc/piler/MANTICORE ]]
    • export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    • PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    • [[ -f /var/run/piler/main.indexer.tmp ]]
    • date
    • touch /var/piler/stat/indexer
    • trap finish EXIT
    • logger -p mail.info
    • echo 'INDEXER INFO: merging to main started'
    • indexer --config /usr/local/etc/piler/sphinx.conf --quiet --merge main1 dailydelta1 --merge-dst-range deleted 0 0 --rotate
    • logger -p mail.info
    • echo 'INDEXER INFO: merging to main finished'
    • sleep 5
    • logger -p mail.info
    • echo 'INDEXER INFO: resetting daily delta started'
    • indexer --config /usr/local/etc/piler/sphinx.conf --quiet dailydelta1 --rotate
    • logger -p mail.info
    • echo 'INDEXER INFO: resetting daily delta finished'
    • sum=0
    • read -r a b
      ++ find /var/piler/sphinx/ -type f -name 'main*.sp[dp]' -printf '%TY%Tm%Td %s\n'
    • sum=1
    • read -r a b
    • sum=2
    • read -r a b
    • sum=3
    • read -r a b
    • sum=4
    • read -r a b
    • sum=5
    • read -r a b
    • sum=6
    • read -r a b
    • sum=7
    • read -r a b
    • sum=8
    • read -r a b
    • printf %d 8
    • finish
    • rm -f /var/run/piler/main.indexer.tmp

    auditor still has 213 messages

    Then i run

    su piler -c /usr/local/libexec/piler/indexer.delta.sh

    • export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    • PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    • MAINTMPFILE=/var/run/piler/main.indexer.tmp
    • DELTATMPFILE=/var/run/piler/delta.indexer.tmp
    • INDEXER='indexer --config /usr/local/etc/piler/sphinx.conf'
    • PRIORITY=mail.info
    • TOUCHFILE=/var/piler/stat/indexer
    • CONFIG_FILE=sphinx.conf
    • [[ -f /usr/local/etc/piler/MANTICORE ]]
    • INDEXER='indexer --config /usr/local/etc/piler/sphinx.conf'
    • [[ -f /var/run/piler/main.indexer.tmp ]]
    • [[ -f /var/run/piler/delta.indexer.tmp ]]
    • date
    • touch /var/piler/stat/indexer
    • trap finish EXIT
    • logger -p mail.info
    • echo 'INDEXER INFO: indexing delta1 started'
    • indexer --config /usr/local/etc/piler/sphinx.conf --quiet delta1 --rotate
    • logger -p mail.info
    • echo 'INDEXER INFO: indexing delta1 finished'
    • sleep 5
    • logger -p mail.info
    • echo 'INDEXER INFO: merging delta to dailydelta started'
    • indexer --config /usr/local/etc/piler/sphinx.conf --quiet --merge dailydelta1 delta1 --merge-dst-range deleted 0 0 --rotate
    • logger -p mail.info
    • echo 'INDEXER INFO: merging delta to dailydelta finished'
    • finish
    • rm -f /var/run/piler/delta.indexer.tmp

    And then all mails were disappeared.

  10. Philipp von Strobl-Albeg reporter

    Hi Janos,

    sorry for the missunderstanding.

    As you mentioned one of the files /var/piler/sphinx/dailydelta1.tmp.tmps was root owned. So to be sure i followed the “How can I delete everything from the archive?” from https://www.mailpiler.org/wiki/faq and start from scratch (delete sphinx/* and store/00/* )

    So permission now is resetted and the permissions for /var/piler is

    mabas ~ # ls -l /var/piler/
    insgesamt 40
    drwx--x--x 2 piler piler 4096 1. Jul 09:18 error
    drwx------ 2 piler piler 4096 1. Jul 09:18 imap
    drwx------ 2 piler piler 4096 3. Jul 18:24 manticore
    drwx------ 2 piler piler 12288 10. Jul 15:20 sphinx
    drwxr-xr-x 2 piler piler 4096 1. Jul 09:22 stat
    drwx------ 3 piler piler 4096 22. Jun 07:56 store
    drwx--x--x 12 piler piler 4096 10. Jul 17:00 tmp
    drwxr-xr-x 15 root root 4096 3. Jul 18:26 www

    ‌

    ls -l /var/piler/sphinx/
    insgesamt 292
    -rw------- 1 piler piler 8 10. Jul 15:20 binlog.001
    -rw------- 1 piler piler 0 10. Jul 14:48 binlog.lock
    -rw------- 1 piler piler 11 10. Jul 15:20 binlog.meta
    -rw-r--r-- 1 piler piler 100 10. Jul 15:20 dailydelta1.spa
    -rw-r--r-- 1 piler piler 5220 10. Jul 15:20 dailydelta1.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 dailydelta1.spe
    -rw-r--r-- 1 piler piler 1164 10. Jul 15:20 dailydelta1.sph
    -rw-r--r-- 1 piler piler 8815 10. Jul 15:20 dailydelta1.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 dailydelta1.spk
    -rw------- 1 piler piler 0 10. Jul 15:20 dailydelta1.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 dailydelta1.spm
    -rw-r--r-- 1 piler piler 2122 10. Jul 15:20 dailydelta1.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 dailydelta1.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 dailydelta1.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 dailydelta1.tmp.tmps
    -rw-r--r-- 1 piler piler 100 10. Jul 15:20 delta1.spa
    -rw-r--r-- 1 piler piler 5224 10. Jul 15:20 delta1.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 delta1.spe
    -rw-r--r-- 1 piler piler 1164 10. Jul 15:20 delta1.sph
    -rw-r--r-- 1 piler piler 8815 10. Jul 15:20 delta1.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 delta1.spk
    -rw------- 1 piler piler 0 10. Jul 15:20 delta1.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 delta1.spm
    -rw-r--r-- 1 piler piler 2122 10. Jul 15:20 delta1.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 delta1.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 delta1.tmps
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 main1.spa
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 main1.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 main1.spe
    -rw-r--r-- 1 piler piler 1164 10. Jul 15:20 main1.sph
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 main1.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 main1.spk
    -rw------- 1 piler piler 0 10. Jul 15:20 main1.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 15:20 main1.spm
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 main1.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 15:20 main1.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 main1.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main1.tmp.tmps
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main2.spa
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main2.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main2.spe
    -rw-r--r-- 1 piler piler 1164 10. Jul 14:52 main2.sph
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main2.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main2.spk
    -rw------- 1 piler piler 0 10. Jul 14:52 main2.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main2.spm
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main2.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main2.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 main2.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main2.tmp.tmps
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main3.spa
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main3.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main3.spe
    -rw-r--r-- 1 piler piler 1164 10. Jul 14:52 main3.sph
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main3.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main3.spk
    -rw------- 1 piler piler 0 10. Jul 14:52 main3.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main3.spm
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main3.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main3.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 main3.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main3.tmp.tmps
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main4.spa
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main4.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main4.spe
    -rw-r--r-- 1 piler piler 1164 10. Jul 14:52 main4.sph
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main4.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main4.spk
    -rw------- 1 piler piler 0 10. Jul 14:52 main4.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 main4.spm
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main4.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main4.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 main4.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 main4.tmp.tmps
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 note1.spa
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 note1.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 note1.spe
    -rw-r--r-- 1 piler piler 872 10. Jul 14:52 note1.sph
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 note1.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 note1.spk
    -rw------- 1 piler piler 0 10. Jul 14:52 note1.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 note1.spm
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 note1.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 note1.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 note1.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 note1.tmp.tmps
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 tag1.spa
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 tag1.spd
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 tag1.spe
    -rw-r--r-- 1 piler piler 871 10. Jul 14:52 tag1.sph
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 tag1.spi
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 tag1.spk
    -rw------- 1 piler piler 0 10. Jul 14:52 tag1.spl
    -rw-r--r-- 1 piler piler 0 10. Jul 14:52 tag1.spm
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 tag1.spp
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 tag1.sps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:48 tag1.tmps
    -rw-r--r-- 1 piler piler 1 10. Jul 14:52 tag1.tmp.tmps

    ‌

    ls -l /var/piler/store/00/
    insgesamt 4
    drwxr-x--- 149 piler piler 4096 10. Jul 15:12 64a

    ‌

    ls -l /usr/local//piler/
    -rw-r--r-- 1 root piler 379 3. Jul 18:24 /usr/local/etc/piler/config-site.dist.php
    -rw-r--r-- 1 root root 1616 28. Jun 23:25 /usr/local/etc/piler/config-site.php
    -rwxr-xr-x 1 root root 6949 4. Jul 16:40 /usr/local/etc/piler/manticore.conf
    -rwxr-xr-x 1 root piler 6966 3. Jul 18:24 /usr/local/etc/piler/manticore.conf.dist
    -rw-r----- 1 root piler 991 2. Jul 00:03 /usr/local/etc/piler/piler.conf
    -rw-r----- 1 root piler 1246 3. Jul 18:24 /usr/local/etc/piler/piler.conf.dist
    -rw------- 1 root root 5256 4. Sep 2016 /usr/local/etc/piler/piler_contrib.pem
    -rw-r----- 1 root piler 56 4. Jul 16:46 /usr/local/etc/piler/piler.key
    -r-xr-x--- 1 piler piler 5599 30. Jan 2022 /usr/local/etc/piler/piler_old.pem
    -r-xr-x--- 1 piler piler 3477 18. Jun 03:36 /usr/local/etc/piler/piler.pem
    -rwxr-xr-x 1 root root 8917 5. Jul 11:22 /usr/local/etc/piler/sphinx.conf
    -rwxr-xr-x 1 root piler 8823 3. Jul 18:24 /usr/local/etc/piler/sphinx.conf.dist
    -rw-r--r-- 1 root root 6337 24. Jun 12:25 /usr/local/etc/piler/sphinx.conf_old
    -rwxr-xr-x 1 root root 6759 3. Jul 18:24 /usr/local/libexec/piler/automated-search.php
    -rwxr-xr-x 1 root root 2942 3. Jul 18:24 /usr/local/libexec/piler/daily-report.php
    -rwxr-xr-x 1 root root 2688 3. Jul 18:24 /usr/local/libexec/piler/generate_stats.php
    -rwxr-xr-x 1 root root 2264 3. Jul 18:24 /usr/local/libexec/piler/get-token.py
    -rwxr-xr-x 1 root root 2504 3. Jul 18:24 /usr/local/libexec/piler/gmail-imap-import.php
    -rwxr-xr-x 1 root root 1202 3. Jul 18:24 /usr/local/libexec/piler/healthcheck.sh
    -rwxr-xr-x 1 root root 6855 3. Jul 18:24 /usr/local/libexec/piler/imapfetch.py
    -rwxr-xr-x 1 root root 225 3. Jul 18:24 /usr/local/libexec/piler/import.sh
    -rwxr-xr-x 1 root root 693 3. Jul 18:24 /usr/local/libexec/piler/indexer.attachment.sh
    -rwxr-xr-x 1 root root 1352 10. Jul 15:19 /usr/local/libexec/piler/indexer.delta.sh
    -rwxr-xr-x 1 root root 1412 10. Jul 15:19 /usr/local/libexec/piler/indexer.main.sh
    -rwxr-xr-x 1 root root 3113 3. Jul 18:24 /usr/local/libexec/piler/mailstat.php
    -rwxr-xr-x 1 root root 8945 3. Jul 18:24 /usr/local/libexec/piler/pilerpurge.py
    -rw-r--r-- 1 root root 468 3. Jul 18:24 /usr/local/libexec/piler/pilersearch.service
    -rw-r--r-- 1 root root 288 3. Jul 18:24 /usr/local/libexec/piler/piler.service
    -rw-r--r-- 1 root root 269 3. Jul 18:24 /usr/local/libexec/piler/piler-smtp.service
    -rwxr-xr-x 1 root root 11178 3. Jul 18:24 /usr/local/libexec/piler/postinstall.sh
    -rwxr-xr-x 1 root root 363 3. Jul 18:24 /usr/local/libexec/piler/purge.sh
    -rwxr-xr-x 1 root root 3703 3. Jul 18:24 /usr/local/libexec/piler/rtindex.py
    -rwxr-xr-x 1 root root 4283 3. Jul 18:24 /usr/local/libexec/piler/sign.php
    -rwxr-xr-x 1 root root 159 22. Mai 2020 /usr/local/libexec/piler/watch_sphinx_main_index.sh
    -rwxr-xr-x 1 root root 177 3. Jul 18:24 /usr/local/share/piler/db-mysql-root.sql.in
    -rwxr-xr-x 1 root root 15016 3. Jul 18:24 /usr/local/share/piler/db-mysql.sql
    -rw-r--r-- 1 piler piler 4096 3. Jul 18:24 /usr/local/share/piler/deduphelper
    -rw-r--r-- 1 root piler 711 3. Jul 18:24 /usr/local/share/piler/piler.cron

  13. Philipp von Strobl-Albeg reporter

    Hi,

    i’m really sorry, but bother you again, but the persmissions i send today were already in place when i run the tests yesterday. So sphinx still throw away their indexes when running index.delta.

    In the logs, i can only see messages like:
    [Mon Jul 10 15:20:51.709 2023] [32336] rotating index 'delta1': started
    [Mon Jul 10 15:20:51.711 2023] [32336] rotating index 'delta1': success
    [Mon Jul 10 15:20:56.724 2023] [32336] rotating index 'dailydelta1': started
    [Mon Jul 10 15:20:56.726 2023] [32336] rotating index 'dailydelta1': success

    ‌

  14. Janos SUTO repo owner

    How about if you initialize the archive to make it empty, then send some emails, and don’t touch it, let piler handle the rest?

  15. Philipp von Strobl-Albeg reporter

    Hi,

    it’s a little bit strange, but i recompiled sphinx with useflags “id64 mysql syslog” (mysql instead of mariadb like before) and just leave it alone. Now it seems to work.

    ‌

  16. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
closed
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!