1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

empty search result for a few users

Issue #1075 closed
John Testi created an issue

Hi there,

my system:

Piler: 1.3.6 build 998
Betriebssystem: Linux XXXXXX-Name-XXXX 3.10.0-1127.8.2.el7.x86_64 #1 SMP Thu May 7 19:30:37 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux

‌

emails are not displayed for a few users. the emails for the users are displayed in the auditor. Unfortunately, I don't know what to do at this point.
I have already compared the configuration of the users with the working ones. the settings look similar. do I have to start a service again or something similar? if any logs or the like are needed, just let me know.

Thanks in advance.😀

Comments (23)

  1. Janos SUTO repo owner

    Check out the sphinx query logged in the mail log for those few users. The gui applies a filter for non-auditor users. Check if the filter is applied properly.

  3. John Testi reporter

    I beg your pardon for the delay.
    Unfortunately i am still a bit unsure in the surroundings of linux.

    When I look in to \var\log\mail.log i cant find anything about the sphinx.
    The log \var\log\query.log is empty
    -rw-r--r--. 1 sphinxsearch sphinxsearch 0 Dec 6 2019 query.log

    have i searched in the wrong place?

  5. John Testi reporter

    this is what i got

    Jun 15 16:33:39 piler-servername piler-webui[1136]: username=problemadress1, event='logged in', ipaddr=xxx.xxx.xxx.xxx
    Jun 15 16:33:48 piler-servername piler-smtp[1013]: connected from xxx.xxx.xxx.xxx:42396 on fd=6 (active connections: 1)
    Jun 15 16:33:48 piler-servername piler-smtp[1013]: received: 2U17JIU3AYR6QP6H, from=othermail, size=79757, client=xxx.xxx.xxx.xxx, fd=6
    Jun 15 16:33:48 piler-servername piler-smtp[1013]: disconnected from xxx.xxx.xxx.xxx on fd=6 (0 active connections)
    Jun 15 16:33:48 piler-servername piler[87332]: 0/2U17JIU3AYR6QP6H: 400000005ee786d6310d8c6c006e83760a5c, size=79757/14296, attachments=2, reference=, message-id=612525672.464283.1592231616911@fpm01.tngpr, retention=2557, folder=0, delay=0.0178, status=stored
    Jun 15 16:34:05 piler-servername piler-webui[1139]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from problemadress1 | @to problemadress1) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=9000' in 0.01 s, 0 hits, 0 total found
    Jun 15 16:34:06 piler-servername piler-smtp[1013]: connected from xxx.xxx.xxx.xxx:47476 on fd=6 (active connections: 1)
    Jun 15 16:34:06 piler-servername piler-smtp[1013]: received: I0FS9ZJV98A2U3HV, from=othermail1, size=2218, client=xxx.xxx.xxx.xxx, fd=6
    Jun 15 16:34:06 piler-servername piler-smtp[1013]: disconnected from xxx.xxx.xxx.xxx on fd=6 (0 active connections)
    Jun 15 16:34:06 piler-servername piler[77364]: 1/I0FS9ZJV98A2U3HV: 400000005ee786e81849720400d104af307c, size=2218/1160, attachments=1, reference=, message-id=637438685.601757.1592231646132.JavaMail.djigzo@cipher, retention=2557, folder=0, delay=0.0146, status=stored
    Jun 15 16:34:06 piler-servername piler-webui[1136]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from problemadress1 | @to problemadress1) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=9000' in 0.00 s, 0 hits, 0 total found
    Jun 15 16:34:08 piler-servername piler-webui[4660]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from problemadress1 | @to problemadress1) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=9000' in 0.00 s, 0 hits, 0 total found
    Jun 15 16:34:09 piler-servername piler-webui[1141]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from problemadress1 | @to problemadress1) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=9000' in 0.00 s, 0 hits, 0 total found
    Jun 15 16:34:10 piler-servername piler-webui[1137]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from problemadress1 | @to problemadress1) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=9000' in 0.00 s, 0 hits, 0 total found
    Jun 15 16:34:10 piler-servername piler-webui[1140]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from problemadress1 | @to problemadress1) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=9000' in 0.00 s, 0 hits, 0 total found
    Jun 15 16:34:37 piler-servername piler-smtp[1013]: connected from xxx.xxx.xxx.xxx:42458 on fd=6 (active connections: 1)
    Jun 15 16:34:37 piler-servername piler-smtp[1013]: received: 5DDOCSRP6HYE1ZRC, from=othermail2, size=41789, client=xxx.xxx.xxx.xxx, fd=6
    Jun 15 16:34:37 piler-servername piler-smtp[1013]: disconnected from xxx.xxx.xxx.xxx on fd=6 (0 active connections)
    Jun 15 16:34:38 piler-servername piler[89712]: error: helper: execl
    Jun 15 16:34:38 piler-servername piler[77364]: 1/5DDOCSRP6HYE1ZRC: 400000005ee787081a63b2fc003848b05127, size=41789/25064, attachments=2, reference=, message-id=274195234.705833.1592231674520@app03, retention=2557, folder=0, delay=0.0248, status=stored
    Jun 15 16:34:51 piler-servername piler-smtp[1013]: connected from xxx.xxx.xxx.xxx:42472 on fd=6 (active connections: 1)
    Jun 15 16:34:51 piler-servername piler-smtp[1013]: received: 4C9U9NFUQWXVGJDF, from=othermail2, size=12078, client=xxx.xxx.xxx.xxx, fd=6
    Jun 15 16:34:51 piler-servername piler-smtp[1013]: disconnected from xxx.xxx.xxx.xxx on fd=6 (0 active connections)

  6. John Testi reporter

    if i only run “grep sphinx /var/log/mail.log” dont get any out put. -.-

    is this bad?

  8. John Testi reporter

    Yes the auditor can see the mails from and for “problemadress1”

    This works fine.

    Is there a task witch isnt startet yet or can i get you some orther logs?

  9. Janos SUTO repo owner

    OK, then as auditor click on the message, check its mail headers, and verify that “problemadress1” is on either From, To or Cc fields.

  11. Janos SUTO repo owner

    OK< one more thing to check. As auditor click on the email, then in the lower preview pane hover over the download link, and notice the id.

    Then go to piler mysql database, and execute the following query: select * from v_messages where id=123; // substitute the actual id.

  12. John Testi reporter

    MariaDB [piler]> select * from v_messages where id=6956942\G;
    *************************** 1. row ***************************
    id: 6956942
    piler_id: 400000005ef08c3235487ae400e886592d96
    from: problem@adresse
    fromdomain: xxxx.de
    to: sender@toproblem.de
    todomain: domain.de
    subject: Titel_92017300164450.txt
    size: 1621
    direction: 2
    sent: 1592822823
    retained: 1813747624
    arrived: 1592822824
    digest: 7f266323ad8caa5552e2e325fc41eb87ee381f9ea517003cc0240885e3865bec
    bodydigest: 3941a53f150ef927873c4a88089502076b8c1e3f0c0cd610cf9a9c5d1e7afdc6
    deleted: 0
    1 row in set (0.01 sec)

    ERROR: No query specified

  15. Janos SUTO repo owner

    Not yet. One more thing to try: reindex this specific email, ie. reindex -f 6956942 -t 6956942, then let the delta indexer run. And finally check the search again.

  17. John Testi reporter

    -sh-4.2$ reindex -f 6956942 -t 6956942
    put 1 messages to sph_index table for reindexing -sh-4.2$ /usr/local/libexec/piler/indexer.delta.sh
    /usr/local/libexec/piler/indexer.delta.sh: line 26: indexer: command not found
    /usr/local/libexec/piler/indexer.delta.sh: line 34: indexer: command not found

    ‌

  19. John Testi reporter

    I confirm

    ‌

    root@piler:~ # ls -lah $(which indexer) -rwxrwxr-x. 1 root root 13M Oct 16 2018 /opt/sphinx/bin/indexer* root@piler:~ # ps auxf | grep sphinx
    piler 97853 0.0 0.0 113284 1304 ? Ss 13:15 0:00 | _ /bin/sh -c /opt/sphinx/bin/indexer --quiet delta1 --rotate --config /usr/local/etc/piler/sphinx.conf && sleep 2 && /opt/sphinx/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate --config /usr/local/etc/piler/sphinx.conf
    piler 97879 99.5 27.0 752296 512428 ? R 13:15 1:48 | _ /opt/sphinx/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate --config /usr/local/etc/piler/sphinx.conf
    root 98133 0.0 0.0 112812 972 pts/0 S+ 13:16 0:00 _ grep --color=auto sphinx
    piler 3258 0.0 0.0 24180 92 ? S Jun17 0:00 /usr/bin/searchd --config /etc/piler/sphinx.conf
    piler 3259 0.3 25.3 805632 480408 ? Rl Jun17 52:10 _ /usr/bin/searchd --config /etc/piler/sphinx.conf

  21. John Testi reporter

    root@piler:~ # ls -lah $(which indexer) -rwxrwxr-x. 1 root root 13M Oct 16 2018 /opt/sphinx/bin/indexer* root@piler:~ # ps auxf | grep sphinx

    piler 97853 0.0 0.0 113284 1304 ? Ss 13:15 0:00 | _ /bin/sh -c /opt/sphinx/bin/indexer --quiet delta1 --rotate --config /usr/local/etc/piler/sphinx.conf && sleep 2 && /opt/sphinx/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate --config /usr/local/etc/piler/sphinx.conf

    piler 97879 99.5 27.0 752296 512428 ? R 13:15 1:48 | _ /opt/sphinx/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate --config /usr/local/etc/piler/sphinx.conf

    root 98133 0.0 0.0 112812 972 pts/0 S+ 13:16 0:00 _ grep --color=auto sphinx

    piler 3258 0.0 0.0 24180 92 ? S Jun17 0:00 /usr/bin/searchd --config /etc/piler/sphinx.conf

    piler 3259 0.3 25.3 805632 480408 ? Rl Jun17 52:10 _ /usr/bin/searchd --config /etc/piler/sphinx.conf

  22. Janos SUTO repo owner

    Only slightly better. However, I notice that searchd uses /etc/piler/sphinx.conf, but indexer uses /usr/local/etc/piler/sphinx.conf.

    Also, I’m confused when running indexer.delta.sh, can it execute the indexer binary? This copy-pasted stuff seems to be out of context, meaningless. I really doubt that the main and delta indexer runs at the same time.

  24. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
closed
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!