1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

No emails to be found as auditor (comp. issue #463)

Issue #493 resolved
Philip Deubner created an issue

As the issue #463 was already closed, I'll open another issue on this.

I use piler 1.1.1.0 on a Debian 7.8 VM with a fresh installation. After the installation, I added some mail to get to know piler (using the pilerimport tool). The message count shows up in the admin use statistics, but no auditor is able to see any mail. When searching (as an auditor) for mails (giving no restrictions) I always receive an empty search result in the webui (Empty search result. Try adding the wildcard character() after a word snippet, eg. duplic to find "duplicate", "duplicated", etc.). This is how syslog reconizes my attempts to search for any email:

Feb  1 16:32:01 archiv piler-webui[32086]: username=philip@local, event='logged in'
Feb  1 16:32:07 archiv piler-webui[32086]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE        MATCH('') ORDER BY `sent` DESC LIMIT 0,1000 OPTION max_matches=1000' in 0.00 s, 0 hits, 0 total found

When checking the audit section, I am able to see at least the search attempts and the logins, being representes in the syslog like

Feb  1 16:29:13 archiv piler-webui[32090]: audit query: 'SELECT * FROM audit  ORDER BY `ts` DESC LIMIT 0,20', param: '' in 0.00 s, 20 hits
Feb  1 16:29:14 archiv piler-webui[32090]: audit query: 'SELECT * FROM audit  ORDER BY `ts` DESC LIMIT 0,20', param: '' in 0.00 s, 20 hits

Furthermore, I tried to get rid of these problems. To do so, I tried to use reindex -a. This showed several failure messages like

failed to add to sph_index table: 7051.eml
failed to add to sph_index table: 7052.eml
failed to add to sph_index table: 7053.eml
failed to add to sph_index table: 7054.eml
failed to add to sph_index table: 7055.eml
failed to add to sph_index table: 7056.eml
failed to add to sph_index table: 7057.eml
failed to add to sph_index table: 7058.eml
failed to add to sph_index table: 7059.eml
failed to add to sph_index table: 7060.eml
failed to add to sph_index table: 7061.eml
put 0 messages to sph_index table for reindexing

My piler database looks like:

+-------------------+
| Tables_in_piler   |
+-------------------+
| archiving_rule    |
| attachment        |
| audit             |
| autosearch        |
| counter           |
| counter_stats     |
| customer_settings |
| domain            |
| domain_user       |
| email             |
| email_groups      |
| folder            |
| folder_extra      |
| folder_message    |
| folder_user       |
| google            |
| google_imap       |
| group             |
| group_email       |
| group_user        |
| import            |
| ldap              |
| metadata          |
| note              |
| online            |
| option            |
| rcpt              |
| remote            |
| retention_rule    |
| search            |
| sph_counter       |
| sph_index         |
| tag               |
| user              |
| user_settings     |
| v_attachment      |
| v_messages        |
+-------------------+
37 rows in set (0.00 sec)

The path to sphinx was changed to /opt/piler/sphinx, the ls -la for this directory shows

root@archiv:~# ls -la /opt/piler/sphinx
insgesamt 168
drwxr-xr-x 2 piler piler 4096 Jan 31 16:47 .
drwxr-x--x 7 piler piler 4096 Jan 31 16:26 ..
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 dailydelta1.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 dailydelta1.spd
-rw-r--r-- 1 piler piler  582 Jan 31 16:39 dailydelta1.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 dailydelta1.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 dailydelta1.spk
-rw------- 1 piler piler    0 Jan 31 16:47 dailydelta1.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 dailydelta1.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 dailydelta1.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 dailydelta1.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 delta1.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 delta1.spd
-rw-r--r-- 1 piler piler  582 Jan 31 16:39 delta1.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 delta1.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 delta1.spk
-rw------- 1 piler piler    0 Jan 31 16:47 delta1.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 delta1.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 delta1.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 delta1.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main1.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main1.spd
-rw-r--r-- 1 piler piler  582 Jan 31 16:39 main1.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main1.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main1.spk
-rw------- 1 piler piler    0 Jan 31 16:47 main1.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main1.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main1.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main1.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main2.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main2.spd
-rw-r--r-- 1 piler piler  582 Jan 31 16:39 main2.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main2.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main2.spk
-rw------- 1 piler piler    0 Jan 31 16:47 main2.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main2.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main2.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main2.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main3.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main3.spd
-rw-r--r-- 1 piler piler  582 Jan 31 16:39 main3.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main3.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main3.spk
-rw------- 1 piler piler    0 Jan 31 16:47 main3.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main3.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main3.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main3.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main4.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main4.spd
-rw-r--r-- 1 piler piler  582 Jan 31 16:39 main4.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main4.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main4.spk
-rw------- 1 piler piler    0 Jan 31 16:47 main4.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 main4.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main4.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 main4.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 note1.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 note1.spd
-rw-r--r-- 1 piler piler  289 Jan 31 16:39 note1.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 note1.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 note1.spk
-rw------- 1 piler piler    0 Jan 31 16:47 note1.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 note1.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 note1.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 note1.sps
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 tag1.spa
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 tag1.spd
-rw-r--r-- 1 piler piler  288 Jan 31 16:39 tag1.sph
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 tag1.spi
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 tag1.spk
-rw------- 1 piler piler    0 Jan 31 16:47 tag1.spl
-rw-r--r-- 1 piler piler    0 Jan 31 16:39 tag1.spm
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 tag1.spp
-rw-r--r-- 1 piler piler    1 Jan 31 16:39 tag1.sps

I would be happy for any help. Thank you in advance, bg,

Philip

Comments (9)

  2. Janos SUTO repo owner

    Yeah, please don't post to closed issues. The problem is that the sphinx indexer hasn't run yet. So either wait a little (I assume the piler cron jobs are installed), or execute /usr/local/libexec/piler/indexer.delta.sh as user piler.

    Also note that the indexer runs in every 30 mins, so when an email arrives, it takes some time until it becomes visible in the gui.

  3. Philip Deubner reporter

    Thank's for your quick answer. When trying to do /usr/local/libexec/piler/indexer.delta.sh as user piler, I get the following error message:

    /usr/local/libexec/piler/indexer.delta.sh: line 14: /var/run/piler/delta.indexer.tmp: No such file or directory
touch: can't touch „/var/piler/stat/indexer“: No such file or directory

    The directory /var/run looks like:

    -rw-r--r-- 1 root         root            5 Jan 31 15:53 acpid.pid
srw-rw-rw- 1 root         root            0 Jan 31 15:53 acpid.socket
drwxr-xr-x 2 root         root           40 Jan 31 16:13 apache2
-rw-r--r-- 1 root         root            5 Feb  1 17:42 apache2.pid
-rw-r--r-- 1 root         root            5 Jan 31 15:53 atd.pid
drwxr-xr-x 2 clamav       root          100 Jan 31 16:23 clamav
-rw-r--r-- 1 root         root            5 Jan 31 15:53 crond.pid
---------- 1 root         root            0 Jan 31 15:53 crond.reboot
drwxr-xr-x 2 messagebus   messagebus     80 Jan 31 16:14 dbus
-rw-r--r-- 1 root         root            5 Jan 31 15:53 dhclient.eth0.pid
drwxr-x--- 2 Debian-exim  Debian-exim    60 Jan 31 15:53 exim4
prw------- 1 root         root            0 Jan 31 15:53 initctl
drwxr-xr-x 2 root         root           40 Jan 31 15:53 initramfs
drwxrwxrwt 3 root         root           60 Jan 31 16:13 lock
-rw-r--r-- 1 root         root            5 Jan 31 16:14 memcached.pid
-rw-r--r-- 1 root         root           64 Jan 31 15:53 motd.dynamic
drwxr-xr-x 2 root         root           60 Jan 31 15:53 mount
drwxr-xr-x 2 mysql        root           80 Jan 31 16:13 mysqld
drwxr-xr-x 2 root         root           60 Jan 31 15:53 network
drwxr-xr-x 2 root         root           40 Feb  1 14:53 proftpd
-rw-r--r-- 1 root         root        12608 Feb  1 15:09 proftpd.delay
-rw-r--r-- 1 root         root            5 Feb  1 15:09 proftpd.pid
-rw-r--r-- 1 root         root          656 Feb  1 19:28 proftpd.scoreboard
-rw-r--r-- 1 root         root            0 Feb  1 15:09 proftpd.scoreboard.lck
srwxrwxrwx 1 root         root            0 Feb  1 14:53 proftpd.sock
drwxr-xr-x 2 root         root           80 Jan 31 15:53 rpcbind
-r--r--r-- 1 root         root            0 Jan 31 15:53 rpcbind.lock
-rw-r--r-- 1 root         root            4 Jan 31 15:53 rpcbind.pid
srw-rw-rw- 1 root         root            0 Jan 31 15:53 rpcbind.sock
-rw-r--r-- 1 statd        nogroup         5 Jan 31 15:53 rpc.statd.pid
-rw-r--r-- 1 root         root            5 Jan 31 15:53 rsyslogd.pid
drwxr-xr-x 2 root         root          100 Jan 31 15:53 sendsigs.omit.d
drwxrwxrwt 2 root         root           60 Jan 31 15:53 shm
-rw------- 1 root         root            5 Jan 31 15:53 sm-notify.pid
drwxr-xr-x 2 sphinxsearch root           40 Jan 31 16:14 sphinxsearch
drwxr-xr-x 2 root         root           40 Jan 31 15:53 sshd
-rw-r--r-- 1 root         root            5 Jan 31 15:53 sshd.pid
drwxr-xr-x 7 root         root          180 Jan 31 15:53 udev
-rw-rw-r-- 1 root         utmp         4608 Feb  1 19:27 utmp

    The second failure sounds "reasonably", as I changed the /var/piler to /opt/piler... Any ideas on this?

  4. Janos SUTO repo owner

    Start the piler daemon to have it create /var/run/piler (or create it for yourself manually), then run the indexer script again.

  5. Philip Deubner reporter

    Starting the daemon is done by either 

    /etc/init.d/rc.piler start
/etc/init.d/rc.searchd start

    or 

    /etc/init.d/rc.pilergetd start

    right?

    Unfortunately, this doesn't change anything.

  6. Janos SUTO repo owner

    This should create /var/run/piler

    /etc/init.d/rc.piler start

    Then run indexer.delta.sh manually, and let's see what happens.

  7. Philip Deubner reporter

    I'll put this issue on hold, as I just created a new, clean installation in another VM - without any "fancy" changes in the installation process. After the installation is done, I will import some mails to see whether I was simply too stupid to get a manual installation done. Thank you for your help. I'll give you a feedback by tomorrow. Bg,

    Philip

  9. Philip Deubner reporter

    Issue is solved, as it was seemingly a misconfiguration. After a clean install, everything worked as it should.

  10. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
2
Jira: the preferred issue tracker for Bitbucket. Join the team!