reindex -a causes segmentation fault

[Mark Clarke]

I am using Ubuntu and the daily cron job delete the mail piler indices. I have followed the FAQ recommendations and disabled the cron job and attempted to recover with "reindex -a" but I get a segmentation fault error.

I am using Ubuntu 12.04.3 The log files says:

Sep 25 15:50:34 srv01 kernel: [ 6731.994791] reindex[4744]: segfault at 7fff00000000 ip 00007f5cddcc25be sp 00007fff5e7977f0 error 4 in libpiler.so.0.1.1[7f5cddcab000+22000]

Here is some additional information. Not sure if it will help: piler 0.1.24-master-branch, build 836, Janos SUTO [e-mail]

Build Date: Tue Sep 17 13:32:14 SAST 2013 ldd version: ldd (Ubuntu EGLIBC 2.15-0ubuntu10.4) 2.15 gcc version: gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) Configure command: ./configure --localstatedir=/var --with-database=mysql --enable-starttls --enable-tcpwrappers

[Janos SUTO]

Eh, just setting up an ubuntu 64 box, and see it for myself.

[Mark Clarke]

thanks for the feedback.

[Janos SUTO]

I managed to reproduce it. The error happens when it tries to get the message from the archive and tries to assemble the full path of the attachment file, and for a mysterious reason it just segfaults on (only) ubuntu (so far).

So my next move is to test it on debian 7.1, and try the same. If it goes well, then I'll create a deb package on debian, and test it on ubuntu.

[Mark Clarke]

Good to know its not just my machine. Will wait for the results of your next test.

[Janos SUTO]

ok, it appears that it's ubuntu. I compiled piler on debian, copied the binaries to ubuntu, used pilerimport to get 300 emails from gmail, then did "reindex -a", and it reported that "put 300 messages to sph_index table for reindexing".

I'll make the binaries available as a quick fix: http://www.mailpiler/download/piler-binaries-amd64-master-branch-build-845.tar.gz

It's a drop in replacement for the binaries under /usr/local/bin and sbin. Just tar zxf it, and chown piler:piler then chmod +s for the binaries under bin (piler* and reindex).

Now I'll search google how to make a deb package.

So I'd like you to try it. Stop piler, then replace its binaries. Make sure you have libzip, pdftotext, unrtf, tnef and catdoc installed.

[Mark Clarke]

I copied the binaries over. I am not sure how to "stop piler" as it doesn't seem to run as a service. I just got sphinx running as a service and apache.

I did a "su piler" then "cd /tmp; reindex -a". I doesnt crash but I just get a messsage

"put 0 messages to sph_index table for reindexing"

When I do a search in piler I get no results returned.

[Janos SUTO]

OK, then you don't have to stop the piler daemon :-) Is reindex owned by piler and a setuid binary? Do you have anything in the sph_index table? And in metadata table?

[Mark Clarke]

cool :) The binaries have the correct permissions and owner settings. The metadata table has 430350 records but the sph_index table is empty.

[Janos SUTO]

Hmm, that's odd. It seems that it just can't insert any row to the sph_index table. Can you add another email to the archive by pilerimport? If so then please check the sph_index table if there's anything in it. Also please show me "describe sph_index;" on the table.

Btw. can you see any errors in your mail.log file?

Update: can you anything as user auditor if you just hit the search button?

Also please try the following, and show me the results:

$ mysql -h 127.0.0.1 -P 9306

mysql> select * from main1;

[Mark Clarke]

Hi jsuto,

Sorry was out of the country for a while. Here is some of the info you requested. I am gathering the rest. Main1 table doesn't exist :(

Nothing unusual in the mail.log file.

describe sph_index;

+------------------+------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------------+------------------+------+-----+---------+-------+ | id | bigint(20) | NO | PRI | NULL | | | from | char(255) | YES | | NULL | | | to | text | YES | | NULL | | | fromdomain | char(255) | YES | | NULL | | | todomain | text | YES | | NULL | | | subject | text | YES | | NULL | | | arrived | int(10) unsigned | NO | | NULL | | | sent | int(10) unsigned | NO | | NULL | | | body | text | YES | | NULL | | | size | int(11) | YES | | 0 | | | direction | int(11) | YES | | 0 | | | folder | int(11) | YES | | 0 | | | attachments | int(11) | YES | | 0 | | | attachment_types | text | YES | | NULL | | +------------------+------------------+------+-----+---------+-------+

Select * from main1;

Table 'piler.main1' doesn't exist

[Janos SUTO]

Please download the latest master branch, it should fix this error.

[Janos SUTO]

ok, I assume this issue is resolved by the master branch

[Mark Clarke]

Hi jsuto,

I am going to test the this now. Sorry, again, for the delay. How do I get the master branch binaries? Also will this make it difficult to upgrade in future if I use the master branch? Last time i just had to copy out some files. From the archive so I assume it will be ok to upgrade in future.

thanks

All the best for 2014 :)