Mails missing from GUI but present in MySQL

[Former user]

Hi! First of all, thank you for such amazing tool! I am importing emails from my Google Apps Free edition. Currently I have around 6k emails imported to Piler, but some of them are not showing in webUI, but I can find them in database in metadata table.

I should say, that I have one primary domain assigned to Google Apps as well as additional alias domain (so user can receive emails on both addresses, [e-mail] and [e-mail]). I've modified /model/user/google.php file to add additional domains to "emails" field, so sphinx queries looks like this:

sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE        MATCH('( @(subject,body) backup ) &  (@from userAXdomainAXcom| userAXdomainBXcom | @to userAXdomainAXcom| userAXdomainBXcom) ') ORDER BY `sent` DESC LIMIT 0,1000 OPTION max_matches=1000' in 0.00 s, 14 hits, 14 total found

This works great, but as I said, some messages can't be found in webui, but are properly imported to MySQL.

Any hints what can be wrong?

[Mateusz]

Hi, I've found another email that is not being shown in webUI (please see attached file: https://copy.com/puUnf8kgAkfMMVEi). userA received this email, because he is in [e-mail] mailing group.

From what I think in this case Piler should use Delivered-To field to determine correct user, but I don't know whether this is correct only for GMail servers or all of them. Thanks in advance.

[Janos SUTO]

Try the same query as an auditor user, and let's see what happens

[Mateusz]

Hi, yes, these mails are visible when I log in as an auditor, although the To: field shows "[e-mail]".

[Janos SUTO]

The problem is that there's no To: field in the email. However there's a solution for new emails: set the following in piler.conf, then restart the daemon:

extra_to_field=Delivered-To:

If you want to fix the existing emails, then it's not that easy, but possible. You have to do 2 things:

• fix the rcpt to table for the given message, and add the Delivered-To: address
• reindex the given message, eg. reindex -f 152 -t 152

[Mateusz]

I've added this option to config file, cleared database/store folder/sphinx index and then redownloaded all mails. Looks like everything is working correctly now :) Thank you!

[Janos SUTO]

You are welcome