Emails Not appearing in the user account

[Yousif Alkhateeb]

Hello,

yesterday I’ve imported about 16000 emails into piler , these emails belongs only to one account .

when i login with the user account only about 9000 emails appear in the search while with the auditor all of them are displayed.

‌

here’s the log when i hit search in both accounts

Feb 17 10:24:06 piler piler-webui[28274]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH('') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=1000' in 0.01 s, 20 hits, 16942 total found
Feb 17 10:24:24 piler piler-webui[31974]: username=yraid@example, event='logged in', ipaddr=172.31.14.34
Feb 17 10:24:26 piler piler-webui[31974]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@from yraidXexampleXcom | @to yraidXexampleXcom) ') ORDER BY sent DESC LIMIT 0,20 OPTION max_matches=1000' in 0.01 s, 20 hits, 9255 total found

[Janos SUTO]

I suspect that the ‘missing’ emails were not sent to [e-mail], perhaps to an alias or a mailing list address. As auditor select such a message and verify its sender and recipient.

[Yousif Alkhateeb]

I think this is not the case because i used to test piler on the same user account and on my previous imports the all of the emails appeared , about after i finished testing i removed everything with the procedure in the FAQ and start a clear piler.

after i did this the emails start to be missing.

i even test this with another account but the same thing happened.

[Janos SUTO]

OK, then tell me what’s the case.

[Yousif Alkhateeb]

I’ve imported [e-mail] before , it was my initial test mailbox when i imported it for the first time all the emails appeared.

Now i removed piler database and emails and I've imported [e-mail] INBOX , please see the attached screen shot for the number of emails after the import and the emails appears in the piler GUI, note that I’ve waited more than an hour after the import.

‌

‌

‌

[Janos SUTO]

I still would like to see the headers of such an email auditor can see, but you can’t. If there’s none, then reset the sphinx database. Remove all sphinx files, initialize it, then reindex all of your emails.

[Yousif Alkhateeb]

I try to reindex , but when i ran the indexer --all command with the piler user i got this error.

FATAL: failed to lock /var/piler/sphinx/main1.spl: Resource temporarily unavailable, will not index. Try --rotate option.

[Janos SUTO]

Stop searchd before you start.

[Yousif Alkhateeb]

after the re indexing all the emails disappeared from the GUI

‌

[Janos SUTO]

Check if the delta indexer script has run.

[Janos SUTO]

No news is good news.

[Yousif Alkhateeb]

Hello,

I’m sorry for the late reply , i needed to test somethings with the import,

I’ve check the delta and the main scripts the ran successfully , even i tried to run them manually but nothing has changed.

After a lot of tests I’ve decided to do a clean os instillation with a new piler instillation , then i ran pilerimport from an imap server.

the import detected about 20,000 emails but in the GUI there’s only aroung 15,000 emails , after about an hour the number of email in the GUI decreased to 11,000 emails.

so i tried to search for old emails in my inbox , some of them appear and some of them don’t.

‌

Thanks for you help in advance.

‌

[Janos SUTO]

Check how many rows you have in metadata table. You have that many messages.

[Yousif Alkhateeb]

I have 20,469 messages in the metadata table which is the exact number of message from the imported mailbox

[Janos SUTO]

If you can still see much less messages in the gui, then reindex what is missing.