Web UI Not showing All Emails

[Justin Watkins]

Hello, I have a fresh install of Mail PIler on Ubuntu 20.04. Everything seems to be working but after a day, all my emails disappear from my user interface. The Health Dashboard shows that all the emails are there but they aren’t in the Web UI. I don’t see any cron jobs as some of the articles have said might be causing this. Could you please help me?

[Janos SUTO]

Hello, show me the output of ls -la /var/piler/sphinx

[Justin Watkins]

Here is the output. So I struggled with the install of the Mail Piler. I used an article I found that walked me through the install. I am certainly not opposed to rebuilding the server as this is a new build that I am testing. If you think my issue can be fixed then great! If you think rebuilding would be easier, I’d be appreciative if you had some instructions I could follow to install it… Thanks in advance for your help!

-rw-r--r-- 1 piler piler 120 Oct 23 19:35 dailydelta1.spa
-rw-r--r-- 1 piler piler 203 Oct 23 19:35 dailydelta1.spd
-rw-r--r-- 1 piler piler 1 Oct 23 19:35 dailydelta1.spe
-rw-r--r-- 1 piler piler 1247 Oct 23 19:35 dailydelta1.sph
-rw-r--r-- 1 piler piler 338 Oct 23 19:35 dailydelta1.spi
-rw-r--r-- 1 piler piler 4 Oct 23 19:35 dailydelta1.spk
-rw-r--r-- 1 piler piler 0 Oct 23 19:49 dailydelta1.spl
-rw-r--r-- 1 piler piler 161 Oct 23 19:35 dailydelta1.spp
-rw-r--r-- 1 piler piler 100 Oct 23 19:35 delta1.spa
-rw-r--r-- 1 piler piler 142 Oct 23 19:35 delta1.spd
-rw-r--r-- 1 piler piler 1 Oct 23 19:35 delta1.spe
-rw-r--r-- 1 piler piler 1247 Oct 23 19:35 delta1.sph
-rw-r--r-- 1 piler piler 285 Oct 23 19:35 delta1.spi
-rw-r--r-- 1 piler piler 47 Oct 23 19:35 delta1.spj
-rw-r--r-- 1 piler piler 4 Oct 23 19:35 delta1.spk
-rw-r--r-- 1 piler piler 0 Oct 23 19:49 delta1.spl
-rw-r--r-- 1 piler piler 127 Oct 23 19:35 delta1.spp
-rw-r--r-- 1 piler piler 0 Oct 23 19:45 note1.spa
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 note1.spd
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 note1.spe
-rw-r--r-- 1 piler piler 911 Oct 23 19:45 note1.sph
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 note1.spi
-rw-r--r-- 1 piler piler 8 Oct 23 19:45 note1.spj
-rw-r--r-- 1 piler piler 0 Oct 23 19:45 note1.spk
-rw-r--r-- 1 piler piler 0 Oct 23 19:49 note1.spl
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 note1.spp
-rw-r--r-- 1 piler piler 0 Oct 23 19:45 tag1.spa
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 tag1.spd
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 tag1.spe
-rw-r--r-- 1 piler piler 910 Oct 23 19:45 tag1.sph
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 tag1.spi
-rw-r--r-- 1 piler piler 8 Oct 23 19:45 tag1.spj
-rw-r--r-- 1 piler piler 0 Oct 23 19:45 tag1.spk
-rw-r--r-- 1 piler piler 0 Oct 23 19:49 tag1.spl
-rw-r--r-- 1 piler piler 1 Oct 23 19:45 tag1.spp

[Janos SUTO]

I’m not sure what guide you followed, did you try the official docs? Anyway, you miss the main1 index files. Check sphinx.config it has a reference for it. If so, then execute “indexer --config /path/to/sphinx.conf main1” as user piler. Then give it a few days and verify that it grows every day. After that we’ll reindex your emails.

[Justin Watkins]

Yeah I used the official docs. It just took me forever as I ran into some issues as I built it. Specifically with Sphinxsearch. I checked the sphinx.config and it had a reference to main1 so I ran the command you mentioned. It seemed to run ok so I’ll give it a few days. How can I verify it grows, at this point should the emails stay in the WebUI? Do I just use the reindex command to reindex all my emails?

[Janos SUTO]

You may verify main1 files with the previous ls -la command. Also be sure to restart searchd to read main1 data, too. When it all seems fine, run reindex -a to reindex everything.

[Justin Watkins]

Great thanks! I did restart the searchd service. I’ll check it again tomorrow to see how things are looking. I appreciate all your help. You may not hear it much but your support is most appreciated. My day to day job is an Office 365 Engineer. Your support is actually quicker than most vendors I have worked with…

[Janos SUTO]

Thanks! Anyway, keep me posted how it goes.

[Justin Watkins]

Hello, the reindex seems to have brought everything back. I can still see all my emails after a few days so it’s looking good! Thanks for your help! One odd thing (I think this may be an issue with my Qmail server but just thought I’d ask) the recipient domain does say [e-mail] when it’s an external domain. Is that an issue with Piler or with my Qmail server?

[Janos SUTO]

Check if the given email has a valid To and/or Cc field.

[Justin Watkins]

Yeah it is. The weird thing is it only shows that undiclosed-recipients in the table. When I click on an email it shows me the email with the recipient. I checked the headers of my emails and it shows a valid address.

[Janos SUTO]

Can you show me such message (perhaps a non sensitive) with full headers? You may send it to my email address as well, see piler -V for it.

[Janos SUTO]

The problem was the empty hostid parameter in piler.conf. After assigning a proper value, eg. archive.yourdomain.com the issue has been solved.