Missing Attachments on old archived mails

[Malek]

Hello All,

On piler archive, new mails with attachments are all OK. But when i search for old mails, they are present but the attachments isn’t displayed. Some of them, when i download the .eml file, it shows the attachment. The other, even if i download the .eml file, the attachment doesn’t appear, but for both cases the size of the mail is correct (it takes in count the attachment file size). I tried to do some re indexing without success.

Please help.

Kind Regards.

[Janos SUTO]

Check what the mail log says.

[Malek]

Hello,

Thank you for the quick response. There is only one message repeated :

/var/log/mail.log.1:Dec  4 14:35:17 archive piler[31918]: error: helper: execl

Nothing related to attachments. Thanks in advance.

[Janos SUTO]

So you are running tail -f /var/log/mail.log, then you select a message with the missing attachment, and you click on download, you can’t see anything in the log, right?

[Malek]

Yes, nothing in the log, only few lines like :

‌

Dec 10 09:05:01 archive piler: INDEXER INFO: indexing delta1 started
Dec 10 09:05:01 archive piler: INDEXER INFO: indexing delta1 finished
Dec 10 09:05:06 archive piler: INDEXER INFO: merging delta to dailydelta started
Dec 10 09:05:07 archive piler: INDEXER INFO: merging delta to dailydelta finished

Nothing related to the attachment. Maybe it’s a sphinx problem but i tried to do a reindex -a with no luck. Thank you for taking time to respond.

[Janos SUTO]

OK, then we have to dig deeper. So you select a message with an attachment. I assume you can see the attachment list in the upper part of the preview pane. Now click on one of the attachments. What happens? The pileraget utility should be called by the gui. If it can’t retrieve the given attachment, then it should syslog the issue. Also I’d like you to more. When you hover over the download link, then you can see the ‘id’ number.

Do the following sql query with it on the piler database (id=1234 in this example, use the seen value):

select * from metadata where id=1234;

Notice the piler_id value, and do another query (use the seen piler_id value):

select * from attachment where piler_id='40000…..';

Be sure to format the results.

[Malek]

Thank you for your response.

So the problem is that when i click on a message with an attachment, i can’t see the list in the upper part of the preview pane, but it has the correct mail size. But when i download the mail, i can see the attachment.

Regards.

[Janos SUTO]

The correct mail size is one thing, if the email content is correct, well it’s another. So please do the debugging I asked above.

[Malek]

Sorry for taking you time, i know that you’re very busy. I think that you misunderstood me. The debugging that you asked can’t be done cause i can’t see the attachment at all. So i can’t click on it to see if the pilertarget is called or not. Thank you.

[Janos SUTO]

Sigh. If you can’t see attachments in the preview pane, then the given message has no attachments. So I need some proof that you have an old message with attachments, but you can’t see the actual attachments.

[Malek]

There is two proof : The first one is the size of mail that is more than 2Mb. The second, is that when i download the mail, i can see the attachment. Thank you.

[Janos SUTO]

OK, then the parser decided that the attachment is not separated to a dedicated file for some reason.

If it’s possible, then please share the eml file that I can investigate further. Otherwise I can’t do much. Btw. what piler version do you use?

[Malek]

Here is the eml content (I’ve hidden some content) :

Date: Tue, 29 May 2018 10:17:52 +0100 (CET)
From: <Hidden Content>
To: <Hidden Content>
Message-ID: <Hidden Content>
Subject: =?utf-8?Q?Script_Cr=C3=A9ation_Folder?=
MIME-Version: 1.0
Content-Type: multipart/mixed; 
    boundary="----=_Part_71_174762847.1527585472347"
X-Originating-IP: <Hidden Content>
X-Mailer: <Hidden Content>
Thread-Index: 0Z2elknuJ5s/TcMvYeV7BJNAe49iqA==
Thread-Topic: Script =?utf-8?Q?Cr=C3=A9ation?= Folder

------=_Part_71_174762847.1527585472347
Content-Type: multipart/alternative; 
    boundary="=_0580bfd3-5da0-4618-8fb6-3d194494693c"

--=_0580bfd3-5da0-4618-8fb6-3d194494693c
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

<mail body>

--=_0580bfd3-5da0-4618-8fb6-3d194494693c
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 12pt; color: #000000"><div><Hidden Content><br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div><Hidden Content><br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div><Hidden Content><br data-mce-bogus="1"></div></div></body></html>
--=_0580bfd3-5da0-4618-8fb6-3d194494693c--

------=_Part_71_174762847.1527585472347
Content-Type: application/octet-stream; name=CreateShareForEachUser.ps1
Content-Disposition: attachment; filename=CreateShareForEachUser.ps1
Content-Transfer-Encoding: base64

c3MgfCBGb3JFYWNoIHsgW1ZvaWRdJEFDTC5SZW1vdmVBY2Nlc3NSdWxlKCRfKSB9DQoJDQoJJE5l
d0FjY2Vzc3J1bGUgPSBOZXctT2JqZWN0IFN5c3RlbS5TZWN1cml0eS5BY2Nlc3NDb250cm9sLkZp
bGVTeXN0ZW1BY2Nlc3NSdWxlKCRVc2VyU0FNLCAiRnVsbENvbnRyb2wiLCAiQ29udGFpbmVySW5o
ZXJpdCxPYmplY3RJbmhlcml0IiwgIk5vbmUiLCAiQWxsb3ciKQ0KCSRBQ0wuU2V0QWNjZXNzUnVs
ZSgkTmV3QWNjZXNzcnVsZSkNCglTZXQtQWNsIC1wYXRoICR1c2VyZm9sZGVyIC1BY2xPYmplY3Qg
JEFDTA0KDQoJIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCgkjIyBQZXJtaXNzaW9uIEZvciBVdGlsaXNhdGV1cnMg
------=_Part_71_174762847.1527585472347--

The Piler version is : 1.2.0-master build 935. Thanks.

‌

[Malek]

Hello Janos,

Any updates ?

Regards.

[Janos SUTO]

Well, I’ve just tested it with your redacted message, and it properly displays (and stores separately) CreateShareForEachUser.ps1.

Note that I’ve tested with the latest master branch. Your build is several versions behind. Anyway, do the following. Run pilertest against this message, and show me its output.

[Malek]

Here is the output :

‌

locale: en_US.UTF-8
build: 935
parsing...
post parsing...
message-id: <Hidden Content> / 4ba8f36951a74ac805a4b88f850bea35f23f5b0dce000cf1c6c8f24e4c7144fb
from: *hidden content  ()*
to: *hidden content  ()*
reference: **
subject: *Script Création Folder*
body: *mail body *
sent: 1527585472, delivered-date: 0
hdr len: 443
body digest: 0efecb3a2dee88f07345e03aa9f48d4459525553e685937f1071c5fd129a95bc
rules check: (null)
folder: 0
retention period: 2199168172
i:1, name=*CreateShareForEachUser.ps1*, type: *application/octet-stream*, size: 3134, int.name: Script-Cr-ation-Folder.eml.a1, digest: 8367cc79def8e341425db20e2c6ab6ed0c2efe5827354faecfe9f42c1a7a0057
attachments:other,
direction: 0
spam: 0
NOT IN mydomains

I think that the problem is related to sphinx (some indexing issue ? ). Thanks in advance.

[Janos SUTO]

OK, be sure to update the ticket if there's anything new.