-
assigned issue to
I cannot see attachments in e-mails / they are still in the EML file
Hi
I don't know if this is the same than #704.
Piler version: 1.2.0-master build 945 Ubuntu 16.04.1 / MySQL 5.7 / PHP 7
I have checked several old e-mail imported with readpst / pilerimport.
I (auditor) don't see the attachments in the GUI.
But, if I download the email, I see the attachment embedded. It's a base64 encoded zip file. The boundary exists and seems to be correct.
--=-vnG+PdvbGusWwtziozLw Content-Disposition: attachment; filename="wz020700.zip" Content-Transfer-Encoding: base64 Content-Type: application/octet-stream
I will send the sample direcly by e-mail
Comments (26)
-
reporter -
repo owner Thanks for the test email. However it looks good testing with the master branch: it can detect the attached zip file properly, and stores it accordingly. Run pilertest against this message, and verify that it can see the attachment. If you download the email, then the attachment should be embedded just as it's in the original email. And you should see a link to the attachment as well.
-
reporter The message was loaded with pilerimport. I will retest with this tomorrow.
Thanks!
-
repo owner Let me know, how it goes.
-
repo owner - changed status to closed
No news is good news.
-
I'm having this issue as well. I'm using the OVA from here: http://download.mailpiler.org/piler-build-935.ova and downloading emails using pilerimport from office365. Some, but not all emails with attachments are not showing the attachments in the webGUI, but they are there when I open the eml file from piler.
-
repo owner That's a pretty old build by now, I'm not sure if I'll upgrade it or provide a docker image instead.
Anyway, check if the problematic email is stored properly, ie. it has all the attachments in the attachment table (check first with the metadata table, and see the attachments column). If it seems OK, then it's a gui bug.
-
Are there instructions anywhere to upgrade from Build 935 to the one you just released?
-
repo owner Well, you should update the database schema using the sql script in util/ directory. Then compile the new version, install it, overwrite the gui, and you are done. Since you have a virtualized environment, I suggest to use snapshots allowing you to roll back in case of a problem, or if you have the required resources, you may copy the VM, and perform the upgrade on the copy to see if it goes smoothly. Anyway, be sure to backup the mysql database, and all piler related config files, keys, etc.
-
ok, I upgraded the virtual machine to the newest everything that apt-get offered, then I upgraded to the 1.2.0 release of piler, and then I did reindex -a. These emails are still not showing their attachments in the UI or the metadata table. Any further things I can try?
-
repo owner Show me the metadata entries for such an email. I'm interested in the id, piler_id, attachments columns. Then we'll see if the email is stored properly, ie. recognized the attachments or not.
-
SELECT * FROM piler.metadata order by size desc limit 10;
http://www.sharecsv.com/s/59e01730fcb2faa3ac55cfa26f3eb520/test.csv
-
repo owner Which email from this list doesn't look right in the gui?
-
The second one for sure, but I think all of them that don't have attachments listed. These are all 30+ megabyte eml files. I don't see how they could not have attachments.
Thanks, Jeremiah
-
repo owner I see that. Run the following commands, and let me know what pilertest reports. I'm interested in only the attachment related output.
cd /tmp pilerget 400000005805644e10b8097400ee25930101 > 1.eml pilertest 1.eml
-
body digest: 97430db0ee3eea1d5ccc7b9382b2f5c9ae18604f5ad00ea54b30b82c4ddac7f1 rules check: (null) folder: 0 retention period: 1698934310 i:1, name=*20140929_160740.pdf*, type: *application/pdf*, size: 34637082, int.name: 1.eml.a1, digest: c815e1bdd5081b5fa0dd3adb9e8b842e2ffd6d384f4f6c0df02336dd9b8ee4a1 attachments:pdf, direction: 1 spam: 0
-
repo owner OK, the parser knows correctly that it has a large pdf attached. Is it possible that these "0 attachment" files were archived with the older version, and the pilertest output came from version 1.2.0?
-
Yes, that's the situation.
Is there any way to reparse everything? I was hoping that the reindex -a would reparse, but I guess not.
Thanks, Jeremiah
-
Any more thoughts on this? Thanks!
-
repo owner Well, there's no way to reparse an already archived email, sorry. Technically it's possible to fix a message after it's stored, but there's no tool for that.
-
BTW, The tools do exist. As they are non-free you will need to contact us.
Only thing is one would need to recreate the archive, pretty much xshok-piler-archive-recovery and then xshok-piler-import-dir will have the entire archive reconstructed with the attachments imported and everything corrected.
-
Are those tools available somewhere that I would have access to? and would I need to wipe out the current database before running them? Or, I guess a better question would be: Is there a step by step guide? Thanks again!
-
repo owner The official piler repo doesn't have such a tool. Keep me posted how it goes.
-
They are non-free , https://bitbucket.org/snippets/extremeshok/dddX9/contact-info
-
I appreciate you bringing these tools to my attention, but this is just a minor annoyance to me, not a showstopper. so I'll pass this time. Thank You!
-
repo owner OK, then case is closed.
- Log in to comment