pilerimport rejects messages with invalid message, hdr_len: 0
pilerimport recjects messages "invalid_message, hdr_len: 0.
[root@triton piler]# piler -V piler 1.2.0, build 952, Janos SUTO sj@acts.hu
Build Date: Wed Jun 14 11:46:14 UTC 2017
ldd version: ldd (GNU libc) 2.17
gcc version: gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
OS: Linux triton 3.10.0-229.1.2.el7.x86_64 #1 SMP Fri Mar 27 03:04:26 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Configure command: ./configure --localstatedir=/var --with-database=mysql --enable-tcpwrappers
MySQL client library version: 5.5.52-MariaDB
Extractors: /usr/bin/pdftotext /usr/bin/catdoc /usr/bin/catppt /usr/bin/xls2csv /usr/local/bin/ppthtml /usr/bin/unrtf /usr/bin/tnef
[root@triton piler]# pilerimport -e email_bug/3241.emlx email_bug/3241.emlx: invalid message, hdr_len: 0
Comments (16)
-
repo owner -
reporter Here is the header part of the elmx File
Received: from BUGSERVER.BUG.local (192.168.51.200) by BUGSERVER.BUG.local (192.168.51.200) with Microsoft SMTP Server (TLS) id 15.0.847.32 via Mailbox Transport; Tue, 26 Sep 2017 08:53:13 +0200 Received: from BUGSERVER.BUG.local (192.168.51.115) by BUGSERVER.BUG.local (192.168.51.115) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 26 Sep 2017 08:52:52 +0200 Received: from BUGSERVER.BUG.local ([fe80::914f:f1a:8a22:51dd]) by BUGSERVER.BUG.local ([fe80::914f:f1a:8a22:51dd%12]) with mapi id 15.00.0847.030; Tue, 26 Sep 2017 08:52:34 +0200 From: BuG Sabina Salkanovic sabina.salkanovic@bug.co.at To: "BuG Markus Stieger" markus.stieger@bug.co.at Subject: BVH Starkfriedgasse | Fuchs | Haupt-AN | Stiegenhaustreppe Thread-Topic: BVH Starkfriedgasse | Fuchs | Haupt-AN | Stiegenhaustreppe Thread-Index: AdM2k/EXk3IT1uEZTjiSSnlCtNz1Bw== Date: Tue, 26 Sep 2017 06:52:33 +0000 Message-ID: e80a3676f2754333a1f27fe5af7aad06@BUGSERVER.BUG.local Accept-Language: de-AT, en-US Content-Language: de-DE X-MS-Exchange-Organization-AuthAs: Internal X-MS-Exchange-Organization-AuthMechanism: 03 X-MS-Exchange-Organization-AuthSource: BUGSERVER.BUG.local X-MS-Has-Attach: yes X-MS-Exchange-Organization-SCL: -1 X-MS-TNEF-Correlator: Content-Type: multipart/mixed; boundary="004_e80a3676f2754333a1f27fe5af7aad06BUGSERVERBUGlocal" MIME-Version: 1.0
--004_e80a3676f2754333a1f27fe5af7aad06BUGSERVERBUGlocal Content-Type: multipart/alternative; boundary="000_e80a3676f2754333a1f27fe5af7aad06BUGSERVERBUGlocal"
-
repo owner No good, I can't use this text, I need the email file intact. If it's not (too) sensitive, please either send it to my address in a zip, or perhaps even better if you could put it to somewhere I can download the complete email. If it's sensitive, then run pilertest on it, and check its output. If it's not good, then download the master branch, compile it (but don't install it or overwrite anything! It's best done on another host), and run the latest version's pilertest on it.
-
reporter ok, here you can find the complte mailsource: http://www.openit.at/download/3241.emlx
and here is the pilertest output:
pilertest email_bug/3241.emlx locale: C build: 952 parsing... post parsing... message-id: null / 74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b from: * () to: * () reference: subject: body: ** sent: 1506504711, delivered-date: 0 hdr len: 0 body digest: rules check: (null) folder: 0 retention period: 1727429511 attachments: direction: 0 spam: 0 NOT IN mydomains
-
repo owner Thanks for the file, you may remove it from your download area. I think user piler has no read access on the given file.
-
reporter Ok, I have changed the permissions of the emlx files and now the import is running with no problems, but in the piler webinterface I see emails which are empty. I have attached 2 screenshots. Not all of the imported emails but a lot of them.
Is there a possibilty to force a reimport?
-
repo owner I can't see any screenshots. Reimport would not work well, because the message-id is recorded to the piler database that prevents the same message to be archived again. If you have all the original emails, then I recommend you the following to try:
- create a new database for piler, eg. piler2
- create a new store dir for piler, eg. /var/piler/store2
- update piler.conf to point to the new database and store dir, eg.
mysqldb=piler2 queuedir=/var/piler/store2
- update config-site.php to use the new database
Then run the import procedure again, and check the results.
-
reporter ok, I did not upload the screeshots; here they are:
http://www.openit.at/download/1.jpg http://www.openit.at/download/2.jpg
-
repo owner OK, I see. Please try the method I suggested.
-
reporter I have done your suggested method:
-
I cloned the complete machine (xenserver virtual machine)
-
reset of piler (FAQ: How can I delete everything from the archive?)
-
import of the emlx files.
and I see the same as before on all emported Emails!
-
-
repo owner Do you mean that all emails are empty? Any error during the import? Any errors in the mail logs? Does user piler have proper permissions on the to be imported emails?
-
reporter The permissions on the directory where the emlx files are located has 755; the emlx files have 666.
This is the output during the import process:
[root@larissa piler]# pilerimport -d email_bug duplicate: email_bug/3320.emlx (duplicate id: 6) Syntax Warning: Invalid Font Weight Syntax Warning: Invalid Font Weight Syntax Warning: Invalid Font Weight Syntax Error: Unknown character collection 'PDFAUTOCAD-Indentity0' duplicate: email_bug/3370.emlx (duplicate id: 33) duplicate: email_bug/3357.emlx (duplicate id: 35) duplicate: email_bug/3336.emlx (duplicate id: 25) duplicate: email_bug/3349.emlx (duplicate id: 10) duplicate: email_bug/3309.emlx (duplicate id: 44) duplicate: email_bug/3340.emlx (duplicate id: 60) duplicate: email_bug/3343.emlx (duplicate id: 39) duplicate: email_bug/3269.emlx (duplicate id: 37) duplicate: email_bug/3299.emlx (duplicate id: 27) duplicate: email_bug/3325.emlx (duplicate id: 2) Syntax Error: Unknown character collection 'PDFAUTOCAD-Indentity0' duplicate: email_bug/3363.emlx (duplicate id: 28) duplicate: email_bug/3249.emlx (duplicate id: 96) duplicate: email_bug/3372.emlx (duplicate id: 1) duplicate: email_bug/3244.emlx (duplicate id: 5) duplicate: email_bug/3245.emlx (duplicate id: 11) duplicate: email_bug/3254.emlx (duplicate id: 43) duplicate: email_bug/1506595197.M618504P12322.io.openit.at,S=47151,W=48047:2,RS (duplicate id: 26) processed: 132
tar -f /var/log/maillog
Oct 2 14:59:00 larissa pilerget[12487]: /var/piler/store/00/59d/99/db/4000000059d2255b1e31aed4007ffe5299db.m: cannot open() Oct 2 14:59:00 larissa pilerget[12490]: /var/piler/store/00/59d/99/db/4000000059d2255b1e31aed4007ffe5299db.m: cannot open()
ls -l /var/piler/store/00/59d/3e/34/4000000059d225473b1aed94009c949a3e34.m
-
repo owner Can I access this cloned environment to see what went wrong?
-
reporter here is a quick update to my problem. If you download the stored email from piler webinterface, then you get back the imported email and it is identically to the original emlx file.
-
repo owner Please remove the comment containing the password asap.
Janos
-
repo owner - changed status to invalid
Piler couldn't create its temp files during parsing the emails.
- Log in to comment
Can I see this file: email_bug/3241.emlx?