pilerimport: cannot open frame file

[supporthq]

Hi,

Something odd has occurred which has managed to completely fill the /tmp and /var/piler/imap drives to 100%.

I suspect it may have something to do with the Gmail and importing emails as the /var/piler/imap has thousands of files that are identical in size/content. There are also hundreds of thousands of files that have a zero byte size which I assume where created after the drives have filled up.

I have moved the contents of /var/piler/imap to a temp location and disabled the cron jobs that import the GMail messages. Seems stable since then but am worried about turning it back on.

Any suggestions on what I should be checking? Any tests I can run over the files in the /var/piler/imap folder? Also does the imap folder only hold data from Gmail imports (eg. does not hold any data messages recieved via SMTP)?

--- sample of mail.log messages (hundreds and thousands of these).

Jul 18 08:51:56 piler pilerimport[6127]: 40000000596d3f9614bd595c00814503ac37: cannot open frame file: 40000000596d3f9614bd595c00814503ac37.m Jul 18 08:51:56 piler pilerimport[6127]: 40000000596d3f9614ccda1c00399cf196a3: cannot open frame file: 40000000596d3f9614ccda1c00399cf196a3.m Jul 18 08:51:56 piler pilerimport[6127]: 40000000596d3f9614d91ad4001e062aa27d: cannot open frame file: 40000000596d3f9614d91ad4001e062aa27d.m Jul 18 08:51:56 piler pilerimport[6127]: 40000000596d3f9615a2eae400d5a39df61a: cannot open frame file: 40000000596d3f9615a2eae400d5a39df61a.m Jul 18 08:51:56 piler pilerimport[5512]: 40000000596d3f9616cd1ebc0034147024fc: cannot open frame file: 40000000596d3f9616cd1ebc0034147024fc.m Jul 18 08:51:56 piler pilerimport[5512]: 40000000596d3f9616d14d0c002baafcb1f4: cannot open frame file: 40000000596d3f9616d14d0c002baafcb1f4.m Jul 18 08:51:56 piler pilerimport[5512]: 40000000596d3f9616d4f68c004612065bff: cannot open frame file: 40000000596d3f9616d4f68c004612065bff.m

---Verison Info root@piler:~# piler -v piler 1.1.1, build 904, Janos SUTO [e-mail]

Build Date: Sun Sep 27 15:05:42 EST 2015 ldd version: ldd (Debian EGLIBC 2.13-38) 2.13 gcc version: gcc version 4.7.2 (Debian 4.7.2-5) Configure command: ./configure --localstatedir=/var --with-database=mysql --enable-starttls --enable-tcpwrappers

[Janos SUTO]

I think your description of what might have happened is accurate. I assume you have plenty of emails in gmail accounts to import, so it just proved too much for pilerimport. Unfortunately this highlights the shortcomings of importing via imap. Not sure what you do exactly, but you may run the import process manually, and see what's going wrong.

[supporthq]

Thanks Janos, It has settle down now after cleaning out the imap folder. Can I ask what are the *.m files? are they encoded messages and can I import them manually?

[Janos SUTO]

.m files are temporary files holding the message (with attachments extracted)