- changed title to Messages discarded with status=duplicate
- edited description
Messages discarded with status=duplicate
Hello Janos,
Here is our instance -
piler 1.3.8, build 998, Janos SUTO sj@acts.hu
Build Date: Tue Mar 31 20:52:54 AEDT 2020
ldd version: ldd (GNU libc) 2.17
gcc version: gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)
OS: Linux mailarchiver.ljhfairfield.com.au 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Configure command: ./configure --localstatedir=/var --with-database=mysql --enable-clamd --enable-memcached
MySQL client library version: 10.4.13
Extractors: /usr/bin/pdftotext /usr/bin/catdoc /usr/bin/catppt /usr/bin/xls2csv /usr/local/bin/ppthtml /usr/bin/unrtf /usr/bin/tnef libzip
Sphynx - 3.2.1
Without any reasons once in a while incoming messages are not indexed any longer and discarded with status=duplicate
Stored messages have a valid references in maillog, those discarded have reference field empty.
-
Jul 24 10:23:05 mailarchiver piler[85193]: 1/3GNHDFS1WDNVKX83: 400000005f1a29f316118b34000000007043, size=1527813/0, attachments=0, reference=, message-id=null, retention=0, folder=0, delay
=0.0114, status=duplicate
2. Jul 24 10:26:06 mailarchiver piler[87353]: 1/SHPI8W49O9WAV6D0: 400000005f1a2aa812ce27d400ded9fa1425, size=59435/10008, attachments=9, reference=<ME4P282MB08232B1B8A04099BF16CC429E2770@ME4P2
82MB0823.AUSP282.PROD.OUTLOOK.COM>, message-id=00440B6D16AE9534CD906448F7770@SYBP282MB0044.AUSP282.PROD.OUTLOOK.COM>, retention=2557, folder=0, delay=0.0927, status=stored
If I restart piler daemon messages begin to be stored again.
I don’t quite understand what’s going on and why piler stops storing the messages. Do you have any idea where the problem comes from ?
Thank you !
Comments (10)
-
reporter -
repo owner Hello Veaceslav, I suspect that those problematic emails has no Message-ID lines in the header. Can you check this theory based on the delivery date and perhaps size?
-
reporter Hello Janos, the problem is that after the first such discarded message only piler restart helps. I usually find about this only via gui statistics.
Anything else we can think about ? Because if we get any wrong message (zero sized or with wrong headers), piler should discard it and continue to store other “good“ messages, but it does not. Everything else which comes is automatically discarded with duplicate status and piler restart allows to store input messages again. I’ll talk with admin who sends messages to check your theory.
Thanks ! -
reporter Hello Janos,
I have located a few of those messages that have been discarded by piler, started from the first one. They all had proper headers, including message_id.
Any other clues of why piler refuses to store messages at some point ?
Thanks. -
repo owner Odd. Check if mysql daemon works properly. Look for any errors in the mysql logs, also check if you have big enough innodb buffers, etc. The FAQ http://www.mailpiler.org/wiki/faq has some starter settings. Don’t be afraid to allocate more memory if you have the resources.
-
reporter Thanks, Janos.
I’m going to play with mysql and come back with the results.
Please don’t close the ticket till then :)
-
repo owner OK, I won’t :-)
-
repo owner Any update on the matter?
-
reporter Hello Janos,
I think you can close the ticket. It seems to be a problem on a client side.
Thank you for your support !
-
repo owner - changed status to wontfix
- Log in to comment