Problem with receiving some emails

[TOmi]

I have Piler 1.3.4 build 994 , it has been working correct without any problems since one year . From some time I noted problems with 3 email address , I have 1000 account in system. When mail system try send email (always_bcc) to piler form one of three account I have error on mail server :

“Reason: lost connection with piler.server.org[192.168.x.x] while sending end of data – message may be sent more than once“

on piler system error log looks like

Mar 25 12:58:05 vPiler piler-smtp[30749]: disconnected from x.x.x.x on descriptor 11 (1 active connections)
Mar 25 12:58:05 vPiler piler-smtp[30749]: client x.x.x.x timeout

mail server is Zimbra 8.8.15

I was trying to adjust piler by changing parameters, but id doesn’t help

smtp_auth from 60 to 90

max_connections from 64 to 128

helper_timeout from 20 to 30

smtp_timeout from 90 to 120

worker_process from 4 to 6

Only 3 account form 1000 has problems !

[Janos SUTO]

Download the latest master branch, and build it. Then upgrade only the piler-smtp binary, and let’s see if it solves the issue.

[TOmi]

it is a production server , new version without test is not good idea, everything was working , everything except 3 account is working now. What was happened ?

[Janos SUTO]

Well, you may test it for sure. Anyway the provided data is not enough for conclusions. Set verbose logging in piler.conf, verbosity=5, and let’s see what happens. My bet is that zimbra’s postfix has 3 messages in the queue giving a hard time for piler-smtp. Probably piler-smtp doesn’t recognize the end of the message. You may even try sniffing the network on port 25 between zimbra and piler. If you can release just those 3 messages somehow.

[TOmi]

I will increase log level. It is not about 3 messages it is about any messages from 3 users

[TOmi]

New clue, yesterday i created new user in email system , now I have problem with his email too. Maybe I have problem some problem with database limits?

Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 92615
Server version: 10.1.37-MariaDB MariaDB Server

+--------------------+---------------+
| DB Name | DB Size in MB |
+--------------------+---------------+
| information_schema | 0.2 |
| mysql | 0.7 |
| performance_schema | 0.0 |
| piler | 8720.6 |
+--------------------+---------------+

‌

[root@vPiler /]# cat /etc/my.cnf.d/server.cnf

[mysqld]

#############################################################

### piler optimilization

innodb_buffer_pool_size = 256M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=64M
innodb_log_file_size=64M
innodb_read_io_threads=6
innodb_write_io_threads=6
innodb_log_files_in_group=2

innodb_file_per_table

Today I will add logfile to database and maybe I will see some error

[Janos SUTO]

Well, definitely you need to give mysql enough resources. You have ~8 GB mysql data. If you can, then double the innodb buffer sizes you just copied. These values in the piler FAQ present merely some starter values, feel free to increase them as needed.

[TOmi]

All didn't deliver messages are stored in directory /var/liler/tmp, what is common, all has this part inside :

X-Amavis-Alert: BAD HEADER SECTION, MIME error: error: unexpected end of preamble

I don’t use antyvirus in my sysytem so it probably comes from users computers,

I’ve changed sql optoins:

innodb_buffer_pool_size = 512M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=128M
innodb_log_file_size=128M
innodb_read_io_threads=6
innodb_write_io_threads=6
innodb_log_files_in_group=2

innodb_file_per_table
log_error = /var/log/mysql/error.log

And restart all piler’s processes, problem still occour. Databese’s error log is ok, no errors.

‌

In pler’s log with parameter verbosity=5 I found it:

Apr 3 07:34:53 vPiler piler-smtp[5910]: connected from xx.yy.zz.aa:38120 on descriptor 6 (active connections: 1)
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: EHLO http://mail.MYDOMAIN.COM#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 250-vpiler.MYDOMAIN.local#015#012250-PIPELINING#015#012250-STARTTLS#015#012250-SIZE#015#012250 8BITMIME
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: STARTTLS#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: starttls request from client
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 220 Ready to start TLS
Apr 3 07:34:53 vPiler piler-smtp[5910]: waiting for ssl handshake
Apr 3 07:34:53 vPiler piler-smtp[5910]: SSL_accept() result, rc=-1, errorcode: 2, error text: error:00000000:lib(0):func(0):reason(0)
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: EHLO mail.MYDOMAIN.COM#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 250-vpiler.MYDOMAIN.local#015#012250-PIPELINING#015#012250-SIZE#015#012250 8BITMIME
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: MAIL FROM:[e-mail] SIZE=17178#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 250 Ok
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: RCPT TO:[e-mail]#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 250 Ok
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: RCPT TO:[e-mail]#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 250 Ok
Apr 3 07:34:53 vPiler piler-smtp[5910]: processing command: DATA#015#012
Apr 3 07:34:53 vPiler piler-smtp[5910]: sent: 354 Send mail data; end it with <CRLF>.<CRLF>

‌

[Janos SUTO]

Then perhas either the piler-smtp or the piler daemons have some issues. Can you send me a non-sensitive mail from one of these 3 users? If possible then from the mail server, then I’d check it in my test environment.

‌

[Janos SUTO]

No news is good news. Let me know if there's any new input on the matter you can share with me.

[[e-mail]]

It's great that I found this information. I practice email marketing and often do bulk mailing to a large email database and now I know how to prevent interruptions in the process. By the way, can you tell me who knows which program is the safest to use on a pc for mass mailing of emails?