No records in Quarantine

[Gerben Wichers]

Since the update of the docker to Version 2.3.14 FREE #2347 2 weeks ago, the Quarantine tab https://x.x.x.x/admin/quarantine does only show “no records” but when I look into the quarantine folder on the filesystem, I see there are files in quarantine
Comparing this with folders that show quarantined records in the web interface there seem to be missing the .meta files.

‌

[Gerben Wichers]

After a reboot of the docker, from that moment on, the .meta files are being created.
Unfortunately the next day, they are not created anymore.

[Gerben Wichers]

Also after updating to 2.3.15 the quarantine interface is not filled.
The .meta files are also not created with this version in the quarantine folder.

[SH]

Hi Gerben, are you looking at the log files? Please have a look at https://poste.io/doc/logs

I need detailed transaction log of some mail that is quarantined (log/delivery/tx/...)

(quarantine list is tested every release and our instances seem to be ok, so I suspect your instance has something different)

[Gerben Wichers]

Hi SH,
Yesterday I've created a new docker instance and moved all mail into this.
This morning, I noticed the same problem again.
I've been looking a bit further into the folder and permission structure and found out that the /data/quarantine/{date} folder is created with the root:root ownership and with drwxr-xr-x permissions.
Also the files within this folder have the root:root ownership.
When I changed the ownership for the /data/quarantine/{date} folder to delivery:mail and drwxrwxr-x permissions, the .meta files are created again but with root:root ownerschip.

‌

[SH]

The haraka process runs with delivery:mail rights, so it looks to me impossible that process would create files as root

Do you use some special mounts for volumes, remote or fuse...something unusual?

[Gerben Wichers]

I'm running the docker on UnRaid. But don’t think that this should matter.

Here are all haraka processes running within the docker container

[SH]

What version of Docker do you have?
Are there any messages in the mail server's docker logs?
I've found that one of our instances have similar problem - older dockerd with statx bug, running with --priveleged flag helps, but solution is to upgrade dockerd to latest version

[Gerben Wichers]

At the moment UnRaid 6.12.5 comes with Docker version 20.10.24, build 297e128.
I don't see any messages in the docker log.
I will try the --privileged flag to see if this will help.

Or i'll wait for Unraid to come with an update with a newer version of Docker.
In the meanwhile, I have created a script that changes the ownership and permissions of the folder after it's created.

[SH]

fix #1022, ref #1018 get rid of statx enabled coretools interaction

→ <<cset c7f8142fe0e4>>

[SH]

Please try latest version, there was some changes which might affect your issue.

[Gerben Wichers]

I just updated to the latest version 2.3.16 FREE # 2422
And immediately I see that all files wihtin the quarantine folder are now have the correct owner (delivery:mail) with permissions (rw-rw-r--)

Will update you tomorrow after the new quarantine folder for 20231205 is created

[Gerben Wichers]

I did not really took a good look at the permissions of the folder, but the .meta files were not created.
I will get back on this tomorrow.

After this update there is a different issue.
I cannot sent mail through roundcube. (The roundcube web interface gives a “SMTP Error (): Authentication failed)
And in the Admin dashboard I get the following: 127.0.0.1 → 587 relay: acl guard helo: localhost important_response local_port: disconnected

[Gerben Wichers]

Yesterday I also upgraded to 2.3.17, but that version had some other problems.

So I went back to 2.3.16 and this morning the .meta files where not created.
The newly created folder does not seem to have the correct owner
drwxr-xr-x 2 root root 28 Dec 6 09:37 20231206/

[SH]

What other problems? (in 2.3.17) It is the same as .16 but the name of a config file has been fixed, see https://poste.io/changelog

Do you have any idea why and how a process running as delivery:mail can create root:root directories? Please try to look at logs of mail server and host machine. I am out of ideas as to where the problem might be.

[Gerben Wichers]

I'm mapping /opt/www/webmail/config/ and /opt/www/webmail/plugins/ to a local storage, so I'm able to add plugins to roundcube.
Which works like a charm.

But during updating to 2.3.17, the des_key value in config.inc.php got some additional characters at the end.
After stopping the docker and removing the characters it seems to be working except for sending mail from roundcube.

Looking in the logs about the quarantine folder, I found the following errors.

s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:05:09.696275500 [INFO] [B10253BB-63CB-4E27-9A9F-9C438D072EA2.1] [queue/quarantine] pass:/data/quarantine/20231206/B10253BB-63CB-4E27-9A9F-9C438D072EA2.1
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:05:09.699511500 [DEBUG] [-] [watch] pass=/data/quarantine/20231206/B10253BB-63CB-4E27-9A9F-9C438D072EA2.1 emit=true
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:05:09.708092500 [CRIT] [-] [core] Error: EPERM: operation not permitted, chmod '/data/quarantine/20231206'
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:12:51.406824500 [INFO] [31AB9217-A0E6-45D0-ACA5-BAED45699106.1] [queue/quarantine] pass:/data/quarantine/20231206/31AB9217-A0E6-45D0-ACA5-BAED45699106.1
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:12:51.410435500 [DEBUG] [-] [watch] pass=/data/quarantine/20231206/31AB9217-A0E6-45D0-ACA5-BAED45699106.1 emit=true
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:12:51.419295500 [CRIT] [-] [core] Error: EPERM: operation not permitted, chmod '/data/quarantine/20231206'
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:17:30.258795500 [INFO] [50EC3EA5-4534-465D-9284-E8439425706B.1] [queue/quarantine] pass:/data/quarantine/20231206/50EC3EA5-4534-465D-9284-E8439425706B.1
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:17:30.261830500 [DEBUG] [-] [watch] pass=/data/quarantine/20231206/50EC3EA5-4534-465D-9284-E8439425706B.1 emit=true
s6/haraka-smtp/@4000000065700f6b2ccdeebc.s:2023-12-06 06:17:30.270427500 [CRIT] [-] [core] Error: EPERM: operation not permitted, chmod '/data/quarantine/20231206'

maybe this will help figuring out why the meta files are not being created

[Gerben Wichers]

It seems like the config.inc.php files is regenerated after booting the docker image.
When I remove the /opt/www/webmail/config/ mount, I'm able to send mails through roundcube again.
Unfortunately, the 2-factor authentication plugin I added to make roundcube more secure, is not being loaded anymone.
This was what I added to the config.inc.php file.

[Gerben Wichers]

I seem to have overcome the mail sending problem through roundcube by mounting only the config.inc.php file in stead of the complete config folder.

[SH]

Issue #1023 was marked as a duplicate of this issue.

[SH]

Please try the latest, the permissions problem is an internal bug of NodeJS which I've downgraded until a fix comes.

[Gerben Wichers]

I've just updated this morning to 2.3.18.
I disabled my user script that changed the permission once the folder is created
Lets see tomorrow how it goes.

Thanks for all your time and effort you put in to debug this problem

[Gerben Wichers]

Sorry for the late response, but still have the problem that the daily created quarantine folder has the wrong permissions.

The files created within the folder are also owned by root:root

[SH]

Nodejs bug - now fixed. Please update to the latest version - if it does not fix itself please fix the permissions manually. If it is still a problem, please open a new ticket