Failed to create "/data/domains/domain.com/alias/Maildir/cur": mkdir(): No space left on device.

[Eric]

hello,

All was working just fine for months, and today i just tried to create a new alias and got this error:

Oops! An Error Occurred

The server returned a "500 Internal Server Error".

Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

Failed to create "/data/domains/domain.com/alias/Maildir/cur": mkdir(): No space left on device.

Symfony\Component\Filesystem\Exception\ IOException

in vendor/symfony/symfony/src/Symfony/Component/Filesystem/Filesystem.php (line 104)

‌

The host filesystem / was at 99% because of journal logs so i purged lots of stuff, then restarted the container but no luck - same error when trying to create an alias. other functions seem to work just fine and i can send/receive emails

Other FS are indeed NOT full: /data2 hosts docker files, and poste.io is installed in /email
Logical_Volume mB_Total mB_Used mB_Free Use% Mounted_on
nvme6n1p1 7,877 6,463 1,399 83% /
nvme5n1 1,952 1,730 104 95% /data
nvme4n1 31,118 20,974 8,695 71% /data2
udev 970 0 970 0% /dev
tmpfs 981 20 962 3% /dev/shm
nvme3n1 3,968 2,821 947 75% /email

The space reported inside the container (if that matters??) also does not look good:

docker exec -it mailserver bash
root@mail:/# df -m
Filesystem 1M-blocks Used Available Use% Mounted on
overlay 31118 20974 8695 71% /
tmpfs 64 0 64 0% /dev
tmpfs 981 0 981 0% /sys/fs/cgroup
shm 64 0 64 0% /dev/shm
/dev/nvme3n1 3968 2822 946 75% /data
/dev/nvme6n1p1 7877 6461 1400 83% /usr/share/zoneinfo/Etc/UTC
/dev/nvme4n1 31118 20974 8695 71% /etc/hosts
tmpfs 981 0 981 0% /proc/acpi
tmpfs 981 0 981 0% /proc/scsi
tmpfs 981 0 981 0% /sys/firmware

I even force update the container but was already at the latest version.

reboot the host server? the load/ram/swap does look good as usual:

I got this line in the container log after reinstalling it tho:

rsyslogd: file '/var/log/mail.err': open error: No space left on device [v8.2002.0 try https://www.rsyslog.com/e/2433 ]

So i deleted all the gz logs under /var/log inside the container and restarted again and… it worked this time! I could create an alias

The space used by / did not change after i deleted all the log history:

docker exec -it mailserver bash
root@mail:/# df
Filesystem 1K-blocks Used Available Use% Mounted on
overlay 31864752 21880028 8500352 73% /

===> what caused this? the FS in both the host and the container were NOT full 100% at any time. is there a setting somewhere looking for free space under / to me at least xx% or is there limit on the number of rotation logs in the container?

‌

[Jan Panoch]

hello,

i had similiar problem and mayber found a solution.

try to look to directory /opt/www/webmail/logs/

if there are a lot of files (about 2 milion in my case) that’s the problem. Those are logs from webmail, which are not directed to /data/log/ directory as others logs are, but stay in container image and the image grows, docker is doing diffs and merges and consuming a lot of file inodes.

My solution which i will suggest to author is to make directory /data/log/webmail and symlink /opt/www/webmail/logs to it.

‌

docker exec -it mailserver bash
root@mail:/#
mkdir /data/log/webmail
chown www-data.www-data /data/log/webmail
rm /opt/www/webmail/logs
ln -s /data/log/webmail /opt/www/webmail/logs

And then restart container.

This moves the log files outside container and it looks fine.

Jan

[SH]

fix #762 move roundcube logs outside container

→ <<cset 963c21b38fbb>>