monit 5.25+ crashes on alpine 3.10 while starting program
Hi, I've been chasing this issue all day and I found out it only happens on Alpine 3.10, tested with monit 5.25.3 which is the current release, 5.26 which is "edge" and also built 5.25.2 just in case. The issue dissapears on Alpine 3.9 with monit 5.23.2.
The issue is that a process monitoring makes monit fail immediately with exit status 162 (couldn't find any reference to it). Running verbose monit doesn't help, since monit closes immediately after trying to run the program and it only logs it's trying to start it. In another check, a program scheduled every 60 cycles also made monit crash, but changing it to every hour in cronjob syntax worked, though I haven’t debugged more on this since I was pressed to run the main program.
For context, I'm running a Ruby on Rails application on a Docker container and using monit as a supervisor. The container runs tini (a tiny init) as pid 1 and this runs monit. Then monit takes charge of starting services.
The program is a simple shell script that runs database migrations and then starts the Rails server. If I run this script manually inside the container it works fine and I can even monitor it with monit. But it only fails when monit tries to start it. I even tried monit 5.26 on my work distribution and the script runs fine and is correctly monitored.
The error code only happens on Alpine 3.10, so my guess is that it's some change between musl 1.1.20 and 1.1.22 (Alpine 3.9 and 3.10, respectively).
Should I report on Alpine instead? Thanks :)
Comments (8)
-
reporter -
repo owner - changed status to closed
It is difficult to say what the problem is as we don’t use this specific setup. It might be better to ask the Alpine guys.
-
I cannot reproduce this with Alpine 3.10 / Monit 5.25.3. I cannot even find the string "162" in Monit's source code...
Since I use the same approach on my containers (tini + Monit as initd), I was in a little fear... ;-)
So for testing I created a minimal debug container that you can find in my monit-858 repo. It contains a fake daemon running
tail -f
, creating a PID-File for monitoring and running tini/Monit.Nothing unusual happening here. It's running, controlling is possible, log file is clean.
Could you...
- ... provide an environment where this is reproducible (Dockerfile + assets)?
- ... test if my testing environment also crashes?
-
reporter Sure, will report to Alpine
@Henning Bopp thanks, will try! You can find the base monit image here: https://0xacab.org/sutty/containers/monit/ and one of the most failing rails here, though it happened with others: https://forja.lainventoria.com.ar/lunar/backend/ (important files: Dockerfile, monit.conf and entrypoint.sh). The failing service is
entrypoint backend
.FWIW, since the downgrade everything has been stable and running for 19 hours.
-
reporter Hi again, just for the record @Tildeslash , it’s an issue with musl’s implementation of
vfork()
, patching monit tofork()
works correctly again. -
repo owner @fauno Thanks for the info.
-
repo owner - changed status to open
-
repo owner - changed status to duplicate
Duplicate of
#867. - Log in to comment
Sorry, when I said “The issue dissapears on Alpine 3.9 with monit 5.23.2“ I mistyped, it’s 5.25.2