monit 5.25+ crashes on alpine 3.10 while starting program

[fauno]

Hi, I've been chasing this issue all day and I found out it only happens on Alpine 3.10, tested with monit 5.25.3 which is the current release, 5.26 which is "edge" and also built 5.25.2 just in case. The issue dissapears on Alpine 3.9 with monit 5.23.2.

The issue is that a process monitoring makes monit fail immediately with exit status 162 (couldn't find any reference to it). Running verbose monit doesn't help, since monit closes immediately after trying to run the program and it only logs it's trying to start it. In another check, a program scheduled every 60 cycles also made monit crash, but changing it to every hour in cronjob syntax worked, though I haven’t debugged more on this since I was pressed to run the main program.

For context, I'm running a Ruby on Rails application on a Docker container and using monit as a supervisor. The container runs tini (a tiny init) as pid 1 and this runs monit. Then monit takes charge of starting services.

The program is a simple shell script that runs database migrations and then starts the Rails server. If I run this script manually inside the container it works fine and I can even monitor it with monit. But it only fails when monit tries to start it. I even tried monit 5.26 on my work distribution and the script runs fine and is correctly monitored.

The error code only happens on Alpine 3.10, so my guess is that it's some change between musl 1.1.20 and 1.1.22 (Alpine 3.9 and 3.10, respectively).

Should I report on Alpine instead? Thanks :)

[fauno]

Sorry, when I said “The issue dissapears on Alpine 3.9 with monit 5.23.2“ I mistyped, it’s 5.25.2

[Tildeslash]

It is difficult to say what the problem is as we don’t use this specific setup. It might be better to ask the Alpine guys.

[Henning Bopp]

I cannot reproduce this with Alpine 3.10 / Monit 5.25.3. I cannot even find the string "162" in Monit's source code...

Since I use the same approach on my containers (tini + Monit as initd), I was in a little fear... ;-)

So for testing I created a minimal debug container that you can find in my monit-858 repo. It contains a fake daemon running tail -f, creating a PID-File for monitoring and running tini/Monit.

Nothing unusual happening here. It's running, controlling is possible, log file is clean.

Could you...

• ... provide an environment where this is reproducible (Dockerfile + assets)?
• ... test if my testing environment also crashes?

[fauno]

Sure, will report to Alpine

@Henning Bopp thanks, will try! You can find the base monit image here: https://0xacab.org/sutty/containers/monit/ and one of the most failing rails here, though it happened with others: https://forja.lainventoria.com.ar/lunar/backend/ (important files: Dockerfile, monit.conf and entrypoint.sh). The failing service is entrypoint backend.

FWIW, since the downgrade everything has been stable and running for 19 hours.

[fauno]

Hi again, just for the record @Tildeslash , it’s an issue with musl’s implementation of vfork(), patching monit to fork() works correctly again.

[Tildeslash]

@fauno Thanks for the info.

[Tildeslash]

Duplicate of #867.