Start program for directory checks not executed anymore?

Issue #290 closed
t.koenig created an issue

I have the following checks in my configuration (simplified)

CHECK PROCESS nginx with pidfile /tmp/pids/nginx.pid
  start program = "..."
  stop program  = "..."
  depends on storage


CHECK DIRECTORY storage with path /Volumes/Storage
  start program = "..." as uid www and gid www
  stop program = "..."
  if does not exist for 1 cycle then alert

In monit 5.14 and earlier the volume was mounted automatically:

[CET Nov 25 12:00:34] info     : Starting Monit 5.14 daemon with http interface at [*]:2812
[CET Nov 25 12:00:34] info     : Starting Monit HTTP server at [*]:2812
[CET Nov 25 12:00:34] info     : Monit HTTP server started
[CET Nov 25 12:00:34] info     : 'app-staging.local' Monit 5.14 started
[CET Nov 25 12:00:34] info     : M/Monit heartbeat started
[CET Nov 25 12:00:34] error    : 'storage' directory doesn't exist
[CET Nov 25 12:00:34] error    : 'nginx' process is not running
[CET Nov 25 12:00:34] info     : 'nginx' trying to restart
[CET Nov 25 12:00:34] info     : 'storage' start: /bin/bash
[CET Nov 25 12:00:37] info     : 'nginx' start: /usr/local/nginx/sbin/nginx

as of 5.15 this doesn't seem to work anymore:

[CET Nov 25 12:06:27] info     : Starting Monit 5.15 daemon with http interface at [*]:2812
[CET Nov 25 12:06:27] info     : Starting Monit HTTP server at [*]:2812
[CET Nov 25 12:06:27] info     : Monit HTTP server started
[CET Nov 25 12:06:27] info     : 'app-staging.local' Monit 5.15 started
[CET Nov 25 12:06:27] info     : M/Monit heartbeat started
[CET Nov 25 12:06:27] error    : 'storage' directory doesn't exist
[CET Nov 25 12:08:29] error    : 'storage' directory doesn't exist

Does this by any chance have anything to do with #249 ?

Comments (7)

  1. t.koenig reporter

    The status of the process "nginx" keeps at "Initializing" and for storage "Does not exist".

    If I manually start the process "nginx" the storage start action gets executed too – it just seems to fail for the inital startup.

  2. Tildeslash repo owner

    The dependency implementation for monit <= 5.14 was just "soft" and it didn't guarantee, that the prerequisite service is up, before starting the dependent service.

    Monit 5.15+ implements hard dependency - if the prerequisite service has error, the child won't be started. For example if "database" depends on "filesystem", it won't make sense to start the database when its filesystem is not mounted.

    In your case if "nginx" can start while the "storage" directory doesn't exist, there is no real dependency and the "depends on storage" should be removed from the nginx configuration. If nginx will fail when the directory doesn't exist, then the dependency is correct.

    The "Initializing" state on nginx is set because it waits for the prerequisite to start - in debug mode monit logs information about the skip reason using one of the following messages:

    '<child>' test skipped as required service '<parent>' is <"initializing" | "not monitored">
    '<child>' test skipped as required service '<parent>' has errors
    

    The cascading of start action to prerequisite is correct - the child tries to start its parent first. Monit <= 5.14 did the same, but it didn't check the prerequisite is up.

  3. t.koenig reporter

    Thanks for your explanation, but still why isn't the start method called for storage?

    As from the documentation it says:

    If the directory does not exist, Monit will call the entry's start method if defined.
    
  4. t.koenig reporter

    Ok, I found the issue now.

    The if does not exist for 1 cycle then alert prevents the call for the start method, I guess this is an expected behavior?

    In monit 5.14 the start method was still called somehow through the dependency I guess.

  5. Tildeslash repo owner

    Yes, the "if does not exist for 1 cycle then alert" should be removed - the implicit action is "restart", which sends alert as well.

  6. Log in to comment