failed to start -- could not start required services

[Philip Foulkes]

I'm running Monit 5.23.0, but also observed this in Monit 5.22.0. We run services that use Redis

CHECK PROCESS redis WITH PIDFILE /var/run/redis/redis-server.pid
   START PROGRAM = "/bin/systemctl start redis"
   STOP PROGRAM  = "/bin/systemctl stop  redis"
   IF DOES NOT EXIST FOR 4 CYCLES THEN ALERT
   IF DOES NOT EXIST THEN START
   IF TOTAL CPU > 5% FOR 4 CYCLES THEN ALERT
   IF TOTAL MEMORY > 10.0 MB FOR 4 CYCLES THEN ALERT

I use the CPU and memory tests, and alert if they are greater than the configured thresholds (I set the above thresholds low to reproduce the problem).

If I view the status of Redis, the status is Resource limit matched

sudo monit status redis
Monit 5.23.0 uptime: 58m

Process 'redis'
  status                       Resource limit matched
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    laststate
  pid                          1351
  parent pid                   1
  uid                          123
  effective uid                123
  gid                          132
  uptime                       1h 51m
  threads                      3
  children                     0
  cpu                          0.1%
  cpu total                    0.1%
  memory                       0.3% [54.2 MB]
  memory total                 0.3% [54.2 MB]
  disk read                    0 B/s [32 kB total]
  disk write                   0 B/s [4 kB total]
  data collected               Fri, 09 Jun 2017 10:34:05

In some instances, I put dependencies on Redis

CHECK PROCESS evt_logger with PIDFILE /opt/vservices/chekae/ngf/share/pid/ngf_evt_logger.00.pid
    DEPENDS ON redis
    .
    .
    .

If I restart the process evt_logger, it will not start as it looks like Monit thinks Redis is not running, whereas it is running. I suspect that Monit thinks that Redis is not running because it's alerting on memory usage.

[SAST Jun  9 10:15:29] error    : 'evt_logger' failed to start -- could not start required services: 'redis'
[SAST Jun  9 10:15:29] info     : 'evt_logger' start action failed
[SAST Jun  9 10:15:29] error    : Aborting event

[Tildeslash]

yes, if you set the dependency, the dependant services wait for the parent to have clean/OK state before starting themselves

[Tildeslash]

The dependency is implemented on purpose so, that children wait for the parent to be OK.

We plan to add error levels support in the future, so it'll be possible to continue if the parent will have just "warning" level errors.

[Philip Foulkes]

Thanks, that would be great. In the production environment, we can't easily restart Redis, and don't want to disable the alerting as it's quite useful.