SSH failes to restart by monit

Issue #59 resolved
Alexander Litvak
created an issue

I am seeing that when sshd needs to be restarted by monit. It is really successful in starting the process but for whatever reason, start command fails to generate a pid file in /var/run/. As the result monit finds process non existent and attempts so to start it again and again.

Here is my config. I attempted to output all information into a file, however the only information I see is sshd started OK.

check process sshd with pidfile "/var/run/sshd.pid"
   #start program = "/etc/init.d/sshd start"
   #stop  program = "/etc/init.d/sshd stop"
   start program = "/bin/bash -c '/etc/init.d/sshd start >>/tmp/sshd.output 2>&1'"
   stop program = "/bin/bash -c '/etc/init.d/sshd stop >>/tmp/sshd.output 2>&1'"
   if failed port 22 protocol ssh then restart
   if 5 restarts within 5 cycles then timeout

I am not sure what causes ssh script to skip writing pid file.

Comments (12)

  1. Tildeslash repo owner

    Is this really a Monit problem? Maybe a permission problem or that /etc/init.d/sshd write the pid file to another location than /var/run/sshd.pid. You should also check that you can run '/etc/init.d/sshd start ' from the console as the same user you can run Monit with.

  2. Alexander Litvak reporter

    Running monit in debug mode. Initiating sshd restart from monit web interface.

    'sshd' restart on user request
    monit daemon with PID 30584 awakened
    Awakened by User defined signal 1
    'sshd' trying to restart
    'sshd' stop: /etc/init.d/sshd
    Stopping sshd:                                             [  OK  ]
    pidfile '/var/run/sshd.pid' does not exist
    pidfile '/var/run/sshd.pid' does not exist
    'sshd' start: /etc/init.d/sshd
    pidfile '/var/run/sshd.pid' does not exist
    Starting sshd:                                             [  OK  ]
    'sshd' restart action done
    

    I am confused why pid file is not written. I read that some one blamed it on sshd starting before sshd stopped but this doesn't look like the case here.

  3. Alexander Litvak reporter

    Restart from console works great. When I restart file is updated. It definitely writing the file to the right place and all operated as root.. Monit was running as root in debug mode and in the init mode. SSHD starts and runs as root.

    I am on Scientific Linux 6.5 which a redhat clone.

  4. Alexander Litvak reporter

    Sorry for adding more confusion. It looks like using sync patch (synchronous stop , and I know it is not going to be implemented in the main code) works. I.e. I can restart the process from web and it produces sshd.pid in the right place. I am going to run more testing tonight just to make sure I am not loosing my mind.

  5. Tildeslash repo owner

    You can also try the new restart option which let the script control how to do restart:

    check process sshd with pidfile "/var/run/sshd.pid" 
           start program = "/etc/init.d/sshd start"
           stop program = "/etc/init.d/sshd stop" 
           restart program = "/etc/init.d/sshd restart"
           if failed port 22 protocol ssh then restart 
             ...
    
  6. Alexander Litvak reporter

    I tried restart feature and it works, at least when I issue restart from the monit webserver I have process back up with pid file updated. There are couple of minor issues

    When I originally ran restart with restart command then I used to get this e-mail

    PID changed Service sshd 
    
        Date:        Tue, 10 Jun 2014 16:04:44
        Action:      alert
        Host:       xxxxxxxxx
        Description: process PID changed from 5734 to 11238
    
    Your faithful employee,
    Monit
    

    Now I get

    PPID changed Service sshd 
    
        Date:        Tue, 10 Jun 2014 16:04:44
        Action:      alert
        Host:       xxxxxxx
        Description: process PPID changed from 0 to 1
    
    Your faithful employee,
    Monit
    

    When I comment out restart command and rely on stop / start then I get no alert e-mail at all.

  7. Log in to comment