Monitoring and automatic recovering of server services
A list of web monitoring services were listed on this blog 3 weeks. As written previously, those services are very simple to setup, free, and provides a quick way to be alerted whenever your website is unaccessible from your visitors. They also have the added advantage of working with any website, regardless if the website is on a shared, virtual or dedicated server.
For those who have dedicated servers, or those who require an additional layer of security, the next step is generally to install and setup a monitoring software on the server. The monitoring software will have the task of pinging key services every minute (or less, depending on your preferences), and will recover them if there’s any trouble.
Note: the following only applies to iWeb customers managing a Linux dedicated or managed servers
If you want a bare-bone auto-recovery service, runit is a standard Linux service, which will track processes running on a Linux machine. If the process dies, runit will automatically restart it. Runit is relatively simple to setup, takes little to no resources on a server, although a few might be discouraged by the command line interface. See a list of services and software that can be managed by runit.
The next step up after runit are software which monitor CPU usage, memory consumption, port availability. One of the simplest you can find is god. God takes very simple config files, with rules on CPU usage and memory consumption limits, and also notification tools. One of its biggest drawbacks is that it uses Ruby, which means you have to install Ruby and other related libraries such as Ruby Gems, and other side-effects such as high memory consumption just for monitoring. Still, if you have a web application running on Ruby on Rails or Merb, god is a no-brainer. See for instance the code snippet here:
w.restart_if do |restart|
restart.condition(:memory_usage) do |c|
c.above = 150.megabytes
c.times = [3, 5] # 3 out of 5 intervals
end
restart.condition(:cpu_usage) do |c|
c.above = 50.percent
c.times = 5
end
end
Much more friendly than runit, isn’it?
Of course, the problem with runit or god is that they might not fit your definition of a monitoring software. Many think of a dashboard, statistical data, plus a graphical UI to monitor all servers. There are 2 choices here: Nagios or Monit. Both are open source software, with Nagios being the most complex, with an extensible plugin architecture, and all the features you might want from a monitoring service (such as historical data, emailing, paging, issue escalation, etc.). On the other hand, Monit is easier to install and setup. See screenshots below of M/Monit:

As other software above, M/Monit or Nagios will monitor any service on your server, with elaborate configurable scenarios in the case of problems. They also add a friendly user interface, plus historical data that can be used for future technological choices.
If you are using any other monitoring software, do leave a comment below.

Blog
Forum
Status

November 10th, 2008 6:24 pm
Very good post ! Congrats Heri :-)
November 11th, 2008 12:53 am
[...] See the original post [...]
November 11th, 2008 12:04 pm
Thanks! Hope this will be useful for webmasters and sysadmins
December 1st, 2008 5:51 pm
[...] of your immediate next steps should be writing detailed procedures on how to handle server issues, cope with growth in traffic, website software development, or even cases like attacks from [...]
January 16th, 2009 10:31 am
[...] Monitoring and Automatic Recovery of server services [...]
January 16th, 2009 10:32 am
[...] Surveillance et Reprise automatisé des services web [...]
October 29th, 2009 8:50 pm
Hello,
I just found that Iweb does not support Mmonit. Could you please write down a tutorial on how to install these software on a CentOs server?