Monitor Server Resources with Monit

Introduction

In this post I will intoroduce Monit and show a few ways how to monitor various server resources like Disk usage etc.

About Monit

Monit is a free and open source process monitoring tool that can be used to monitor system process.

You can monitor, for size and permission changes, following:

  • files,
  • directories,
  • devices.

Monit also able to monitor common services like:

  • Apache,
  • Nginx,
  • FTP,
  • SSH,
  • MySQL,
  • etc.

You can monitor remote hosts by:

  • TCP/IP ports,
  • protocols,
  • ping.

Monit has an ability to stop a specific process that has high resource usage and keep watch on your server and send alerts about any critical error conditions and recovery status.

Installation and Setup

This is how to install and configure Monit on a Debian/Ubuntu server.

Install monit:

By default, Monit is on Debian repository, and we can install it with command:

$ apt-get install monit

Enable and Start Monit

Before Monit is started the first time, you can test the control file for syntax errors:

$ monit -t

You should get

Output:
$ Control file syntax OK

If there was an error, Monit will print an error message to the console, including the line number and where the error was found.

After installatin start Monit service and enable it to start on boot time:

$ systemctl start monit
$ systemctl enable monit

After instalalation, we will add processes to config file

Configuring Monit

Monit is very easy to configure and files are created to be very easily readable and making them easier for users to understand.

Monit default configuration file monitrc is located at /etc/monit/ directory and other process and server specific configuration files are located at /etc/monit/conf.d/ directory. It is possible to include additional configuration parts from other files or directories.

$ nano /etc/monit/monitrc

Uncomment the lines in file from:

# set httpd port 2812 and
# use address localhost # only accept connection from localhost
# allow localhost # allow localhost to connect to the server and
# allow admin:monit # require user 'admin' with password 'monit'

Check if monit is running on port 2812

We will check port with netstat command.

$ netstat -tlnp | grep :2812

Display monit details

We can see status of all the services with the following command:

$ monit status

Monitoring Disk Usage

Adding file to conf.d to configurate monit for disk monitoring

$ nano /etc/monit/conf.d/disk

Adding script to disk file so we want to for monitoring disk space

check filesystem rootfs with path /
if space usage > 80% then alert

Monitoring ICMP

Add new file for ICMP monitoring

$ nano /etc/monit/conf.d/icmp

Add script to icmp file

 check host www.example.ba with address www.example.ba
       if failed
          icmp type echo count 30 with timeout 60 seconds
       then alert

Add Nginx service monitoring

We will add Nginx service. Add file for nginx monitoring

$ nano /etc/monit/conf.d/nginx

Add script to nginx file

check process NginX with pidfile /run/nginx.pid
       start "/etc/init.d/nginx start"
       stop  "/etc/init.d/nginx stop"
       if failed host 127.0.0.1 port 80 then alert
       if failed host 127.0.0.1 port 80 then restart
       if 5 restarts within 5 cycles then timeout

Add Memory and Cpu monitoring

Add file to monit conf folder:

$ nano /etc/monit/conf.d/ramcpu

Add script to ramcpu file:

check System domain.com
    if memory usage > 75% then alert
    if cpu usage (user) > 80% then alert
    if cpu usage (system) > 30% then alert
    if cpu usage (wait) > 20% then alert

Once you’ve configured all programs for monitoring, check monit syntax for errors. If found any errors fix them, it’s not so tough to figure out what’s went wrong.

$ monit -t

Output:
$ Control file syntax OK

You can verify that monit service is started by checking log file.

$ tail -f /var/log/monit.log

Setting up email notifications

Setting account because we want to sent notification mails:

set mailserver smtp.gmail.com port 587
    username "username@mail.com" password "pasword"
    using tlsv1
    with timeout 30 second

Formatting mail

set mail-format {
   from: username@mail.com
   subject: monit alert --  Server_name
   message: $EVENT Service $SERVICE
                 Date:        $DATE
                 Action:      $ACTION
                 Host:        $HOST
                 Description: $DESCRIPTION

            Sent from Server_name
 }

set alert (mail of recipient)

Reload monit

$ systemctl reload monit.service

Example: Testing disk usage

To test Monit we have to generate big files until the disk usage is over 80%

$ cat /dev/urandom > big_file

or

$ fallocate -l 50G big_file

This way, we manually triggered the monit disk usage check and we should get an email notification about disk usage being over 80%.