SANJAY WILLIE'S Human Language. Asterisk | Nagios | OpenSource | Microsoft

Version 1.1 – 07 June 2016

NOTES:

Copyright and registered trademarks are the properties of their respective companies/individuals
By all means, these are not my own guides but a collection of guides online with my own hacks here and there that didn’t work for me and now they do. THANK YOU ALL THOSE RESPECTABLE BLOGGERS FOR THEIR CONTRIBUTION
I do not provide any warranty whatsoever for using this guide or the OVA2 images herein
This guide doesn’t include how to make Nagios work for you, that’s something you need to learn this guide is just for how to get Nagios working, period.
Copy paste as single lines unless otherwise told, the – denotes to run on cli, copy paste them line by line after the dash space (- ), e.g. – apt-get update, copy “apt-get update” without the quotes and paste into your putty/ssh session.

Software & versions:

Nagios Core 4.1.1
Nagios plugins including community add-ons and check_nrpe
MRTG graphing engine to monitor Nagios itself.
PNP4Nagios 0.6 with RRD
NagiosQL 3.20 for Web GUI to edit and manage Nagios configs

VM Edition download:

If you’re not interested to do all of this below yet want to get Nagios 4.1.1 up and running, then download the OVA2 format VM image from Sourceforge: https://sourceforge.net/projects/debiannagios. Be sure to read the notes there.

Build by hand guide:

Download and install the latest iteration of Debian 7 64bit netinst here: http://cdimage.debian.org/cdimage/archive/7.10.0/amd64/iso-cd/debian-7.10.0-amd64-netinst.iso
Update and install a couple of packages, at one point the mrtg installation will a question, just press enter to continue.

- apt-get update
- apt-get upgrade

- apt-get install --force-yes apache2 build-essential libgd2-xpm-dev libssl-dev exim4 heirloom-mailx wget apache2-utils curl daemon apt-file libnet-snmp-perl libperl5.14 libpq5 libfreeradius-dev libfreeradius2 libsensors4 libsnmp-base libtalloc2 libtdb1 libwbclient0 samba-common samba-common-bin smbclient snmp mrtg libmysqlclient-dev libcgi-pm-perl librrds-perl libgd-gd2-perl python build-essential ssh sudo expect linux-headers-`uname -r` curl sox apache2 libssl-dev libncurses5-dev bison subversion libnewt-dev libcurl4-openssl-dev libnet-ssleay-perl openssl libauthen-pam-perl libio-pty-perl vim iftop tcpdump iptraf ngrep strace ltrace lsof htop sysstat nmap dstat powertop ntpdate ntp ssh libdbi-perl libhtml-template-perl libnet-daemon-perl libterm-readkey-perl mysql-client mysql-common psmisc ethtool apt-show-versions libapt-pkg-perl libmyodbc whois libusb-dev libdigest-md5-file-perl uuid-dev uuid screen autoconf automake git module-init-tools iotop iftop nmap unzip checkinstall libcgi-pm-perl librrds-perl libapache2-mod-python libapache2-mod-php5 php5-sqlite php5-mcrypt libgd2-xpm-dev libdbi1 libdbi-dev libapache2-mod-proxy-html snmp php5 libsnmp15 libnagios-plugin-perl php5-gd graphviz graphviz-dev php5-mysql sqlite3 libssh2-php fping smokeping screen
Create users and groups (this use will also be used for apache, explain why later), place a password for user nagios, enter it twice

- useradd nagios && passwd nagios
- groupadd nagios
- usermod -a -G nagios nagios
- usermod -a -G nagios www-data
Create some directories

- mkdir /usr/local/nagios
- mkdir -p /usr/local/nagios/share/{stylesheets,images}
- chown -R nagios:nagios /usr/local/nagios
Get Nagios core 4.11 and Nagios plugins

- cd /usr/src
- wget http://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.1.1/nagios-4.1.1.tar.gz
- wget http://www.nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz
- tar -xzvf nagios-4.1.1.tar.gz
- tar -xzvf nagios-plugins-2.1.1.tar.gz
Configure, make Nagios core

- cd nagios-4.1.1/
- ./configure --prefix=/usr/local/nagios --with-nagios-user=nagios --with-nagios-group=nagios --with-command-user=nagios --with-command-group=nagios
- make all
- make install
- make install-init
- make install-commandmode
- make install-config
- make install-exfoliation
- make install-webconf
Copy over some scripts (useful for SNMPTT, explained later)

- cp -R contrib/eventhandlers/ /usr/local/nagios/libexec/
- chown -R nagios:nagios /usr/local/nagios/libexec/eventhandlers
Create your nagiosadmin user!, this will be the user you will use to login to the core web console via http://.
- htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
Nagios init file doesn’t fulfil some LSB requirements, so edit the init file and add two lines like this

- nano /etc/init.d/nagios

# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
Alright, let’s restart apache for nagios web to work, set nagios to start automatically as well. Also, start the nagios core service. At this point, Nagios core is already installed, you can head over to http://<yourip>/nagios . Note, you may get check errors from the local services (localhost) that’s being monitored, that’s because we’ve not installed nagios plugins, yet. As long as you can see the webUI, that’s fine for now.

- a2ensite nagios
- service apache2 restart

- ln -s /etc/init.d/nagios /etc/rc2.d/S20nagios
- update-rc.d nagios enable

- service nagios start
Alright, let’s get the plugins installed now, after this is done, the web should show all plugins showing OK by right. You can recheck that if you want.

- cd ..
- cd nagios-plugins-2.1.1
- ./configure --with-nagios-user=nagios --with-nagios-group=nagios --with-openssl=/usr/bin/openssl --enable-perl-modules --enable-libtap
- make && make install
Now, we will download and install NRPE just to get that binary to be used in almost all nagios agent based checks in your near future. After compiling, we will copy the binary to the libexec folder of Nagios.

- cd ..
- wget http://kent.dl.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.15/nrpe-2.15.tar.gz
tar -zxvf nrpe-2.15.tar.gz
- cd nrpe-2.15

- ./configure --with-ssl=/usr/bin/openssl --with-ssl-lib=/usr/lib/x86_64-linux-gnu
- make
- cp src/check_nrpe /usr/local/nagios/libexec/
You could also add some apt based contributed plugins like this below. This step is completely optional, though recommended. Note, this may install many other dependencies.

- apt-get install nagios-plugins-contrib
- cp -r /usr/lib/nagios/plugins/* /usr/local/nagios/libexec/
Now, let’s get MRTG configured

- cp /usr/src/nagios-4.1.1/sample-config/mrtg.cfg /usr/local/nagios/etc/
- mkdir /usr/local/nagios/share/stats

Edit the mrtg.cfg file and add a line right on the top of the file
- nano /usr/local/nagios/etc/mrtg.cfg

Add this to top of that file, save and exit.

WorkDir: /usr/local/nagios/share/stats

Run these at the command line.
- env LANG=C mrtg /usr/local/nagios/etc/mrtg.cfg
- indexmaker /usr/local/nagios/etc/mrtg.cfg --output=/usr/local/nagios/share/stats/index.html

Now, we need to add the mrtg graphing to cron to run
Create and edit a new crontab file, like below;

- nano /etc/cron.d/nagiostats

Paste this into that file, save and exit.
*/5 * * * * root env LANG=C /usr/bin/mrtg /usr/local/nagios/etc/mrtg.cfg
Now, let’s go get pnp4nagios installed!, get rrdtool, download pnp4nagios and do the backend setup.

- apt-get install rrdtool
- cd /usr/src
- wget -O pnp4nagios-0.6.25.tar.gz http://downloads.sourceforge.net/project/pnp4nagios/PNP-0.6/pnp4nagios-0.6.25.tar.gz?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fpnp4nagios%2F&ts=1464497035&use_mirror=pilotfiber

- tar -zxvf pnp4nagios-0.6.25.tar.gz
- cd pnp4nagios-0.6.25
- ./configure --with-rrdtool=/usr/bin/rrdtool

- make all
- make fullinstall
- a2enmod rewrite
- service apache2 restart
Now, let’s enable a cool pnp4nagios popup graph like you see in NagiosXI

- cp contrib/ssi/status-header.ssi /usr/local/nagios/share/ssi/
- chown nagios:nagios /usr/local/nagios/share/ssi/status-header.ssi
- chmod 644 /usr/local/nagios/share/ssi/status-header.ssi
At this point, you need to access the pnp4nagios website http://<yourIP>/pnp4nagios and proceed with the instructions therein. Basically, everything should appear in green i.e. ALL GOOD, then, you can delete this install file

- rm /usr/local/pnp4nagios/share/install.php
Now, we will attempt to add a few definitions inside Nagios core configs to show our graphs, please follow these steps carefully and repeat for other services that produce performance data.

- nano /usr/local/nagios/etc/nagios.cfg

Paste the following at the end of the file. Everything is a single line

process_performance_data=1
service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=process-service-perfdata-file
host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=process-host-perfdata-file
Next, enable the perf data command file

- nano /usr/local/nagios/etc/objects/commands.cfg

Paste the following at the end of that file as well

define command {
        command_name process-service-perfdata-file
        command_line /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
        }
define command {
        command_name process-host-perfdata-file
        command_line /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
        }
Here’s an important note, we need to enable these graphs inside either hosts or services that produces performance data. Usually, hosts will always produce such data if you use the standard host check commands, however, services depends on how it is configured and what the plugin returns. In our example, we will use localhost (the nagios server itself) as hosts and current user count as a service in which we want performance graphs to be generated.

- nano /usr/local/nagios/etc/objects/localhost.cfg

Under host localhost, locate hostname localhost, below address 127.0.0.1, add a line like this and as shown in the example below that as well.. This can be done for all hosts!

action_url /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=_HOST_' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=_HOST_

Now, locate the “current users” service definition add a line like this as shown in the example below that as well. This can be used for all services that has performance data.
action_url /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=$SERVICEDESC$' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=$SERVICEDESC$
Now, let’s verify if we have done the above correctly or not by running the pre-flight nagios check like this

- /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

It should show zero errors and zero warnings. If it doesn’t, you’ve made a typo or an error with the steps above.
Now, if all goes well as above, restart nagios process and start the npcd process, which is the nagios performance data service, we will also set the npcd to start at boot, after the restart, go back to the webpage and you should see some extra icons appearing infront of “localhost” and “current users”. Do not panic if you see errors or no performance data, give it like 10 to 30 minutes for it to generate some data.

- service npcd start
- update-rc.d npcd enable

- service nagios restart
Go ahead and add the same service and host perf data action url to other hosts or services you’ve created and perf graphs will appear in the main nagios page. If you don’t want to show it on that page, they will be automatically generated and you can access them via http://<yourIP>/pnp4nagios
Now, let’s install NagiosQL, one of the many tools out there to perform edits on Nagios configs without having to touch text editors and what not.
- cd /usr/src
- wget -O nagiosql_320.tar.gz http://downloads.sourceforge.net/project/nagiosql/nagiosql/NagiosQL%203.2.0/nagiosql_320.tar.gz?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fnagiosql%2Ffiles%2Fnagiosql%2FNagiosQL%25203.2.0%2F&ts=1464465362&use_mirror=ufpr

- tar -zxvf nagiosql_320.tar.gz

- cp -R nagiosql32 /var/www/nagiosql
- chown -R nagios:nagios /var/www/nagiosql
Edit the timezone in php.ini. This is an important step to ensure NagiosQL knows when files were edited etc. Locate the date.timezone definition and define it by uncommenting it. Refer to this for a list of countries and how to write that definition. In my case, i am of course going to set it to Kuala Lumpur, Malaysia, where i live.

- nano /etc/php5/apache2/php.ini

date.timezone = Asia/Kuala_Lumpur

- service apache2 restart
Create some directories in which NagiosQL will write all config files.

- mkdir /usr/local/nagios/etc/nagiosql
- chown -R nagios:nagios /usr/local/nagios/etc/nagiosql

Now, let’s create a NagiosQL apache2 definition

- nano /etc/apache2/sites-available/nagiosql

Paste, this below, save and exit.

Alias /nagiosql /var/www/nagiosql/

<Directory /var/www/nagiosql/>
Options None
Order allow,deny
allow from all
</Directory>

Load this config above and restart apache

- a2ensite nagiosql
- service apache2 reload
Edit apache envvars to change apache from its default www-user to nagios, this will be useful for nagiosql to write stuff into nagios directories and control the nagios process.

- nano /etc/apache2/envvars

Change www: data to nagios for both user and group

- chown -R nagios:nagios /var/lock/apache2/
- chown -R nagios:nagios /var/www/
- chown -R nagios:nagios /var/lib/php5
- service apache2 restart
Get mysql and related files to for NagiosQL, at this point you will get prompted for the root password of mysql , please enter a valid password and remember it for later use during NagiosQL installation.

- apt-get install mysql-server php5-mysql libmysqlclient15-dev
Now, go to http://<yourIP>/nagiosql, it is important to note on these settings below
- Click on START INSTALLATION (there should not be errors which will stop you from clicking next, if you see, “REFRESH” that means you’ve missed something above or its in error)
- Click next
- Modify accordingly and as show in the image below
- DBserver type=mysql
- dbserver=localhost
- hostname=127.0.0.1
- DBName=<leave default>
- dbuser=<leave default>
- dbpass=<leave default>
- Admin user=root
- Admin db pass = <password created during mysql installation>
- Nagiosql user=admin
- NagiosQL pass=<any password> & repeat
- Import nagios sample = CHECKED
- Create config paths = CHECKED
- NagiosQL Config path = /usr/local/nagios/etc/nagiosql/
- Nagios config path= /usr/local/nagios/etc/objects/

Click on next and finish, you should not get any errors and it should land you to the login page. Important, please follow next guide before proceeding.
Ensure we set the right permissions for NagiosQL
- chgrp nagios /usr/local/nagios/etc/
- chgrp nagios /usr/local/nagios/etc/nagios.cfg
- chgrp nagios /usr/local/nagios/etc/cgi.cfg
- chmod 775 /usr/local/nagios/etc/
- chmod 664 /usr/local/nagios/etc/nagios.cfg
- chmod 664 /usr/local/nagios/etc/cgi.cfg
- chown nagios:nagios /usr/local/nagios/bin/nagios
- chmod 660 /usr/local/nagios/var/rw/nagios.cmd
- chown nagios:nagios /usr/local/nagios/var/rw/nagios.cmd
Now, login to the NagiosQL page and edit the following

Click on Administration, click on Config Target, click on modify for localhost (the wrench icon).
Set the following;
Method = Change to SSH, then enter your nagios user and password created at the beginning of this document.
Nagios Command File= /usr/local/nagios/var/rw/nagios.cmd
Nagios Binary= /usr/local/nagios/bin/nagios
Nagios Process file= /usr/local/nagios/var/nagios.lock
Nagios config file/usr/local/nagios/etc/nagios.cfg
Now,let’s modify Nagios core main config file to enable NagiosQL folders and disable the default one.

Click on Tools, Nagios Config

Disable / comment out the default ones by adding a # infront of them, like below
#cfg_file=/usr/local/nagios/etc/objects/commands.cfg
#cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
#cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg
#cfg_file=/usr/local/nagios/etc/objects/templates.cfg

#cfg_file=/usr/local/nagios/etc/objects/localhost.cfg

Add the following under config directories
cfg_dir=/usr/local/nagios/etc/nagiosql/

Once done, click on Save.

Now, click on Tools, Nagios Control.
In the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.

For some reason, the import doesn’t really import the process-host-perfdata-file and process-service-perfdata-file commands definitions. Let’s add them otherwise, after the import, you may not get perf graphs working.

Click on Commands, click on definitions. Click on Add, add like below
Command: process-host-perfdata-file
Command line: /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
Rest leave as default. Click on Save.

Click on Add, again.
Command: process-service-perfdata-file
Command line: /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
Rest leave as default. Click on Save.

Now, click on Write Config File.

Click on tools, click on Nagios control, in the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.
Also, the action URLs go missing in the host and service we defined earlier. You can add it yourself in NagiosQL. Follow this to do bring it back into Nagios. Like shown above as an example, we will be adding the host=localhost and service=current users back into Nagios perf data.

To add for a sample host.
Click on Supervision, click on Host. Click to edit localhost (wrench icon). Click on Addon Settings, add the following line in Action URL:
/pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=_HOST_' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=_HOST_

Click on Save.

Now, to add for a sample service.
Click on Supervision, click on Service, search for “Current Users”. Click to edit (wrench icon). Click on Addon Settings, add the following line in Action URL:
/pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=$SERVICEDESC$' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=$SERVICEDESC$

Click on Save.

Now, click on Write Config File.
Click on tools, click on Nagios control, in the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.
You might also want to delete the sample hosts and definitions created by NagiosQL, use NagiosQL to delete them in the order of deleting services, followed by the sample hosts. You can do bulk deletion for hosts or services like shown in the image below. Warning, do not delete localhost and localhost related services.

Now, click on Write Config File.
Click on tools, click on Nagios control, in the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.

This should leave you with the samples created by Nagios core installation monitoring Nagios itself.
Let’s create some nice HTTP links to access a few tools we just created
- nano /usr/local/nagios/share/side.php

Locate the “General” section, add the following lines after the final </div> of that section. Paste these texts below, literally below that.

<div class="navsection">
        <div class="navsectiontitle">Add-ons</div>
        <div class="navsectionlinks">
                <ul class="navsectionlinks">
                        <li><a href="/nagios/stats" target="<?php echo $link_target;?>">Nagiostats</a></li>
                        <li><a href="/pnp4nagios" target="<?php echo $link_target;?>">Nagiosgraph</a></li>
                        <li><a href="/nagiosql" target="<?php echo $link_target;?>">Nagios Configurator</a></li>
                        <li><a href=http://www.astiostech.com target="<?php echo $link_target;?>">Nagios Support</a></li>
                        <li><a href=highsecurity.blogspot.com target="<?php echo $link_target;?>">Nagios Core 4.1.1 Install Guide</a></li>
                </ul>
        </div>
</div>

Example like below;
Finally, refresh your /nagios page in your browser [F5] and see those links like below
Other stuff (And addons for future blog posts)
- Be sure to configure periodic timesync
- Install adminer to manage DB if required
- Install MK livestatus to replace NDO
- Install mod_gearman to replace Nagios default broker modules
- Install Nagvis for visualization (and Install ndo2db for it or MK livestatus)