Monday, June 6, 2016

The 60 minute Nagios Core 4 install guide on Debian 7 Wheezy (Nagios 4, PNP4Nagios -with MRTG & NagiosQL) built from source or just use the VM OVA2.0

Version 1.1 – 07 June 2016


NOTES:

  • Copyright and registered trademarks are the properties of their respective companies/individuals
  • By all means, these are not my own guides but a collection of guides online with my own hacks here and there that didn’t work for me and now they do. THANK YOU ALL THOSE RESPECTABLE BLOGGERS FOR THEIR CONTRIBUTION
  • I do not provide any warranty whatsoever for using this guide or the OVA2 images herein
  • This guide doesn’t include how to make Nagios work for you, that’s something you need to learn  this guide is just for how to get Nagios working, period.
  • Copy paste as single lines unless otherwise told, the – denotes to run on cli, copy paste them line by line after the dash space (- ), e.g. – apt-get update, copy “apt-get update” without the quotes and paste into your putty/ssh session.

Software & versions:

  • Nagios Core 4.1.1
  • Nagios plugins including community add-ons and check_nrpe
  • MRTG graphing engine to monitor Nagios itself.
  • PNP4Nagios 0.6 with RRD
  • NagiosQL 3.20 for Web GUI to edit and manage Nagios configs

 

VM Edition download:

If you’re not interested to do all of this below yet want to get Nagios 4.1.1 up and running, then download the OVA2 format VM image from Sourceforge: https://sourceforge.net/projects/debiannagios. Be sure to read the notes there.

Build by hand guide:

  1. Download and install the latest iteration of Debian 7 64bit netinst here: http://cdimage.debian.org/cdimage/archive/7.10.0/amd64/iso-cd/debian-7.10.0-amd64-netinst.iso
  2. Update and install a couple of packages, at one point the mrtg installation will a question, just press enter to continue.

    - apt-get update
    - apt-get upgrade

    - apt-get install --force-yes apache2 build-essential libgd2-xpm-dev libssl-dev exim4 heirloom-mailx wget apache2-utils curl daemon apt-file libnet-snmp-perl libperl5.14 libpq5 libfreeradius-dev libfreeradius2 libsensors4 libsnmp-base libtalloc2 libtdb1 libwbclient0 samba-common samba-common-bin smbclient snmp mrtg libmysqlclient-dev libcgi-pm-perl librrds-perl libgd-gd2-perl python build-essential ssh sudo expect linux-headers-`uname -r` curl sox apache2 libssl-dev libncurses5-dev bison subversion libnewt-dev libcurl4-openssl-dev libnet-ssleay-perl openssl libauthen-pam-perl libio-pty-perl vim iftop tcpdump iptraf ngrep strace ltrace lsof htop sysstat nmap dstat powertop ntpdate ntp ssh libdbi-perl libhtml-template-perl libnet-daemon-perl libterm-readkey-perl mysql-client mysql-common psmisc ethtool apt-show-versions libapt-pkg-perl libmyodbc whois libusb-dev  libdigest-md5-file-perl uuid-dev uuid screen autoconf automake git module-init-tools iotop iftop nmap unzip checkinstall libcgi-pm-perl librrds-perl libapache2-mod-python libapache2-mod-php5 php5-sqlite php5-mcrypt libgd2-xpm-dev libdbi1 libdbi-dev libapache2-mod-proxy-html snmp php5 libsnmp15 libnagios-plugin-perl php5-gd graphviz graphviz-dev php5-mysql sqlite3 libssh2-php fping smokeping screen

  3. Create users and groups (this use will also be used for apache, explain why later), place a password for user nagios, enter it twice

    - useradd nagios && passwd nagios
    - groupadd nagios
    - usermod -a -G nagios nagios
    - usermod -a -G nagios www-data

  4. Create some directories

    - mkdir /usr/local/nagios
    - mkdir -p /usr/local/nagios/share/{stylesheets,images}
    - chown -R nagios:nagios /usr/local/nagios

  5. Get Nagios core 4.11 and Nagios plugins

    - cd /usr/src
    - wget
    http://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.1.1/nagios-4.1.1.tar.gz
    - wget http://www.nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz
    - tar -xzvf nagios-4.1.1.tar.gz
    - tar -xzvf nagios-plugins-2.1.1.tar.gz

  6. Configure, make Nagios core

    - cd nagios-4.1.1/
    - ./configure --prefix=/usr/local/nagios --with-nagios-user=nagios --with-nagios-group=nagios --with-command-user=nagios --with-command-group=nagios
    - make all
    - make install
    - make install-init
    - make install-commandmode
    - make install-config
    - make install-exfoliation
    - make install-webconf

  7. Copy over some scripts (useful for SNMPTT, explained later)

    - cp -R contrib/eventhandlers/ /usr/local/nagios/libexec/
    - chown -R nagios:nagios /usr/local/nagios/libexec/eventhandlers
  8. Create your nagiosadmin user!, this will be the user you will use to login to the core web console via http://.
    - htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
  9. Nagios init file doesn’t fulfil some LSB requirements, so edit the init file and add two lines like this

    - nano /etc/init.d/nagios

    # Default-Start:      2 3 4 5
    # Default-Stop:       0 1 6


    image
  10. Alright, let’s restart apache for nagios web to work, set nagios to start automatically as well. Also, start the nagios core service. At this point, Nagios core is already installed, you can head over to http://<yourip>/nagios . Note, you may get check errors from the local services (localhost) that’s being monitored, that’s because we’ve not installed nagios plugins, yet. As long as you can see the webUI, that’s fine for now.

    - a2ensite nagios
    - service apache2 restart

    - ln -s /etc/init.d/nagios /etc/rc2.d/S20nagios
    - update-rc.d nagios enable

    - service nagios start
  11. Alright, let’s get the plugins installed now, after this is done, the web should show all plugins showing OK by right. You can recheck that if you want.

    - cd ..
    - cd nagios-plugins-2.1.1
    - ./configure --with-nagios-user=nagios --with-nagios-group=nagios --with-openssl=/usr/bin/openssl --enable-perl-modules --enable-libtap

    - make && make install

  12. Now, we will download and install NRPE just to get that binary to be used in almost all nagios agent based checks in your near future. After compiling, we will copy the binary to the libexec folder of Nagios.

    - cd ..
    - wget
    http://kent.dl.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.15/nrpe-2.15.tar.gz
    tar -zxvf nrpe-2.15.tar.gz
    - cd nrpe-2.15

    - ./configure --with-ssl=/usr/bin/openssl --with-ssl-lib=/usr/lib/x86_64-linux-gnu
    - make
    - cp src/check_nrpe /usr/local/nagios/libexec/
  13. You could also add some apt based contributed plugins like this below. This step is completely optional, though recommended. Note, this may install many other dependencies.

    - apt-get install nagios-plugins-contrib
    - cp -r /usr/lib/nagios/plugins/* /usr/local/nagios/libexec/

  14. Now, let’s get MRTG configured

    - cp /usr/src/nagios-4.1.1/sample-config/mrtg.cfg /usr/local/nagios/etc/
    - mkdir /usr/local/nagios/share/stats


    Edit the mrtg.cfg file and add a line right on the top of the file
    - nano /usr/local/nagios/etc/mrtg.cfg

    Add this to top of that file, save and exit.

    WorkDir: /usr/local/nagios/share/stats

    Run these at the command line.
    - env LANG=C mrtg /usr/local/nagios/etc/mrtg.cfg
    - indexmaker /usr/local/nagios/etc/mrtg.cfg --output=/usr/local/nagios/share/stats/index.html

    Now, we need to add the mrtg graphing to cron to run
    Create and edit a new crontab file, like below;

    - nano /etc/cron.d/nagiostats

    Paste this into that file, save and exit.
    */5 * * * *  root  env LANG=C /usr/bin/mrtg /usr/local/nagios/etc/mrtg.cfg
  15. Now, let’s go get pnp4nagios installed!, get rrdtool, download pnp4nagios and do the backend setup.

    - apt-get install rrdtool

    - cd /usr/src
    - wget -O pnp4nagios-0.6.25.tar.gz 
    http://downloads.sourceforge.net/project/pnp4nagios/PNP-0.6/pnp4nagios-0.6.25.tar.gz?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fpnp4nagios%2F&ts=1464497035&use_mirror=pilotfiber

    - tar -zxvf pnp4nagios-0.6.25.tar.gz 
    - cd pnp4nagios-0.6.25
    - ./configure --with-rrdtool=/usr/bin/rrdtool

    - make all
    - make fullinstall
    - a2enmod rewrite
    - service apache2 restart

  16. Now, let’s enable a cool pnp4nagios popup graph like you see in NagiosXI

    - cp contrib/ssi/status-header.ssi /usr/local/nagios/share/ssi/
    - chown nagios:nagios /usr/local/nagios/share/ssi/status-header.ssi
    - chmod 644 /usr/local/nagios/share/ssi/status-header.ssi

  17. At this point, you need to access the pnp4nagios website http://<yourIP>/pnp4nagios and proceed with the instructions therein. Basically, everything should appear in green i.e. ALL GOOD, then, you can delete this install file

    - rm /usr/local/pnp4nagios/share/install.php
  18. Now, we will attempt to add a few definitions inside Nagios core configs to show our graphs, please follow these steps carefully and repeat for other services that produce performance data.

    - nano /usr/local/nagios/etc/nagios.cfg

    Paste the following at the end of the file. Everything is a single line

    process_performance_data=1
    service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata
    service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$
    service_perfdata_file_mode=a
    service_perfdata_file_processing_interval=15
    service_perfdata_file_processing_command=process-service-perfdata-file
    host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata
    host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$
    host_perfdata_file_mode=a
    host_perfdata_file_processing_interval=15
    host_perfdata_file_processing_command=process-host-perfdata-file

    image
  19. Next, enable the perf data command file

    - nano /usr/local/nagios/etc/objects/commands.cfg

    Paste the following at the end of that file as well

    define command {
            command_name process-service-perfdata-file
            command_line /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
            }
    define command {
            command_name process-host-perfdata-file
            command_line /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
            }

    image
  20. Here’s an important note, we need to enable these graphs inside either hosts or services that produces performance data. Usually, hosts will always produce such data if you use the standard host check commands, however, services depends on how it is configured and what the plugin returns. In our example, we will use localhost (the nagios server itself) as hosts and current user count as a service in which we want performance graphs to be generated.

    - nano /usr/local/nagios/etc/objects/localhost.cfg

    Under host localhost, locate hostname localhost, below address 127.0.0.1, add a line like this and as shown in the example below that as well.. This can be done for all hosts!

    action_url              /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=_HOST_' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=_HOST_

    image

    Now, locate the “current users” service definition add a line like this as shown in the example below that as well. This can be used for all services that has performance data.

    action_url              /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=$SERVICEDESC$' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=$SERVICEDESC$


    image

  21. Now, let’s verify if we have done the above correctly or not by running the pre-flight nagios check like this

    - /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

    It should show zero errors and zero warnings. If it doesn’t, you’ve made a typo or an error with the steps above.
    image
  22. Now, if all goes well as above, restart nagios process and start the npcd process, which is the nagios performance data service, we will also set the npcd to start at boot, after the restart, go back to the webpage and you should see some extra icons appearing infront of “localhost” and “current users”. Do not panic if you see errors or no performance data, give it like 10 to 30 minutes for it to generate some data.

    - service npcd start
    - update-rc.d npcd enable

    - service nagios restart
  23. Go ahead and add the same service and host perf data action url to other hosts or services you’ve created and perf graphs will appear in the main nagios page. If you don’t want to show it on that page, they will be automatically generated and you can access them via http://<yourIP>/pnp4nagios
  24. Now, let’s install NagiosQL, one of the many tools out there to perform edits on Nagios configs without having to touch text editors and what not.

    - cd /usr/src
    - wget -O  nagiosql_320.tar.gz
    http://downloads.sourceforge.net/project/nagiosql/nagiosql/NagiosQL%203.2.0/nagiosql_320.tar.gz?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fnagiosql%2Ffiles%2Fnagiosql%2FNagiosQL%25203.2.0%2F&ts=1464465362&use_mirror=ufpr

    - tar -zxvf nagiosql_320.tar.gz

    - cp -R nagiosql32 /var/www/nagiosql
    - chown -R nagios:nagios /var/www/nagiosql

  25. Edit the timezone in php.ini. This is an important step to ensure NagiosQL knows when files were edited etc. Locate the date.timezone definition and define it by uncommenting it. Refer to this for a list of countries and how to write that definition. In my case, i am of course going to set it to Kuala Lumpur, Malaysia, where i live.

    - nano /etc/php5/apache2/php.ini

    date.timezone = Asia/Kuala_Lumpur

    image

    - service apache2 restart
  26. Create some directories in which NagiosQL will write all config files.

    - mkdir /usr/local/nagios/etc/nagiosql
    - chown -R nagios:nagios /usr/local/nagios/etc/nagiosql


    Now, let’s create a NagiosQL apache2 definition

    - nano /etc/apache2/sites-available/nagiosql

    Paste, this below, save and exit.

    Alias /nagiosql /var/www/nagiosql/

    <Directory /var/www/nagiosql/>
    Options None
    Order allow,deny
    allow from all
    </Directory>

    Load this config above and restart apache

    - a2ensite nagiosql
    - service apache2 reload

  27. Edit apache envvars to change apache from its default www-user to nagios, this will be useful for nagiosql to write stuff into nagios directories and control the nagios process.

    - nano /etc/apache2/envvars

    Change www: data to nagios for both user and group

    image

    - chown -R nagios:nagios /var/lock/apache2/
    - chown -R nagios:nagios /var/www/
    - chown -R nagios:nagios /var/lib/php5
    - service apache2 restart
  28. Get mysql and related files to for NagiosQL, at this point you will get prompted for the root password of mysql , please enter a valid password and remember it for later use during NagiosQL installation.

    - apt-get install mysql-server php5-mysql libmysqlclient15-dev
  29. Now, go to http://<yourIP>/nagiosql, it is important to note on these settings below
    - Click on START INSTALLATION (there should not be errors which will stop you from clicking next, if you see, “REFRESH” that means you’ve missed something above or its in error)
    - Click next
    - Modify accordingly and as show in the image below
    - DBserver type=mysql
    - dbserver=localhost
    - hostname=127.0.0.1
    - DBName=<leave default>
    - dbuser=<leave default>
    - dbpass=<leave default>
    - Admin user=root
    - Admin db pass = <password created during mysql installation>
    - Nagiosql user=admin
    - NagiosQL pass=<any password> & repeat
    - Import nagios sample = CHECKED
    - Create config paths = CHECKED
    - NagiosQL Config path = /usr/local/nagios/etc/nagiosql/
    - Nagios config path= /usr/local/nagios/etc/objects/


    Click on next and finish, you should not get any errors and it should land you to the login page. Important, please follow next guide before proceeding.
  30. Ensure we set the right permissions for NagiosQL

    - chgrp nagios /usr/local/nagios/etc/
    - chgrp nagios /usr/local/nagios/etc/nagios.cfg
    - chgrp nagios /usr/local/nagios/etc/cgi.cfg
    - chmod 775 /usr/local/nagios/etc/
    - chmod 664 /usr/local/nagios/etc/nagios.cfg
    - chmod 664 /usr/local/nagios/etc/cgi.cfg
    - chown nagios:nagios /usr/local/nagios/bin/nagios
    - chmod 660 /usr/local/nagios/var/rw/nagios.cmd
    - chown nagios:nagios /usr/local/nagios/var/rw/nagios.cmd

  31. Now, login to the NagiosQL page and edit the following

    Click on Administration, click on Config Target, click on modify for localhost (the wrench icon).
    Set the following;
    Method = Change to SSH, then enter your nagios user and password created at the beginning of this document.
    Nagios Command File=  /usr/local/nagios/var/rw/nagios.cmd
    Nagios Binary= /usr/local/nagios/bin/nagios
    Nagios Process file= /usr/local/nagios/var/nagios.lock
    Nagios config file/usr/local/nagios/etc/nagios.cfg

  32. Now,let’s modify Nagios core main config file to enable NagiosQL folders and disable the default one.

    Click on Tools, Nagios Config

    Disable / comment out the default ones by adding a # infront of them, like below
    #cfg_file=/usr/local/nagios/etc/objects/commands.cfg
    #cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
    #cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg
    #cfg_file=/usr/local/nagios/etc/objects/templates.cfg

    #cfg_file=/usr/local/nagios/etc/objects/localhost.cfg

    image

    Add the following under config directories
    cfg_dir=/usr/local/nagios/etc/nagiosql/

    image

    Once done, click on Save.

    Now, click on Tools, Nagios Control.
    In the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.

    For some reason, the import doesn’t really import the process-host-perfdata-file and process-service-perfdata-file commands definitions. Let’s add them otherwise, after the import, you may not get perf graphs working.

    Click on Commands, click on definitions. Click on Add, add like below
    Command: process-host-perfdata-file
    Command line: /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
    Rest leave as default. Click on Save.

    Click on Add, again.
    Command: process-service-perfdata-file
    Command line: /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
    Rest leave as default. Click on Save.

    Now, click on Write Config File.

    Click on tools, click on Nagios control, in the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.
  33. Also, the action URLs go missing in the host and service we defined earlier. You can add it yourself in NagiosQL. Follow this to do bring it back into Nagios. Like shown above as an example, we will be adding the host=localhost and service=current users back into Nagios perf data.

    To add for a sample host.
    Click on Supervision, click on Host. Click to edit localhost (wrench icon). Click on Addon Settings, add the following line in Action URL:
    /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=_HOST_' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=_HOST_

    Click on Save.

    Now, to add for a sample service.
    Click on Supervision, click on Service, search for “Current Users”. Click to edit (wrench icon). Click on Addon Settings, add the following line in Action URL:
    /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=$SERVICEDESC$' class='tips' rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=$SERVICEDESC$

    Click on Save.

    Now, click on Write Config File.
    Click on tools, click on Nagios control, in the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.
  34. You might also want to delete the sample hosts and definitions created by NagiosQL, use NagiosQL to delete them in the order of deleting services, followed by the sample hosts. You can do bulk deletion for hosts or services like shown in the image below. Warning, do not delete localhost and localhost related services.

    image

    Now, click on Write Config File.
    Click on tools, click on Nagios control, in the following order, click Write Monitoring Data, Click Write Additional Data, Click on Check Configuration Files (you should NOT have errors btw), and finally, click on Restart Nagios.

    This should leave you with the samples created by Nagios core installation monitoring Nagios itself.
  35. Let’s create some nice HTTP links to access a few tools we just created

    - nano /usr/local/nagios/share/side.php

    Locate the “General” section, add the following lines after the final </div> of that section. Paste these texts below, literally below that.

    <div class="navsection">
            <div class="navsectiontitle">Add-ons</div>
            <div class="navsectionlinks">
                    <ul class="navsectionlinks">
                            <li><a href="/nagios/stats" target="<?php echo $link_target;?>">Nagiostats</a></li>
                            <li><a href="/pnp4nagios" target="<?php echo $link_target;?>">Nagiosgraph</a></li>
                            <li><a href="/nagiosql" target="<?php echo $link_target;?>">Nagios Configurator</a></li>
                            <li><a href=http://www.astiostech.com target="<?php echo $link_target;?>">Nagios Support</a></li>
                            <li><a href=highsecurity.blogspot.com target="<?php echo $link_target;?>">Nagios Core 4.1.1 Install Guide</a></li>
                    </ul>
            </div>
    </div>



    Example like below;
    image

  36. Finally, refresh your /nagios page in your browser [F5] and see those links like below

    image
  37. Other stuff (And addons for future blog posts)
    - Be sure to configure periodic timesync
    - Install adminer to manage DB if required
    - Install MK livestatus to replace NDO
    - Install mod_gearman to replace Nagios default broker modules
    - Install Nagvis for visualization (and Install ndo2db for it or MK livestatus)

We hope this guide has helped you get your Nagios awesome monitoring tool up and running. Thank you and as usual, we’d appreciate feedbacks.