Extending Nagios 3 with Nagiosgraph and Birdseye

01 Mar 2012

In one of my previous posts we explained how to set up Nagios under Debian. Today we continue with some Nagios plugins to make your life easier. In the article below we will show you how to add more functionality to Nagios with Nagiosgraph and Birdseye 2.

Problem situation

Apart from the information when your host is up, it might also be interesting to know how your hosts are performing over a period of time. It is also possible we want something we can put on a screen that only gives warnings when things go wrong. When you have 10 hosts with 10 services you can watch 100 lines. If you have 100 hosts with 10 services it gets quite hard. For this problem I want to use birdseye 2.

Nagiosgraph

The first plugin we will install is Nagiosgraph, we can just get it from sourceforge here. Download the .deb file to your server. First we will need to install some dependensies and then we can load the .deb, keep in mind that everything needs to be done as root:

apt-get install librrds-perl
dpkg -i nagiosgraph.deb

nagiosgraph.deb refers to the .deb we downloaded from scourceforge. We need to edit two lines in the /etc/nagios3/nagios.cfg :

     process_performance_data=1
     service_perfdata_file=/var/nagios/perfdata.log
     service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
     service_perfdata_file_mode=a
     service_perfdata_file_processing_interval=30
     service_perfdata_file_processing_command=process-service-perfdata

restart Nagios 3 :

/etc/init.d/nagios3 restart

Now we need to define an extra command to our service templates so we automatically get the graphs per host.

define service{
        ...
            action_url /nagiosgraph/cgi-bin/show.cgi?host=$HOSTNAME$&service=$SERVICEDESC$
        register                        0  
}

Now restart apache2 and nagios3 again :

/etc/init.d/nagios3 restart
/etc/init.d/apache2 restart

Now there is an extra icon you can see here : Click it and you will get nice graphs.

Birdseye 2

Birdseye is a very nice GUI for Nagios, it only shows hosts that have a problem. I looks very cool and is nice to put on a big screen in your office. It is a webpage that gives an overview of all services that have warnings or are in a critical state. The first preliminary is getting NDOUtils. The NDOUtils (Nagios Data Output Utils) addon allows you to move status and event information from Nagios to a database for later retrieval and processing. If you haven't installed it please proceed by doing so:

apt-get install ndoutils ndoutils-nagios3-mysql

Second thing we need is Birdseye 2 package. Unzip into a temporary folder on your server (tmp). We need to configure NDOUtils. Enable it by editing /etc/default/ndoutils.

ENABLE_NDOUTILS=1

Add a broker module to /etc/nagios3/nagios.cfg by adding this line :

broker_module=/usr/lib/ndoutils/ndomod-mysql-3x.o config_file=/etc/nagios3/ndomod.cfg

Log into your mysql :

mysql -u username -p

Now we will add a new database called 'events'.

create database events;

Now we will use this database, meaning we can edit it :

use events

Create a table called 'd', grant rights on it to ndoutils user :

create table d (object_id int(11) not null, host_object_id int(11), status_change_time datetime, primary key(object_id));
grant select on events.d to 'ndoutils'@'localhost';

Next we will use the ndoutils database, we will define a new delimiter so we can sate new triggers. Just copy everything below into mysql :

use ndoutils
delimiter |
CREATE TRIGGER hup AFTER UPDATE ON nagios_hoststatus FOR EACH ROW BEGIN
IF NEW.current_state > 0 THEN
REPLACE INTO events.d SELECT NEW.host_object_id, NEW.host_object_id, now();
ELSEIF OLD.current_state > 0 THEN
DELETE FROM events.d WHERE NEW.host_object_id = object_id;
END IF;
END;
|
CREATE TRIGGER sup AFTER UPDATE ON nagios_servicestatus FOR EACH ROW BEGIN
IF NEW.current_state > 0 THEN
REPLACE INTO events.d SELECT NEW.service_object_id, (SELECT host_object_id FROM nagios_services WHERE service_object_id = NEW.service_object_id), now();
ELSEIF OLD.current_state > 0 THEN
DELETE FROM events.d WHERE NEW.service_object_id = object_id;
END IF;
END;
|
CREATE TRIGGER hin AFTER INSERT ON nagios_hoststatus FOR EACH ROW BEGIN
IF NEW.current_state > 0 THEN
REPLACE INTO events.d SELECT NEW.host_object_id, NEW.host_object_id, now();
ELSE
DELETE FROM events.d WHERE NEW.host_object_id = object_id;
END IF;
END;
|
CREATE TRIGGER sin AFTER INSERT ON nagios_services FOR EACH ROW BEGIN
IF (SELECT current_state FROM nagios_servicestatus nss WHERE nss.service_object_id = NEW.service_object_id) > 0 THEN
REPLACE INTO events.d SELECT NEW.service_object_id, NEW.host_object_id, now();
ELSE
DELETE FROM events.d WHERE NEW.service_object_id = object_id;
END IF;
END;
|
CREATE TRIGGER hdel AFTER DELETE ON nagios_hoststatus FOR EACH ROW BEGIN
IF OLD.current_state > 0 THEN
DELETE FROM events.d WHERE object_id = old.host_object_id;
END IF;
END;
|
CREATE TRIGGER sdel AFTER DELETE ON nagios_servicestatus FOR EACH ROW BEGIN
IF OLD.current_state > 0 THEN
DELETE FROM events.d WHERE object_id = OLD.service_object_id;
END IF;
END;
|
delimiter ;

Now to make it safer the creator decided not to use a direct connection to the database, but rather use a generated XML file to get information from. The application is written in flex and php. The XML file gets read every so often to give you updates. To generate the XML file we will access the database. However we will not directly access the database of NDOUTILS, but we will define another table that will get filled by triggers on the other table. We need to put the php in a non-publical accessible folder. I suggest using /etc/nagios3/birdseye.

mkdir /etc/nagios3/birdseye
cp /tmp/birdseye/Install/PHP-Scripts/* /etc/nagios3/birdseye

We need to edit the Settings.php file :

mkdir /etc/nagios3/birdseye
cp /tmp/birdseye/Install/PHP-Scripts/* /etc/nagios3/birdseye

In the settings file you need to fill in the password for the ndoutils user. This can be found by doing following regex :

cat /etc/dbconfig-common/ndoutils-mysql.conf | egrep dbpass=

Now your password is the sequence of characters between the quotes (' ') after the = sign. We also need to change the directory of where the XML file will be generated. Make sure this directory is created!

$DBPassword = 'yourpassword';
$XMLFileBirdseye = '/var/www/birdseye2/XMLBirdseye.xml';
mkdir /var/www/birdseye2

We need to get the pear package for php :

apt-get install php-pear

We will also have to install the pear MDB2 package for mysql and the MDB2 drivers :

pear install MDB2
pear install MDB2#mysql
apt-get install php-mdb2
apt-get install php-mdb2-driver-mysql

To test the php script enter the following in a terminal:

php XMLBirdseye.php

You will see a lot of warnings, ignore them. If you have errors please correct them. Make sure the directory you are trying to generate the XML file into is properly defined in the Settings.php file and that the directory exists. Some of you have noted that you need to run the php files to generate the XML files. Now fear not for I have a cunning plan. We will use crontab to generate output every minute.(Should you want a longer interval, please change the 1 to the amount of minutes you wish the process to be executed). Remember to do this as a user who has the nescesarry rights. I did this with root.

crontab -e

Add this line

*/1 * * * * /usr/bin/php /etc/nagios3/birdseye/XMLBirdseye.php

Copy the Flex folder to your webfolder.

 cp -r /tmp/birdseye/Install/Flex/* /var/www/birdseye2/

restart nagios3

/etc/init.d/nagios3 restart

Next we need to alter some settings in the XML. I found them to deliver errors so I just deleted them in /var/www/birdseye2/settings/settings.xml, below you can find my XML. You need to edit URL and delete a few lines.

<settings>
        <birdseye>
                <url>http://myserversurl/birdseye2/XMLBirdseye.xml</url>
                <maxhosts>12</maxhosts>
                <maxevents>10</maxevents>
                <interval>60000</interval>
                <maxfont>15</maxfont>
                <minfont>13</minfont>
<panel>
                        <width>200</width>
                </panel>
        </birdseye>
</settings>

Why I did this is because the nagios line was giving security errors. I didn't need the tabs to add websites in, so I also deleted those. Now one more problem that will arrise is that you need to be logged in into nagios3 to access some of the features. So let's put our birdseye behind the same htaccess as our nagios3 webui.

        <Directory /var/www/birdseye2>
        AuthName "Nagios Access"
        AuthType Basic
        AuthUserFile /etc/nagios3/htpasswd.users
        # nagios 1.x:
        #AuthUserFile /etc/nagios/htpasswd.users
        require valid-user
        </Directory>

After this restart Apache2 :

/etc/init.d/apache2 restart

Now go to your website at https://myserversurl/birdseye2 login-in using your Nagios3 credentials. If all is well, it will just show a black page with timestamps. If something is wrong, there will be warnings on the screen.