Cross Dominant

"Monitoring EC2 Servers with Zabbix"

Amazon Cloudfront is great for monitoring EC2 servers but it can only do external monitoring so it cannot see things like memory being exhausted etc. I tried using Cacti but it was a nightmare to configure with plugins. Nagios always looks like it'd take weeks to sort out.

A bit of Googling found Zabbix and people seem to like it. Install was a doddle but configuring can be a pain unless you just copy the demo settings.

You can obviously install all of this on the server to be monitored but it is only useful in non-catastrophic cases. If the server runs out of memory or goes to 100% CPU you won't be able to login to see what happened in the run-up. So ideally use a separate server. We use one of our test servers for monitoring since they spend most of their time unstressed.

The Monitoring Server Install

sudo aptitude install zabbix-server-mysql zabbix-frontend-php
sudo vi /etc/php5/apache2/php.ini
Increase some of the PHP variables in that file (you will be warned when you login to the Admin Panel)

sudo /etc/init.d/apache2 restart
sudo apt-get install zabbix-agent

go to http://name.of.server/zabbix
Login with user=admin, password=zabbix

Each Monitored Server Install

sudo apt-get install zabbix-agent
cd /etc/zabbix
sudo vim zabbix_agent.conf
Add the Monitoring Server's Public and Private IP addresses
Open up ports 10050 and 10051 in your EC2 security group (using Elasticfox or similar) and allow the monitoring server's private IP

sudo /etc/init.d/zabbix-agent restart

SNMP (Optional)

You can use SNMP monitoring too but the Zabbix agent seems ok so far.

Extra Config Afterwards

The easiest way to start monitoring is to do a full clone of the demo localhost monitoring and just change the IP address and DNS name to the monitored server.

Authenticated SMTP Alerts

We use AuthSMTP for all email sending from EC2. Zabbix doesn't handle authenticated SMTP out of the box so you have to use an external script.

sudo apt-get install sendEmail

Create a script "zabbix_sendemail" (chmod 755) in the AlertScriptsPath folder specified by /etc/zabbix/zabbix_server.conf (/etc/zabbix/alert.d/)
--------------------------------------
#!/bin/sh

export smtpemailfrom=zabbix@yourdomain.com
export zabbixemailto=$1
export zabbixsubject=$2
export zabbixbody=$3
export smtpserver=yoursmtpserver.com
export smtplogin=smtpuser
export smtppass=smtppassword

/usr/bin/sendEmail -f $smtpemailfrom -t $zabbixemailto -u $zabbixsubject -m $zabbixbody -s $smtpserver:25 -xu $smtplogin -xp $smtppass
--------------------------------------

Set a Media Type (Administration / Media types) script to zabbix_sendemail
Add that Media to the Admin user (Administration / Users / user)
Set the Action (Configurations / Actions) for the alert(s) you want to trigger on and who should get them (Admin) 

SMS Alerts

Sign up for something like Clickatell and buy a block of 400 credits for €17. Then repeat the steps for SMTP above but with the following script. I have both SMS and email set to be sent on certain triggers like low memory.

#! /bin/sh

/usr/bin/wget --spider --no-check-certificate "http://api.clickatell.com/http/s
endmsg?api_id=NNNNNNN&user=XXXXXXXXX&password=YYYYYYY&to=$1&text=$2"

Monitor What?

Zabbix allows you to monitor a wide range of parameters and processes. So everything from network activity to whether your FTP server is contactable. It's not the easiest web-app to configure but so far I'm finding it extremely useful.

Discuss

comments powered by Disqus