Shawn on the net

Recently I started to run into issues with my load balancing solution.  I was running HAProxy on a GoGrid instance and it was working pretty well.  Eventually though as our traffic went up problems started to appear.  After completing as many networking optimizations as I could it was clear that I needed to find another solution.  I was debating between HAProxy on a dedicated server at ServePath, a self managed hardware load balancer or a managed load balancer from ServePath when another contender entered the ring.

The nice people at Zeus set me up with a trial of their ZXTM load balancing solution to try so I figured it was worth a shot.  Setup was pretty easy well ok very easy on a fresh GoGrid instance, I made some networking configuration optimizations then went about setting up my pools.  To ensure it wouldn’t just completely die with our traffic I set it up as a server under our existing HAProxy setup then just gradually ramped up the percentage of requests going to it.  Once it was almost at 100% I made preparations and swapped it out with our HAProxy system.

There was immediately a noticeable improvement in latency which if you’ve read my blog before will know is very important to me.  The interface was a pleasure to work with as well enabling me to easily monitor traffic levels and issues on our operations screen.  After setup it was pretty much left to its own while I continued investigating the other solutions.

Performance wise it went very well.  During the testing period we peaked at I believe around 4000 requests per second and commonly ran at over 2000 per second for hours at a time.  While we had some slowdowns during this time it wasn’t anything dramatic and probably had more to do with running it on a virtual server than a problem with Zeus itself.

ZXTM also offers a pretty cool ability to move some application logic forward into the load balancer.  Though it wasn’t suitable for my needs I could certainly think of a lot of uses in other situations.

In the end my month with ZXTM was certainly a good experience and I can strongly recommend their software.  As for my setup, while I ended up going back to HAProxy only this time running it on dedicated hardware and it is still going strong.

I’ve spoke to Amazon a few times about a US-West region as so far the reply has always been that they are considering it but nothing is in motion yet.  Well the other day while making a request for a higher limit on my elastic ips I went to select the region I wanted them for and to my surprise saw US-West in the list.  Needless to say I was quite excited!

Unfortunately I couldn’t find the option to create new west coast instances and today the option is gone from the elastic ip page.  Hopefully though this is a sign of things to come very soon as I would love to be able to host all our servers with Amazon.

Zabbix is a network monitoring system which uses a central server and remote agents which report back to it. Today I setup my central server and after much document reading, and Google searching got it all configured and working with one agent. Going forward now that I’m familiar with the system the rest of the agents should be a lot easier to setup. Next task is to configure the screens.

During the server setup I ran into a few problems that you might have as well so read on and hopefully you’ll save yourself a lot of head scratching.

1. Zabbix doesn’t support SMTP-Auth. This is certainly a pain but thanks to this Forum Post it’s not that hard to work around. I started developing a patch to add in SMTP-Auth support and I might finish it off this week but I needed the alerts up and running right away so thats on hold.

2. Hosts, Items, Triggers, Actions oh my!
- Hosts are the systems you want to monitor, these need to have zabbix_agent/zabbix_agentd installed and configured for the server. When adding hosts select a template to get a bunch of default items/triggers out of the box, these will save you a lot of time.
- Items are things to watch IE. the amount of free space on a drive.
- Triggers are conditions of items that cause an alert. For example the amount of free space on partition /usr drops below 100MB
When you select a host template a lot of these will be setup so you’ll probably want to start off by just cloning/editing existing ones.

Thats really about it to get started I have personally just scratched the surface so I can’t go much further but this should be enough to get you up and running fast and give you some extra time to study the docs.

Grab Zabbix Today.