RabbitMQ First Look
06 Aug 2011
I spent some time looking at RabbitMQ this week. It does what it says on the tin, but I encountered a couple of gotchas.
There is a Chef cookbook for
which made installation easy. It uses the
(in the default attributes),
which ends up being used by the /etc/rabbitmq/rabbitmq-env.conf and is used
in the name of data directories.
But I’m installing on EC2, where stopping and starting change change the
hostname, which then causes problems.
This is mentioned on the RabbitMQ on EC2
page, where the recommended action is
to change the hostname and give it a localhost address in /etc/hosts.
I’m not particularly happy about that, because it could cause problems to
other parts of the system:
hostname --fqdn no
longer gives a fully qualified name; and other software may have started
using the old name and get confused.
In my case this runs on a dedicated host and I can from Chef script the
hostname changing to occur before the RabbitMQ install, so it will have to do.
There are a bunch of clients available (see Client & Developer Tools). My first requirement was for Ruby, and there is a collection of clients on ruby-amqp on github, with an overview of their relations here. I initially used amqp, but switched to bunny which is easier in scripts because it’s synchronous.
For future Java integration I tried rabbitmq-java-client which worked from a simple project in IDEA.
The get started guide takes you from a simple queue to full topics, and I recommend you follow all of them. One thing that irked me was that the simple use cases turn out to be just gloss over the complex use cases. That means that what you learn in the first lesson actually misleads you (you can’t send message directly to a queue, it just looks like it), and as you try to move to more complex models it’s not always clear what smoke and mirrors was used to make the simpler ones work (Is it because you’re using the default exchange? Is it because there is special queuename/routing_key combination? etc). I would have preferred being told the full model, and have explicit declaration of those configuration items that affect routing.
When I look at a message queue, I expect it to work like a mail server: when I inject a message I expect the MQ to do its damndest to make sure that a receiver gets it. What I found that with RabbitMQ and bunny, the default configuration doesn’t: unless you deliver to a queue that has been explicitly marked durable, via an exchange that is explicitly marked durable, with a message that is explicitly marked persistent, a simple server restart can lose messages. This is documented, in tutorial 2 under “Message durability”, and in the scenarios FAQ under “Reliable persistent message delivery” and “Store-and-forward”. I just find it strange that that’s not the default.
While experimenting, I used rabbitmqctl to inspect the server state, which is useful, but the Management Plugin. looks even more promising. I had not found a tool to drink from the firehose, but it didn’t take long to make a quick-and-dirty rabbitmq-trace-logger.
Now I’m looking forward to introducing this into my system architecture and put it to real work.