🥋 🎙️ 😏 Simple integration of RabbitMQ and Spring Boot 🌽 🌂 🔛

A translation of the article was prepared ahead of the start of the course "Developer on the Spring Framework" .

Hello everyone!

I would like to share with you an open source library that facilitates the integration of RabbitMQ with applications on Spring Boot. In addition, this library offers a new, improved concept of retries (compared to the standard Spring AMQP approach).

Simplifies? But how?

Autoconfiguration

Suppose we want to implement interaction (for example, using SSL) with RabbitMQ using Spring AMQP. We need to create a few beans, such as ConnectionFactory, RabbitAdmin, RabbitTemplateand AbstractRabbitListenerContainerFactory.

But imagine that there are several virtual hosts in RabbitMQ. By default, in Spring AMQP, you need to create such an ecosystem for each virtual host with manual configuration of RabbitTemplateand RabbitListener.

As you can see, all this configuration takes time and can be a headache.

Aware of this problem, the proposed library automatically configures all these beans for you. The only thing you need to do is determine the configuration in application.properties, and magic will happen!

Wow, but what does the configuration look like in application.properties?

Everything is very simple. Firstly, there is a prefix for the properties - this spring.rabbitmq.custom.

After that, you must specify the name of the event. This is a very important part, since all the settings that the library makes are based on this name.

After the event name, you can specify the property and value. All properties are described in the documentation .

So, the template is as follows: Below is an example configuration for two different connections.

spring.rabbitmq.custom.<YOUR_EVENT>.<PROPERTY>=<VALUE>

spring.rabbitmq.custom.some-event.host=localhost
spring.rabbitmq.custom.some-event.port=5672
spring.rabbitmq.custom.some-event.ttlRetryMessage=5000
spring.rabbitmq.custom.some-event.maxRetriesAttempts=5
spring.rabbitmq.custom.some-event.ttlMultiply=2
spring.rabbitmq.custom.some-event.queueRoutingKey=ROUTING.KEY.TEST
spring.rabbitmq.custom.some-event.exchange=ex.custom.direct
spring.rabbitmq.custom.some-event.exchangeType=direct
spring.rabbitmq.custom.some-event.queue=queue.custom.test
spring.rabbitmq.custom.some-event.autoCreate=true
spring.rabbitmq.custom.some-event.concurrentConsumers=1
spring.rabbitmq.custom.some-event.maxConcurrentConsumers=1
spring.rabbitmq.custom.some-event.virtualHost=tradeshift
spring.rabbitmq.custom.some-event.primary=true
spring.rabbitmq.custom.some-event.sslConnection=true
spring.rabbitmq.custom.some-event.tlsKeystoreLocation=file:///etc/tradeshift/your-service/tls-keystore.pkcs12
spring.rabbitmq.custom.some-event.tlsKeystorePassword=${RABBITMQ_PASS_CERT}

spring.rabbitmq.custom.another-event.host=localhost
spring.rabbitmq.custom.another-event.port=5672
spring.rabbitmq.custom.another-event.ttlRetryMessage=5000
spring.rabbitmq.custom.another-event.maxRetriesAttempts=5
spring.rabbitmq.custom.another-event.queueRoutingKey=TEST.QUEUE
spring.rabbitmq.custom.another-event.exchange=ex_test_1
spring.rabbitmq.custom.another-event.exchangeType=direct
spring.rabbitmq.custom.another-event.queue=queue_test_1
spring.rabbitmq.custom.another-event.autoCreate=true
spring.rabbitmq.custom.another-event.concurrentConsumers=1
spring.rabbitmq.custom.another-event.maxConcurrentConsumers=1
spring.rabbitmq.custom.another-event.username=guest
spring.rabbitmq.custom.another-event.password=${RABBITMQ_PASS}

As you can see, we have not written a single line of code !!!

Good, but you were saying something about the new replay strategy, right?

Yes my friend said!

But before explaining this new strategy, let's take a look at RabbitMQ and Spring AMQP's default behavior.

By default, RabbitMQ does not provide retry processing that would allow you to control the entire message life cycle.

For example, prior to RabbitMQ 3.8 , the message header did not have a property to control the number of retries made.

RabbitMQ default behavior:

If you have not defined time-to-live (TTL, lifetime), then RabbitMQ will constantly try to queue your message.
If you defined TTL but did not define dlx , then after TTL the message will be removed from the queue and you will lose it.
If you have defined TTL and dlx , then after TTL the message will be sent to the exchanger defined in dlx.

RabbitMQ default behavior

But what if we want an increasing TTL (for example, in case of unstable operation) and control the number of retries?

Great question!

It's time to explain how Spring AQMP and Spring Rabbit Tuning library work!

Retry strategies

Spring AMQP by default

When using Spring AMQP by default, you can define retry options using the properties below.

But this approach has problems. By default, Spring AMQP will block your queue when you try to deliver a message again.

This problem can be solved in a roundabout way: use parallelism. But in this way we load the JVM, and this is not the best approach. If you have five problematic messages (as in the example), then a bottleneck will again arise, and we still need to manually determine the beans in the @Configuration- bin for each connection and container.

spring.rabbitmq.listener.simple.retry.enabled=true
spring.rabbitmq.listener.simple.retry.initial-interval=2000
spring.rabbitmq.listener.simple.retry.max-attempts=5
spring.rabbitmq.listener.simple.retry.multiplier=2
spring.rabbitmq.listener.simple.max-concurrency=5
spring.rabbitmq.listener.simple.concurrency=5

Spring RabbitMQ Tuning

This library takes a different approach with a separate queue for retries. This way we can control TTL using the expiration message parameter, and the x-death parameter to control the number of retries.

But how?

We use the dlx concept in the retry queue to resend the message to the main queue. Thus, we have access to the x-death parameter and we can programmatically determine the lifetime of the message.

Note. Since this library is an extension of Spring AMQP, you can use the default retry strategy, and use this library only for automatic configuration of beans.

Spring RabbitMQ Tuning Retry

Can I use your retry strategy without autoconfiguration? I already have many beans, and I don’t have time to rewrite them.

We know that the world is not perfect, and for this situation, we have a flag that allows you to disable auto-configuration and use only our approach to repetition processing and other advantages, such as bin management.

You can disable autoconfiguration using the following property:

spring.rabbitmq.enable.custom.autoconfiguration=false

But I have two different connections, what should I do in this case?

If you have more than one connection and you want to disable autoconfiguration, then you need to specify the bean name RabbitTemplatefor each connection. This is described here . You can still use our bean to make working with Spring AMQP easier.

spring.rabbitmq.custom.<YOUR_EVENT>.rabbitTemplateBeanName= <RABBITTEMPLATE_BEAN_NAME>

RabbitTemplateHandler

Very well! So, I want to use this. How can i do this? Do you have an example or documentation?

Yes!

We have an example project in this repository , where you can see how to use this library for publishers and subscribers.

But it is so simple that I will set an example here!

Publisher

There is a class in the library RabbitTemplateHandlerand it is very easy to use. You need to call the method getRabbitTemplateand pass the virtual host as a parameter to get the bean RabbitTemplate. After that, you can call the convertAndSend method and pass it in the exchange and routing key parameters.

Note : exchange and routing key can be obtained using annotationValue.

An example :

@Value("${spring.rabbitmq.custom.some-event.exchange}")
private String exchangeSomeEvent;

@Value("${spring.rabbitmq.custom.some-event.queueRoutingKey}")
private String routingKeySomeEvent;

@Autowired
private final RabbitTemplateHandler rabbitTemplateHandler;

public void sendMessage(final String message) {
   rabbitTemplateHandler.getRabbitTemplate("some-event").convertAndSend(exchangeSomeEvent, routingKeySomeEvent, message);
}

Subscriber

Subscribers are also very simple. The only thing you need to do is add an annotation to the method RabbitListener(Spring AMQP annotation by default) and pass the name containerFactory.

How to find out the correct containerFactory name for a virtual host?

You do not need to know it, just pass the name of the event, and the library will do all the magic for you! This library also has the ability to include retries and dlq, which are recommended.

Turning them on is very simple. You need to add annotation to the method EnableRabbitRetryAndDlqand pass the name of the property as an argument.

Note : you can also specify which exceptions to handle with retries and dlq. It is processed by default Exception.class, which means handling all exceptions.

Example:

@RabbitListener(containerFactory = "some-event", queues = "${spring.rabbitmq.custom.some-event.queue}")
@EnableRabbitRetryAndDlq(event = "some-event", exceptions = { IllegalArgumentException.class, RuntimeException.class })
public void onMessage(Message message) {
   ...
}

It's all! I hope you enjoy this library as much as we do!

And, most importantly: feel free to contribute or send feedback!

Special thanks to

Andre Luis Gomes
Rogerio Bueno
Leonardo Ferreira

Simple integration of RabbitMQ and Spring Boot