Broker Connections

Instead of waiting for clients to connect, a broker can initiate a connection to another endpoint on a specific protocol.

Currently, this feature supports only the AMQP protocol. However, in the future, it might be expanded to other protocols.

Broker connections are configured by the <broker-connections> XML element in the broker.xml configuration file.

  1. <broker-connections>
  2.   ...
  3. </broker-connections>

AMQP Server Connections

An ActiveMQ Artemis broker can initiate connections using the AMQP protocol. This means that the broker can connect to another AMQP server (not necessarily ActiveMQ Artemis) and create elements on that connection.

To define an AMQP broker connection, add an <amqp-connection> element within the <broker-connections element in the broker.xml configuration file. For example:

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://HOST:PORT" name="other-server" retry-interval="100" reconnect-attempts="-1" user="john" password="doe">
  3.     ...
  4.   </amqp-connection>
  5. </broker-connections>
  • uri: tcp://host:myport (this is a required argument)
  • name: Name of the connection used for management purposes
  • user: User name with which to connect to the endpoint (this is an optional argument)
  • password: Password with which to connect to the endpoint (this is an optional argument)
  • retry-interval: Time, in milliseconds to wait before retrying a connection after an error. The default value is 5000.
  • reconnect-attempts: default is -1 meaning infinite
  • auto-start : Should the broker connection start automatically with the broker. Default is true. If false it is necessary to call a management operation to start it.

Notice: If auto-start is disabled on the broker connection, the start of the broker connection will only happen after the management method startBrokerConnection(connectionName) is called on the ServerController.

Important: The target endpoint needs permission for all operations that configured. Therefore, If a security manager is being used, ensure to perform the configured operations with a user with sufficient permissions.

AMQP Server Connection Operations

The following types of operations are supported on an AMQP server connection:

  • Mirrors
    • The broker uses an AMQP connection to another broker and duplicates messages and sends acknowledgements over the wire.
  • Senders
    • Messages received on specific queues are transferred to another endpoint.
  • Receivers
    • The broker pulls messages from another endpoint.
  • Peers
    • The broker creates both senders and receivers on another endpoint that knows how to handle them. This is currently implemented by Apache Qpid Dispatch.

Reconnecting and Failover

It is possible to determine how reconnection will happen on a broker connection.

These are the attributes are available on amqp-connection XML element:

  • reconnect-attempts: default as -1 (infinite). How many attempts will be done after a failed connection
  • retry-interval: default as 5000, in milliseconds, the wait between each retry in connections.

It is also possible to specify alternate hosts on a broker connection by appending a comma separated list after a # at the end of the URI. The broker connection would keep trying on the alternate list until one of the targets is available to connect. Example:

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://ServerA:5672#BackupA:5672,BackupB:5672" name="ServerA" reconnect-attempts="-1" retry-interval="5000">
  3.     ...
  4.   </amqp-connection>
  5. </broker-connections>

Broker Connection Failover Figure 1. Broker Connection - Reconnecting and Failover.

The previous example portrays a case of connection failure towards ServerA. The system would try to connect to serverA, backupA, and backupB until it successfully connects to one of these nodes.

Mirroring

The idea of mirroring is to send events that happen on a broker towards another broker, without blocking any operations from producers and consumers, allowing them to keep operating as fast as possible. It can be used for Disaster Recovery, and depending on the requirements even for failing over the data.

The following events are sent through mirroring:

  • Message sending
    • Messages sent to one broker will be “replicated” to the target broker.
  • Message acknowledgement
    • Acknowledgements removing messages at one broker will be sent to the target broker.
    • Note that if the message is pending for a consumer on the target mirror, the ack will not succeed and the message might be delivered by both brokers.
  • Queue and address creation.
  • Queue and address deletion.

Mirror configuration

Add a <mirror> element within the <amqp-connection> element to configure mirroring to the target broker.

The following optional arguments can be utilized:

  • queue-removal: Specifies whether a queue- or address-removal event is sent. The default value is true.
  • message-acknowledgements: Specifies whether message acknowledgements are sent. The default value is true.
  • queue-creation: Specifies whether a queue- or address-creation event is sent. The default value is true.

An example of a mirror configuration is shown below:

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://HOST:PORT" name="mirror">
  3.     <mirror/>
  4.   </amqp-connection>
  5. </broker-connections>

Store and Forward Queue

Mirror events are always stored on a local queue prefixed as “$ACTIVEMQARTEMIS_MIRROR“ and then concatenated with the broker connection’s configured name.

So, in the following configuration mirror events will be stored on a queue named “$ACTIVEMQ_ARTEMIS_MIRROR_brokerB”.

  1. <broker-connection>
  2.   <amqp-connection uri="tcp://brokerB:5672" name="brokerB">
  3.     <mirror/>
  4.   </amqp-connection>
  5. </broker-connection>

These messages are then transferred to brokerB:5672. A producer to the address $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will be created towards brokerB. If there is a security manager configured, security roles must be provided to the user on the broker connection.

Notice the queue $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will not actually exist on brokerB and so it wont be visible on the administration console. The target broker will treat these messages accordingly as mirror events and perform the appropriate operations at the target broker.

Pre Existing Messages

The broker will only mirror messages arriving from the point in time the mirror was configured. Previously existing messages will not be forwarded to other brokers.

Dual Mirror (Disaster Recovery)

ActiveMQ Artemis supports automatic fallback mirroring. Every sent message and every acknowledgement is asynchronously replicated to the mirrored broker.

On the following diagram, there will be two servers called DataCenter1, and DataCenter2. In order to have a dual mirror configuration, it is necessary is to add the mirror broker connection on each broker.xml:

Broker Connection DR Figure 2. Broker Connection - Disaster Recovery.

on DataCenter1, the following code should be added on broker.xml:

  1. <broker-connections>
  2.    <amqp-connection uri="tcp://DataCenter2:5672" name="DC2">
  3.       <mirror/>
  4.    </amqp-connection>
  5. </broker-connections>

The following xml should be added on DataCenter2’s broker.xml:

  1. <broker-connections>
  2.    <amqp-connection uri="tcp://DataCenter1:5672" name="DC1">
  3.       <mirror/>
  4.    </amqp-connection>
  5. </broker-connections>

The broker connections will replicate sends and acknowledgements to the other broker, no matter where they originated. If messages are sent on DC1 (DataCenter1) these will be automatically transferred to DC2 (DataCenter2). Messages acknowledgements received on DC2 will be automatically related back to DC1. The only exception to that rule would be if there were already consumers with pending messages on any server, where a mirrored acknowledgement will not prevent the message being consumed by both consumers. It is recommended to not have active consumers on both servers.

Example

There is an example as part of the distribution showing dual broker configuration (or disaster recovery) under ./examples/features/broker-connection/disaster-recovery.

On the example two brokers are configured to mirror each other and whatever happens in one broker is immediately copied over to the other broker.

Senders and Receivers

It is possible to connect an ActiveMQ Artemis broker to another AMQP endpoint simply by creating a sender or receiver broker connection element.

For a sender, the broker creates a message consumer on a queue that sends messages to another AMQP endpoint.

For a receiver, the broker creates a message producer on an address that receives messages from another AMQP endpoint.

Both elements function as a message bridge. However, there is no additional overhead required to process messages. Senders and receivers behave just like any other consumer or producer in ActiveMQ Artemis.

Specific queues can be configured by senders or receivers. Wildcard expressions can be used to match senders and receivers to specific addresses or sets of addresses. When configuring a sender or receiver, the following properties can be set:

  • address-match: Match the sender or receiver to a specific address or set of addresses, using a wildcard expression
  • queue-name: Configure the sender or receiver for a specific queue

Some examples are shown below.

Using address expressions:

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://HOST:PORT" name="other-server">
  3.     <sender address-match="queues.#"/>
  4.     <!-- notice the local queues for remotequeues.# need to be created on this broker -->
  5.     <receiver address-match="remotequeues.#"/>
  6.   </amqp-connection>
  7. </broker-connections>
  9. <addresses>
  10.   <address name="remotequeues.A">
  11.     <anycast>
  12.       <queue name="remoteQueueA"/>
  13.     </anycast>
  14.   </address>
  15.   <address name="queues.B">
  16.     <anycast>
  17.       <queue name="localQueueB"/>
  18.     </anycast>
  19.   </address>
  20. </addresses>

Using queue names:

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://HOST:PORT" name="other-server">
  3.     <receiver queue-name="remoteQueueA"/>
  4.     <sender queue-name="localQueueB"/>
  5.   </amqp-connection>
  6. </broker-connections>
  8. <addresses>
  9.    <address name="remotequeues.A">
  10.      <anycast>
  11.        <queue name="remoteQueueA"/>
  12.      </anycast>
  13.    </address>
  14.    <address name="queues.B">
  15.      <anycast>
  16.        <queue name="localQueueB"/>
  17.      </anycast>
  18.    </address>
  19. </addresses>

Important: Receivers can only be matched to a local queue that already exists. Therefore, if receivers are being used, ensure that queues are pre-created locally. Otherwise, the broker cannot match the remote queues and addresses.

Important: Do not create a sender and a receiver to the same destination. This creates an infinite loop of sends and receives.

Peers

The broker can be configured as a peer which connects to the Apache Qpid Dispatch Router and instructs it that the broker will act as a store-and-forward queue for a given AMQP waypoint address configured on the router. In this scenario, clients connect to a router to send and receive messages using a waypointed address, and the router routes these messages to or from the queue on the broker.

The peer configuration causes ActiveMQ Artemis to create a sender and receiver pair for each destination matched in the broker-connection configuration, with these carrying special configuration to let Qpid Dispatch know to collaborate with the broker. This replaces the traditional need of a router-initiated connection and auto-links.

Qpid Dispatch Router offers a lot of advanced networking options that be used together with ActiveMQ Artemis.

With a peer configuration, the same properties are present as when there are senders and receivers. For example, a configuration where queues with names beginning “queue.” act as storage for the matching router waypoint address would be:

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://HOST:PORT" name="router">
  3.     <peer address-match="queues.#"/>
  4.   </amqp-connection>
  5. </broker-connections>
  7. <addresses>
  8.    <address name="queues.A">
  9.      <anycast>
  10.        <queue name="queues.A"/>
  11.      </anycast>
  12.    </address>
  13.    <address name="queues.B">
  14.      <anycast>
  15.        <queue name="queues.B"/>
  16.      </anycast>
  17.    </address>
  18. </addresses>

There must be a matching address waypoint configuration on the router instructing it that the particular router addresses the broker attaches to should be treated as waypoints. For example, a similar prefix- based router address configuration would be:

  1. address {
  2.     prefix: queue
  3.     waypoint: yes
  4. }

For more information refer to the “brokered messaging” documentation for Apache Qpid Dispatch Router.

Important: Do not use this feature to connect to another broker, otherwise any message sent will be immediately ready to consume creating an infinite echo of sends and receives.

Important: It is not necessary to configure the router with a connector or auto-links to communicate with the broker. The brokers peer configuration replaces these aspects of the router waypoint usage.

Address Consideration

It is highly recommended that address name and queue name are the same. When a queue with its distinct name (as in the following example) is used, senders and receivers will always use the address name when creating the remote endpoint.

  1. <broker-connections>
  2.   <amqp-connection uri="tcp://HOST:PORT" name="other-server">
  3.     <sender address-match="queues.#"/>
  4.   </amqp-connection>
  5. </broker-connections>
  6. <addresses>
  7.   <address name="queues.A">
  8.     <anycast>
  9.       <queue name="distinctNameQueue.A"/>
  10.     </anycast>
  11.   </address>
  12. </addresses>

In the above example the broker connection would create an AMQP sender towards “queues.A”.

Important: To avoid confusion it is recommended that address name and queue name are kept the same.