Instead of waiting for clients to connect, a broker can initiate a connection to another endpoint on a specific protocol.

Currently, this feature supports only the AMQP protocol. However, in the future, it might be expanded to other protocols.

Broker connections are configured by the <broker-connections> XML element in the broker.xml configuration file.

<broker-connections>
  ...
</broker-connections>

1. AMQP Server Connections

An ActiveMQ Artemis broker can initiate connections using the AMQP protocol. This means that the broker can connect to another AMQP server (not necessarily ActiveMQ Artemis) and create elements on that connection.

To define an AMQP broker connection, add an <amqp-connection> element within the <broker-connections element in the broker.xml configuration file. For example:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server" retry-interval="100" reconnect-attempts="-1" user="john" password="doe">
    ...
  </amqp-connection>
</broker-connections>

uri

: tcp://host:myport[?options] (this is a required argument)

name

: Name of the connection used for management purposes

user

: User name with which to connect to the endpoint (this is an optional argument)

password

: Password with which to connect to the endpoint (this is an optional argument)

retry-interval

: Time, in milliseconds to wait before retrying a connection after an error. The default value is 5000.

reconnect-attempts

: default is -1 meaning infinite

auto-start

: Should the broker connection start automatically with the broker. Default is true. If false it is necessary to call a management operation to start it.

2. AMQP Server Connection Operations

The following types of operations are supported on an AMQP server connection:

• Mirrors

  • The broker uses an AMQP connection to another broker and duplicates messages and sends acknowledgements over the wire.

• Senders

  • Messages received on specific queues are transferred to another endpoint.

• Receivers

  • The broker pulls messages from another endpoint.

• Peers

  • The broker creates both senders and receivers on another endpoint that knows how to handle them. This is currently implemented by Apache Qpid Dispatch.

3. Reconnecting and Failover

It is possible to determine how reconnection will happen on a broker connection.

These are the attributes that are available on the amqp-connection XML element:

• reconnect-attempts: default is -1 (infinite). How many attempts will be done after a failed connection

• retry-interval: default is 5000, in milliseconds, the wait between each retry of a connection.

It is also possible to specify alternate hosts on a broker connection by appending a comma separated list after a # at the end of the URI. The broker connection would keep trying on the alternate list until one of the targets is available to connect. Example:

<broker-connections>
  <amqp-connection uri="tcp://ServerA:5672#BackupA:5672,BackupB:5672" name="ServerA" reconnect-attempts="-1" retry-interval="5000">
    ...
  </amqp-connection>
</broker-connections>

Figure 1. Broker Connection - Reconnecting and Failover.

The previous example portrays a case of connection failure towards ServerA. The system would try to connect to serverA, backupA, and backupB until it successfully connects to one of these nodes.

4. Mirroring

Mirroring will reproduce any operation that happened on the source brokers towards a target broker.

The following events are sent through mirroring:

• Message sending

  • Messages sent to one broker will be “replicated” to the target broker.

• Message acknowledgement

  • Acknowledgements removing messages at one broker will be sent to the target broker.

  • Note that if the message is pending for a consumer on the target mirror, the ack will not succeed and the message might be delivered by both brokers.

• Queue and address creation.

• Queue and address deletion.

By default every operation is sent asynchronously without blocking any clients. However if you set sync=”true” on the mirror configuration, the clients will always wait for the mirror on every blocking operation.

4.1. Mirror configuration

Add a <mirror> element within the <amqp-connection> element to configure mirroring to the target broker.

The following optional arguments can be utilized:

queue-removal

: Specifies whether a queue- or address-removal event is sent. The default value is true.

message-acknowledgements

: Specifies whether message acknowledgements are sent. The default value is true.

queue-creation

: Specifies whether a queue- or address-creation event is sent. The default value is true.

address-filter

: An optional comma-separated list of inclusion and/or exclusion filter entries used to govern which addresses (and related queues) mirroring events will be created for on this broker-connection. That is, events will only be mirrored to the target broker for addresses that match the filter. An address is matched when it begins with an inclusion entry specified in this field, unless the address is also explicitly excluded by another entry. An exclusion entry is prefixed with ! to denote any address beginning with that value does not match. If no inclusion entry is specified in the list, all addresses not explicitly excluded will match. If the address-filter attribute is not specified, then all addresses (and related queues) will match and be mirrored.

Examples:

• ‘eu’ matches all addresses starting with ‘eu’

• ‘!eu’ matches all address except for those starting with ‘eu’

• ‘eu.uk,eu.de’ matches all addresses starting with either ‘eu.uk’ or ‘eu.de’

• ‘eu,!eu.uk’ matches all addresses starting with ‘eu’ but not those starting with ‘eu.uk’

sync

By default is false. If set it to true any client blocking operation will be held until the mirror has confirmed receiving the operation.

• Notice that a disconnected node would hold all operations from the client. If you set sync=true you must reconnect a mirror before performing any operations.

An example of a mirror configuration is shown below:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="mirror">
    <mirror/>
  </amqp-connection>
</broker-connections>

4.2. Store and Forward Queue

Mirror events are always stored on a local queue prefixed as “$ACTIVEMQ_ARTEMIS_MIRROR_“ and then concatenated with the broker connection’s configured name.

So, in the following configuration mirror events will be stored on a queue named “$ACTIVEMQ_ARTEMIS_MIRROR_brokerB”.

<broker-connection>
  <amqp-connection uri="tcp://brokerB:5672" name="brokerB">
    <mirror/>
  </amqp-connection>
</broker-connection>

These messages are then transferred to brokerB:5672. A producer to the address $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will be created towards brokerB. If there is a security manager configured, security roles must be provided to the user on the broker connection.

Notice the queue $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will not actually exist on brokerB and so it wont be visible on the administration console. The target broker will treat these messages accordingly as mirror events and perform the appropriate operations at the target broker.

4.3. Pre Existing Messages

The broker will only mirror messages arriving from the point in time the mirror was configured. Previously existing messages will not be forwarded to other brokers.

5. Dual Mirror (Disaster Recovery)

ActiveMQ Artemis supports automatic fallback mirroring. Every sent message and every acknowledgement is asynchronously replicated to the mirrored broker.

On the following diagram, there will be two servers called DataCenter1, and DataCenter2. In order to have a dual mirror configuration, it is necessary is to add the mirror broker connection on each broker.xml:

Figure 2. Broker Connection - Disaster Recovery.

on DataCenter1, the following code should be added on broker.xml:

<broker-connections>
   <amqp-connection uri="tcp://DataCenter2:5672" name="DC2">
      <mirror/>
   </amqp-connection>
</broker-connections>

The following xml should be added on DataCenter2’s broker.xml:

<broker-connections>
   <amqp-connection uri="tcp://DataCenter1:5672" name="DC1">
      <mirror/>
   </amqp-connection>
</broker-connections>

The broker connections will replicate sends and acknowledgements to the other broker, no matter where they originated. If messages are sent on DC1 (DataCenter1) these will be automatically transferred to DC2 (DataCenter2). Messages acknowledgements received on DC2 will be automatically related back to DC1. The only exception to that rule would be if there were already consumers with pending messages on any server, where a mirrored acknowledgement will not prevent the message being consumed by both consumers. It is recommended to not have active consumers on both servers.

6. Example

There is an example as part of the distribution showing dual broker configuration (or disaster recovery) under ./examples/features/broker-connection/disaster-recovery.

On the example two brokers are configured to mirror each other and whatever happens in one broker is immediately copied over to the other broker.

7. Senders and Receivers

It is possible to connect an ActiveMQ Artemis broker to another AMQP endpoint simply by creating a sender or receiver broker connection element.

For a sender, the broker creates a message consumer on a queue that sends messages to another AMQP endpoint.

For a receiver, the broker creates a message producer on an address that receives messages from another AMQP endpoint.

Both elements function as a message bridge. However, there is no additional overhead required to process messages. Senders and receivers behave just like any other consumer or producer in ActiveMQ Artemis.

Specific queues can be configured by senders or receivers. Wildcard expressions can be used to match senders and receivers to specific addresses or sets of addresses. When configuring a sender or receiver, the following properties can be set:

address-match

: Match the sender or receiver to a specific address or set of addresses, using a wildcard expression

queue-name

: Configure the sender or receiver for a specific queue

Some examples are shown below.

Using address expressions:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server">
    <sender address-match="queues.#"/>
    <!-- notice the local queues for remotequeues.# need to be created on this broker -->
    <receiver address-match="remotequeues.#"/>
  </amqp-connection>
</broker-connections>
<addresses>
  <address name="remotequeues.A">
    <anycast>
      <queue name="remoteQueueA"/>
    </anycast>
  </address>
  <address name="queues.B">
    <anycast>
      <queue name="localQueueB"/>
    </anycast>
  </address>
</addresses>

Using queue names:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server">
    <receiver queue-name="remoteQueueA"/>
    <sender queue-name="localQueueB"/>
  </amqp-connection>
</broker-connections>
<addresses>
   <address name="remotequeues.A">
     <anycast>
       <queue name="remoteQueueA"/>
     </anycast>
   </address>
   <address name="queues.B">
     <anycast>
       <queue name="localQueueB"/>
     </anycast>
   </address>
</addresses>

8. Peers

The broker can be configured as a peer which connects to the Apache Qpid Dispatch Router and instructs it that the broker will act as a store-and-forward queue for a given AMQP waypoint address configured on the router. In this scenario, clients connect to a router to send and receive messages using a waypointed address, and the router routes these messages to or from the queue on the broker.

The peer configuration causes ActiveMQ Artemis to create a sender and receiver pair for each destination matched in the broker-connection configuration, with these carrying special configuration to let Qpid Dispatch know to collaborate with the broker. This replaces the traditional need of a router-initiated connection and auto-links.

Qpid Dispatch Router offers a lot of advanced networking options that be used together with ActiveMQ Artemis.

With a peer configuration, the same properties are present as when there are senders and receivers. For example, a configuration where queues with names beginning “queue.” act as storage for the matching router waypoint address would be:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="router">
    <peer address-match="queues.#"/>
  </amqp-connection>
</broker-connections>
<addresses>
   <address name="queues.A">
     <anycast>
       <queue name="queues.A"/>
     </anycast>
   </address>
   <address name="queues.B">
     <anycast>
       <queue name="queues.B"/>
     </anycast>
   </address>
</addresses>

There must be a matching address waypoint configuration on the router instructing it that the particular router addresses the broker attaches to should be treated as waypoints. For example, a similar prefix- based router address configuration would be:

address {
    prefix: queue
    waypoint: yes
}

For more information refer to the “brokered messaging” documentation for Apache Qpid Dispatch Router.

9. Address Consideration

It is highly recommended that address name and queue name are the same. When a queue with its distinct name (as in the following example) is used, senders and receivers will always use the address name when creating the remote endpoint.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server">
    <sender address-match="queues.#"/>
  </amqp-connection>
</broker-connections>
<addresses>
  <address name="queues.A">
    <anycast>
      <queue name="distinctNameQueue.A"/>
    </anycast>
  </address>
</addresses>

In the above example the broker connection would create an AMQP sender towards “queues.A”.

10. Federation

Broker federation allows the local broker to create remote receivers for addresses or queues that have local demand, conversely the broker connection can send federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection.

Add a <federation> element within the <amqp-connection> element to configure federation to the broker instance, the <amqp-connection> contains all the configuration for authentication and reconnection handling, see the above sections to configure those values.

The broker connection federation configuration consists of one or more policies that define either local or remote federation configurations for addresses or queues.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="federation-example">
    <federation>
       <local-address-policy name="example-local-address-policy">
         <include address-match="local-address.#" />
         <exclude address-match="local-address.excluded" />
       </local-address-policy>
       <local-queue-policy name="example-local-queue-policy">
         <include address-match="address" queue-match="local-queue" />
       </local-queue-policy>
       <remote-address-policy name="example-remote-address-policy">
         <include address-match="remote-address" />
       </remote-address-policy>
       <remote-queue-policy name="example-remote-queue-policy">
         <include address-match="#" queue-match="remote-queue" />
         <exclude address-match="excluded.#" queue-match="remote-queue-excluded" />
       </remote-queue-policy>
    </federation>
  </amqp-connection>
</broker-connections>

Local and remote address federation

Local or Remote address federation configures the local or remote broker to watch for demand on addresses and when demand exists it will create a consumer on the matching address on the opposing broker. Because the consumers are created on addresses on the opposing broker the authentication credentials supplied to the broker connection must have sufficient access to read from (local address federation) or write to the matching address (remote address federation) on the opposing broker.

An example of address federation configuration is shown below, the local and remote address policies have identical configuration parameters only the policy element names are different.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="federation-example">
    <federation>
       <local-address-policy name="example-local-address-policy"
                             auto-delete="false"
                             auto-delete-delay="1"
                             auto-delete-message-count="12"
                             max-hops="1"
                             enable-divert-bindings="true">
         <include address-match="local-address.#" />
         <exclude address-match="local-address.excluded" />
       </local-address-policy>
    </federation>
  </amqp-connection>
</broker-connections>

name

The name of the policy, these names should be unique within a broker connection’s federation policy elements.

include

The address-match pattern to use to match included addresses, multiple of these can be set. If none are set all addresses are matched.

exclude

The address-match pattern to use to match excluded addresses, multiple of these can be set or it can be omitted if no excludes are needed.

max-hops

The number of hops that a message can have made for it to be federated, the default value is zero and will work for most simple federatio0n deployments however is certain topologies a higher value may be required to prevent looping.

auto-delete

For address federation, a durable queue is created on the opposing broker for the matching address. This option is used to mark if the queue should be deleted once the initiating broker disconnects, and the delay and message count parameters have been met. This is useful if you want to automate the clean up, though you may wish to disable this if you want messages to queued for the opposing broker when it disconnects. The default value is false and the federation consumer queue will not be auto deleted.

auto-delete-delay

The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for auto-delete. The default value is zero if the option is omitted.

auto-delete-message-count

The amount number messages in the remote queue that the message count must be equal or below before the initiating broker has disconnected before the queue can be eligible for auto deletion. The default value is zero if the option is omitted.

enable-divert-bindings

Setting to true will enable divert bindings to be listened for demand. If there is a divert binding with an address that matches the included addresses for the federation, any queue bindings that match the forward address of the divert will create demand. The default value for this option is false.

Local and remote queue federation

Local or Remote queue federation configures the local or remote broker to watch for demand on queues and when demand exists it will create a consumer on the matching queue on the opposing broker. Because the consumers are created on queues on the opposing broker the authentication credentials supplied to the broker connection must have sufficient access to read from (local queue federation) or write to the matching queue (remote queue federation) on the opposing broker.

An example of queue federation configuration is shown below, the local and remote queue policies have identical configuration parameters only the policy element names are different.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="federation-example">
    <federation>
       <local-queue-policy name="example-local-queue-policy">
         <include address-match="#" queue-match="remote-queue" />
         <exclude address-match="excluded.#" queue-match="remote-queue-excluded" />
       </local-queue-policy>
    </federation>
  </amqp-connection>
</broker-connections>

name

The name of the policy, these names should be unique within a broker connection’s federation policy elements.

include

The queue-match pattern to use to match included queues, multiple of these can be set. If none are set all queues are matched.

exclude

The queue-match pattern to use to match excluded queues, multiple of these can be set or it can be omitted if no excludes are needed.

priority-adjustment

When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue. The default value for this configuration option if not set is -1.

include-federated

Controls if consumers on a queue which come from federation instances should be counted when observing a queue for demand, by default this value is false and federation consumers are not counted.