概述
JBoss 系列五十三：JBoss 7/WildFly 集群之 HornetQ Messaging中讨论了JBoss 7/WildFly 集群中HornetQ Messaging的高可用性和负载均衡的原理和配置注意事项，本文通过实验，模拟一个HornetQ Messaging的集群环境，演示说明JBoss 7/WildFly 集群中HornetQ Messaging的高可用性和负载均衡。本实验运行示意图：
如图所示本实验中：
- 两个JBoss节点（node1和node2）构成一个集群，两个节点上分别有live消息服务器和backup消息服务器，且node1上的live消息服务器和node2上的backup消息服务器位于group-1，而node2上的live消息服务器和node1上的backup消息服务器位于group-2
- 消息队列DistributedQueue部署在两个节点上的live服务器，消息驱动Bean（MessageDrivenBean）作为消息队列的消费者监听于DistributedQueue之上
- JMS客户端（JMSClient）通过node1发送连续的10条消息到集群
- JBoss node1部署于物理机器10.66.218.47上，JBoss node2部署于物理机器10.66.218.48上
JBoss 端配置
我们需要配置live消息服务器和backup消息服务器，我们分别给出node1和node2的配置。
node1配置
在<subsystem xmlns="urn:jboss:domain:messaging:中配置hornetq-server：
<subsystem xmlns="urn:jboss:domain:messaging:1.3">
<hornetq-server>
<persistence-enabled>true</persistence-enabled>
<cluster-password>clusterPassword1!</cluster-password>
<backup>false</backup>
<allow-failback>true</allow-failback>
<failover-on-shutdown>false</failover-on-shutdown>
<shared-store>false</shared-store>
<journal-type>NIO</journal-type>
<journal-min-files>2</journal-min-files>
<check-for-live-server>true</check-for-live-server>
<backup-group-name>group-1</backup-group-name>
<connectors>
<netty-connector name="netty" socket-binding="messaging" />
<netty-connector name="netty-throughput"
socket-binding="messaging-throughput">
<param key="batch-delay" value="50" />
</netty-connector>
<in-vm-connector name="in-vm" server-id="1" />
</connectors>
<acceptors>
<netty-acceptor name="netty" socket-binding="messaging" />
<netty-acceptor name="netty-throughput"
socket-binding="messaging-throughput">
<param key="batch-delay" value="50" />
<param key="direct-deliver" value="false" />
</netty-acceptor>
<in-vm-acceptor name="in-vm" server-id="1" />
</acceptors>
<broadcast-groups>
<broadcast-group name="bg-group1">
<socket-binding>messaging-group</socket-binding>
<broadcast-period>5000</broadcast-period>
<connector-ref>netty</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="dg-group1">
<socket-binding>messaging-group</socket-binding>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="my-cluster">
<address>jms</address>
<connector-ref>netty</connector-ref>
<discovery-group-ref discovery-group-name="dg-group1" />
</cluster-connection>
</cluster-connections>
<security-settings>
<security-setting match="#">
<permission type="send" roles="guest" />
<permission type="consume" roles="guest" />
<permission type="createNonDurableQueue" roles="guest" />
<permission type="deleteNonDurableQueue" roles="guest" />
</security-setting>
</security-settings>
<address-settings>
<!--default for catch all -->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<redistribution-delay>1000</redistribution-delay>
<max-size-bytes>10485760</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
<message-counter-history-day-limit>10
</message-counter-history-day-limit>
</address-setting>
</address-settings>
<jms-connection-factories>
<connection-factory name="InVmConnectionFactory">
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/ConnectionFactory" />
</entries>
</connection-factory>
<connection-factory name="RemoteConnectionFactory">
<connectors>
<connector-ref connector-name="netty" />
</connectors>
<entries>
<entry name="java:jboss/exported/jms/RemoteConnectionFactory" />
</entries>
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1.0</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
</connection-factory>
<pooled-connection-factory name="hornetq-ra">
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
<transaction mode="xa" />
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/JmsXA" />
</entries>
</pooled-connection-factory>
</jms-connection-factories>
</hornetq-server>
<hornetq-server name="backup-server">
<persistence-enabled>true</persistence-enabled>
<cluster-password>clusterPassword1!</cluster-password>
<backup>true</backup>
<allow-failback>true</allow-failback>
<failover-on-shutdown>false</failover-on-shutdown>
<shared-store>false</shared-store>
<journal-type>NIO</journal-type>
<journal-min-files>2</journal-min-files>
<check-for-live-server>true</check-for-live-server>
<backup-group-name>group-2</backup-group-name>
<paging-directory path="messagingpagingbackupa" />
<bindings-directory path="messagingbindingsbackupa" />
<journal-directory path="messagingjournalbackupa" />
<large-messages-directory path="messaginglargemessagesbackupa" />
<connectors>
<netty-connector name="netty" socket-binding="messaging-backup" />
<in-vm-connector name="in-vm" server-id="2" />
</connectors>
<acceptors>
<netty-acceptor name="netty" socket-binding="messaging-backup" />
<in-vm-acceptor name="in-vm" server-id="2" />
</acceptors>
<broadcast-groups>
<broadcast-group name="bg-group1">
<socket-binding>messaging-group</socket-binding>
<broadcast-period>5000</broadcast-period>
<connector-ref>netty</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="dg-group1">
<socket-binding>messaging-group</socket-binding>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="my-cluster">
<address>jms</address>
<connector-ref>netty</connector-ref>
<discovery-group-ref discovery-group-name="dg-group1" />
</cluster-connection>
</cluster-connections>
<security-settings>
<security-setting match="#">
<permission type="send" roles="guest" />
<permission type="consume" roles="guest" />
<permission type="createNonDurableQueue" roles="guest" />
<permission type="deleteNonDurableQueue" roles="guest" />
</security-setting>
</security-settings>
<address-settings>
<!--default for catch all -->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<redistribution-delay>1000</redistribution-delay>
<max-size-bytes>10485760</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
<message-counter-history-day-limit>10
</message-counter-history-day-limit>
</address-setting>
</address-settings>
<jms-connection-factories>
<connection-factory name="InVmConnectionFactory">
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/ConnectionFactory" />
</entries>
</connection-factory>
<connection-factory name="RemoteConnectionFactory">
<connectors>
<connector-ref connector-name="netty" />
</connectors>
<entries>
<entry name="java:jboss/exported/jms/RemoteConnectionFactory" />
</entries>
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1.0</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
</connection-factory>
<pooled-connection-factory name="hornetq-ra">
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
<transaction mode="xa" />
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/JmsXA" />
</entries>
</pooled-connection-factory>
</jms-connection-factories>
</hornetq-server>
</subsystem>在<socket-binding-group name="standard-sockets"添加messaging-backup socket-binding如下：
<socket-binding name="messaging-backup" port="5446" />node2配置
在<subsystem xmlns="urn:jboss:domain:messaging:中配置hornetq-server
<subsystem xmlns="urn:jboss:domain:messaging:1.3">
<hornetq-server>
<persistence-enabled>true</persistence-enabled>
<cluster-password>clusterPassword1!</cluster-password>
<backup>false</backup>
<allow-failback>true</allow-failback>
<failover-on-shutdown>false</failover-on-shutdown>
<shared-store>false</shared-store>
<journal-type>NIO</journal-type>
<journal-min-files>2</journal-min-files>
<check-for-live-server>true</check-for-live-server>
<backup-group-name>group-2</backup-group-name>
<connectors>
<netty-connector name="netty" socket-binding="messaging" />
<netty-connector name="netty-throughput"
socket-binding="messaging-throughput">
<param key="batch-delay" value="50" />
</netty-connector>
<in-vm-connector name="in-vm" server-id="1" />
</connectors>
<acceptors>
<netty-acceptor name="netty" socket-binding="messaging" />
<netty-acceptor name="netty-throughput"
socket-binding="messaging-throughput">
<param key="batch-delay" value="50" />
<param key="direct-deliver" value="false" />
</netty-acceptor>
<in-vm-acceptor name="in-vm" server-id="1" />
</acceptors>
<broadcast-groups>
<broadcast-group name="bg-group1">
<socket-binding>messaging-group</socket-binding>
<broadcast-period>5000</broadcast-period>
<connector-ref>netty</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="dg-group1">
<socket-binding>messaging-group</socket-binding>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="my-cluster">
<address>jms</address>
<connector-ref>netty</connector-ref>
<discovery-group-ref discovery-group-name="dg-group1" />
</cluster-connection>
</cluster-connections>
<security-settings>
<security-setting match="#">
<permission type="send" roles="guest" />
<permission type="consume" roles="guest" />
<permission type="createNonDurableQueue" roles="guest" />
<permission type="deleteNonDurableQueue" roles="guest" />
</security-setting>
</security-settings>
<address-settings>
<!--default for catch all -->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<redistribution-delay>1000</redistribution-delay>
<max-size-bytes>10485760</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
<message-counter-history-day-limit>10
</message-counter-history-day-limit>
</address-setting>
</address-settings>
<jms-connection-factories>
<connection-factory name="InVmConnectionFactory">
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/ConnectionFactory" />
</entries>
</connection-factory>
<connection-factory name="RemoteConnectionFactory">
<connectors>
<connector-ref connector-name="netty" />
</connectors>
<entries>
<entry name="java:jboss/exported/jms/RemoteConnectionFactory" />
</entries>
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1.0</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
</connection-factory>
<pooled-connection-factory name="hornetq-ra">
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
<transaction mode="xa" />
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/JmsXA" />
</entries>
</pooled-connection-factory>
</jms-connection-factories>
</hornetq-server>
<hornetq-server name="backup-server">
<persistence-enabled>true</persistence-enabled>
<cluster-password>clusterPassword1!</cluster-password>
<backup>true</backup>
<allow-failback>true</allow-failback>
<failover-on-shutdown>false</failover-on-shutdown>
<shared-store>false</shared-store>
<journal-type>NIO</journal-type>
<journal-min-files>2</journal-min-files>
<check-for-live-server>true</check-for-live-server>
<backup-group-name>group-1</backup-group-name>
<paging-directory path="messagingpagingbackupa" />
<bindings-directory path="messagingbindingsbackupa" />
<journal-directory path="messagingjournalbackupa" />
<large-messages-directory path="messaginglargemessagesbackupa" />
<connectors>
<netty-connector name="netty" socket-binding="messaging-backup" />
<in-vm-connector name="in-vm" server-id="2" />
</connectors>
<acceptors>
<netty-acceptor name="netty" socket-binding="messaging-backup" />
<in-vm-acceptor name="in-vm" server-id="2" />
</acceptors>
<broadcast-groups>
<broadcast-group name="bg-group1">
<socket-binding>messaging-group</socket-binding>
<broadcast-period>5000</broadcast-period>
<connector-ref>netty</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="dg-group1">
<socket-binding>messaging-group</socket-binding>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="my-cluster">
<address>jms</address>
<connector-ref>netty</connector-ref>
<discovery-group-ref discovery-group-name="dg-group1" />
</cluster-connection>
</cluster-connections>
<security-settings>
<security-setting match="#">
<permission type="send" roles="guest" />
<permission type="consume" roles="guest" />
<permission type="createNonDurableQueue" roles="guest" />
<permission type="deleteNonDurableQueue" roles="guest" />
</security-setting>
</security-settings>
<address-settings>
<!--default for catch all -->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<redistribution-delay>1000</redistribution-delay>
<max-size-bytes>10485760</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
<message-counter-history-day-limit>10
</message-counter-history-day-limit>
</address-setting>
</address-settings>
<jms-connection-factories>
<connection-factory name="InVmConnectionFactory">
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/ConnectionFactory" />
</entries>
</connection-factory>
<connection-factory name="RemoteConnectionFactory">
<connectors>
<connector-ref connector-name="netty" />
</connectors>
<entries>
<entry name="java:jboss/exported/jms/RemoteConnectionFactory" />
</entries>
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1.0</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
</connection-factory>
<pooled-connection-factory name="hornetq-ra">
<ha>true</ha>
<block-on-acknowledge>true</block-on-acknowledge>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>1</retry-interval-multiplier>
<reconnect-attempts>-1</reconnect-attempts>
<transaction mode="xa" />
<connectors>
<connector-ref connector-name="in-vm" />
</connectors>
<entries>
<entry name="java:/JmsXA" />
</entries>
</pooled-connection-factory>
</jms-connection-factories>
</hornetq-server>
</subsystem>在<socket-binding-group name="standard-sockets"添加messaging-backup socket-binding如下：
<socket-binding name="messaging-backup" port="5446" />注意，如下我实验过程中配置文件standalone-full-ha-node1.xml和standalone-full-ha-node2.xml。
https://github.com/kylinsoong/cluster/blob/master/demo/jms/standalone-full-ha-node1.xml
https://github.com/kylinsoong/cluster/blob/master/demo/jms/standalone-full-ha-node2.xml
创建Application User
我们需要创建Application User来与JBoss进行交换，具体使用JBOSS_HOME/bin/add_user.sh或JBOSS_HOME/bin/add_user.bat创建Application User democlient/passowrd1!。
部署高可用实验应用
本实验需要部署消息队列到JBoss，根据之前系列（JBoss 系列六：JBoss 7/WildFly中配置使用JMS消息队列）部署DistributedQueue到JBoss。我们可以使用拷贝queue-jms.xml文件到deployments目录下面的方式部署DistributedQueue。queue-jms.xml内容如下所示：
<?xml version="1.0" encoding="UTF-8"?>
<messaging-deployment xmlns="urn:jboss:messaging-deployment:1.0">
<hornetq-server>
<jms-destinations>
<jms-queue name="DistributedQueue">
<entry name="queue/DistributedQueue" />
<entry name="java:jboss/exported/jms/queue/DistributedQueue"/>
<durable>true</durable>
</jms-queue>
</jms-destinations>
</hornetq-server>
</messaging-deployment>本实验应用代码位于github（ https://github.com/kylinsoong/cluster/tree/master/demo/jms），使用 系列一方法下载编译代码，会生成部署应用cluster-demo-jms.jar，我们可以使用拷贝cluster-demo-jms.jar文件到deployments目录下面的方式部署应用。
部署完成我们通过如下命令分别启动两个节点：
./standalone.sh -c standalone-full-ha-node1.xml -b 10.66.218.47 -bmanagement=10.66.218.47 -u 239.255.100.100 -Djboss.node.name=node1
./standalone.sh -c standalone-full-ha-node2.xml -b 10.66.218.48 -bmanagement=10.66.218.48 -u 239.255.100.100 -Djboss.node.name=node2高可用和负载均衡过程测试
我们通过如下步骤测试JBoss 7/WildFly消息服务高可用：
1. 发送连续10条消息
运行JMSClient通过node1发送连续的10条消息到集群，我们发现node1处理第一条和第三条消息，输出如下日志
05:20:09,741 INFO [stdout] (Thread-8 (HornetQ-client-global-threads-62067681)) 1: JBoss 7/WildFly HornetQ Messaging High Available
05:20:19,750 INFO [stdout] (Thread-8 (HornetQ-client-global-threads-62067681)) 3: JBoss 7/WildFly HornetQ Messaging High Available与此同时node2处理第二条和第四条消息，输出如下日志
05:20:11,626 INFO [stdout] (Thread-7 (HornetQ-client-global-threads-1878339755)) 2: JBoss 7/WildFly HornetQ Messaging High Available
05:20:21,648 INFO [stdout] (Thread-7 (HornetQ-client-global-threads-1878339755)) 4: JBoss 7/WildFly HornetQ Messaging High Available此处体现了JBoss 7/WildFly消息服务器的负载均衡功能，10条消息分别给node1分发五条，给node2分发五条，node1分发五条为（第1，3，5，7，9），node2分发五条为（第2，4，6，8，10）。
2. 关闭node1，故障转移高可用测试
当node1处理第1条和第3条消息后，还有3条消息（5，7，9）未被处理，这个时候我们强制关闭node1，模拟发生故障，这时5，7，9消息被分发到node2，node2处理5，7，9消息，node2处理日志如下
05:20:31,658 INFO [stdout] (Thread-7 (HornetQ-client-global-threads-1878339755)) 6: JBoss 7/WildFly HornetQ Messaging High Available
05:20:41,679 INFO [stdout] (Thread-7 (HornetQ-client-global-threads-1878339755)) 8: JBoss 7/WildFly HornetQ Messaging High Available
05:20:51,688 INFO [stdout] (Thread-7 (HornetQ-client-global-threads-1878339755)) 10: JBoss 7/WildFly HornetQ Messaging High Available
05:21:28,059 INFO [stdout] (Thread-8 (HornetQ-client-global-threads-1878339755)) 5: JBoss 7/WildFly HornetQ Messaging High Available
05:21:38,068 INFO [stdout] (Thread-8 (HornetQ-client-global-threads-1878339755)) 7: JBoss 7/WildFly HornetQ Messaging High Available
05:21:48,076 INFO [stdout] (Thread-8 (HornetQ-client-global-threads-1878339755)) 9: JBoss 7/WildFly HornetQ Messaging High Available注意，处理第5条和第10条消息之间的间隔为30多秒，这说明故障转移的时间为30多秒。
转载于:https://my.oschina.net/iwuyang/blog/197149