Introduction to MQTT Mosquitto Bridge and Cluster Environment Construction
If the original article is reproduced, please indicate the source (http://blog.csdn.net/hui6075/).
Contents:
Introduction to MQTT Protocol
Mosquitto Bridge Mode
Mosquitto Cluster Mode
Introduction to MQTT protocol
MQTT is a set of application-layer instant communication protocols defined by IBM for environments such as the Internet of Things. It provides subscription/publishing methods through message middleware and provides decoupling for communication between different devices through "topics".
Similar protocols include XMPP, COAP, etc., but the MQTT protocol has relatively low requirements on the processing capability of the device and the network environment due to its few signaling types, less control information, and high information carrying rate. Just imagine, let a 200MHz, 64K memory embedded module parse HTTP/JASON/XML strings, maybe the next piece of data will arrive before the packet is parsed, haha. Of course, there are also many people who use MQTT as the Android push protocol, which can also greatly improve the user experience. For example, the reserved messages in the MQTT protocol allow the phone to receive some messages that have been released by the system or other devices before the phone is turned on.
The message middleware is usually called broker. There are many brokers that support MQTT, such as Mosquitto based on C language, EMQ based on Erlang , RabbitMQ + plug-in, ActiveMQ based on Java , and even Mosca based on Node.js. I have tried Mosquitto and RabbitMQ myself, Mosquitto fully implements all the content of the MQTT protocol, while RabbitMQ's support for MQTT is not perfect, such as not supporting QoS=2 messages and reserved messages. In addition, the EMQ developed by the Chinese is also a very good MQTT broker, which supports clustering and message persistence.
Mosquitto Bridge Mode
The broker used in my production environment is Mosquitto . To be honest, Mosquitto was originally designed to run on an embedded platform. The IO model and timer used are relatively simple. Usually, optimization is required to support high-traffic and high-concurrency business scenarios, and the traffic In the case of too large size, Mosquitto also supports connecting multiple brokers through bridge. Usually, one broker is selected as the bridge node, and the bridge option is added in its configuration file (mosquitto.conf):
connection bridge1
address 192.168.1.102:1883
topic # both 0
connection bridge2
address 192.168.1.103:1883
topic /sensor/temperature in 1
Among them, bridge1/bridge2 is the name for other brokers, the address is the IP and port of other brokers, topic specifies the message topic that is allowed to be transmitted between the local broker and the peer broker, and when it is matched with #, all topic messages will be delivered. both/in/out refers to the allowed message direction, and in refers to the forwarding direction of this topic message can only be from the peer broker to the local broker, not from the local broker to the peer broker, so if a client connects to the local broker to subscribe For the topic /sensor/temperature, the local client can also receive the /sensor/temperature topic message published to the peer broker, and vice versa.
From the above introduction, we can know that the topic configuration in bridge mode is very inflexible, especially in some scenarios, the message publishing client and the subscribing client communicate through MAC address, UUID, etc. as topic names, and it is impossible to manually publish/publish/ Subscribe to the device to configure.
Mosquitto cluster mode
Mosquitto also has a cluster implementation based on secondary development .
The clustering method is dynamic subscription. Each broker is both a publisher and a subscriber. For a topic subscribed by a client connected to the local broker, the local broker will subscribe to other brokers in the cluster, so there is no need to dynamically configure topic filters in advance. In addition, all brokers in the cluster are in the same position, so there is no single point of failure. It is also very simple to configure, in each machine's mosquitto.conf:
node_name node1
node_address 192.168.1.101:1883
node_name node2
node_address 192.168.1.102:1883
node_name node3
node_address 192.168.1.103:1883
When compiling mosquitto, comment out WITH_BRIDGE in config.mk, uncomment WITH_CLUSTER:=yes, then make && make install install mosquitto, and start mosquitto on all machines.
function test:
Subscribe to the topic "clustermsg" on host 1, publish a message with the topic "clustermsg" on host 2, and host 1 can receive this message.
benchmark：
The overall throughput rate of the cluster is tested under the conditions of 1 broker, 2 brokers, 3 brokers, 4 brokers, and 9 brokers respectively. The overall throughput rate gradually increases from 1 broker to 3 brokers, and the growth coefficient is about Around 0.8, when the number of brokers exceeds 3, the overall throughput stops increasing due to the bottleneck of the stress measurement tool.
Test parameter description: The message length is fixed at 1000 bytes, n10k means that each client sends 10k messages, c100 means a total of 100 clients, and QoS is selected as 2.
The principle of the cluster and a more detailed performance test report can be found in the readme.md of the mosquitto cluster@github project.