Introduction and basic concepts of MQTT protocol

1. MQTT protocol

• Official website: https://mqtt.org/
• Official document: https://www.emqx.com/zh/mqtt-guide

1 Introduction

MQTT (MQ Telemetry Transport) is a lightweight message transmission protocol based on the publish/subscribe model. It is specially designed for IoT applications in low-bandwidth and unstable network environments. It can provide real-time and reliable services for networked devices with very little code. message service.

MQTT is a machine-to-machine (M2M)/Internet of Things (IoT) connectivity protocol.

2. Features of MQTT:

• Open message protocol, simple and easy to implement
• Message QoS support, reliable transmission guarantee (equipment network environment is complex)
• Publish and subscribe mode, one-to-many message publishing
• Based on TCP/IP network connection, it provides orderly, lossless, two-way connection.
• 1-byte fixed header, 2-byte heartbeat message, minimize transmission overhead and protocol exchange, effectively reduce network traffic (lightweight and save bandwidth)
• Data-independent (don't care about Payload data format)
• Persistent session awareness (know whether the device is online at all times)

3. Application scenarios:

The MQTT protocol is widely used in the Internet of Things, mobile Internet, intelligent hardware, Internet of Vehicles, smart cities, telemedicine, electric power, oil and energy and other fields.

• IoT M2M communication, IoT big data collection
• Android message push, WEB message push
• Mobile instant messaging, such as Facebook Messenger
• Smart hardware, smart furniture, smart appliances
• Vehicle networking communication, electric station pile collection
• smart city, telemedicine, distance education
• Power, oil and energy industry markets

4. MQTT 5.0 and 3.1.1

Four years after MQTT 3.1.1 was released and became an OASIS standard, MQTT 5.0 is officially released. This is a major improvement and upgrade. Its purpose is not only to meet the needs of the industry at this stage, but also to make sufficient preparations for the future development and changes of the industry.

On the basis of version 3.1.1, MQTT 5.0 adds features such as session/message delay, reason code, topic alias, user attributes, shared subscription, etc. that are more in line with the requirements of modern IoT applications, and improves the performance, stability and reliability of large systems. scalability. Currently, MQTT 5.0 has become the preferred protocol for most IoT companies, and we recommend that developers who are new to MQTT use this version directly.

MQTT v5.0 is now an official OASIS standard:

OASIS has released the official MQTT v5.0 standard - a huge leap forward in improvements and capabilities for the messaging protocol already in use for the Internet of Things (IoT). Based on the earlier v3.1.1 standard, it features significant updates while minimizing incompatibilities with existing versions.

Highlights of the new release include:

• Better error reporting - In particular, reason codes (PUBACK/PUBREC) have been added to posted responses. MQTT originated in use cases such as sensors along oil pipelines - if their message publications fail to transmit, the sensors take no action. However, the use cases for MQTT are much wider now, and an app on a phone might want to warn the user if data is not successfully transmitted. Return codes now appear on all acknowledgments (along with an optional reason string containing human-readable error diagnostics).
• Shared Subscription - If the rate of messages on a subscription is high, you can use a shared subscription to load balance messages among multiple receiving clients.
• Message Attributes - Metadata in message headers. These are used to implement other functions in this list, but also allow user-defined properties, such as to assist message encryption by telling the receiver which key to use to decrypt the message content
• Message Expiration - You can choose to discard a message if it cannot be delivered within a user-defined period of time.
• Session Expiration - If a client does not connect within a user-defined period of time, state (for example, subscriptions and buffered messages) can be discarded without cleanup.
• Topic Aliases - Allows the topic string in a message to be replaced with a single number, reducing the number of bytes that need to be transferred when the publisher reuses the same topic.
• Will Delay - Allows publishing of messages if the client disconnects for longer than the user-defined period of time. Allow notifications about critical client application outages without being overwhelmed by false positives.
• Allowed Capabilities Discovery - At the start of a connection, a limit on the maximum packet size and number of (QoS > 0) messages that can be transmitted to inform the client of what is allowed to be performed. A complete list of new features can be found in Appendix C of the standard.

5. Why is MQTT the best protocol for IoT?

Under such a large-scale Internet of Things demand, massive device access and device management have brought huge challenges to network bandwidth, communication protocols, and platform service architecture. For the Internet of Things protocol, several key problems in the communication of IoT devices must be solved in a targeted manner: complex and unreliable network environment, small memory and flash memory capacity, and limited processor capacity.

The MQTT protocol was created to deal with the above problems. After years of development, it has become the preferred protocol for the IoT industry due to its lightweight, efficient, reliable message delivery, massive connection support, and secure two-way communication.

So, with the rapid development of the Internet of Things (IoT), the MQTT protocol is widely used by many companies and developers.

6、MQTT vs HTTP

• The smallest message of MQTT is only 2 bytes, which occupies less network overhead than HTTP.
• Both MQTT and HTTP can use TCP connection and realize stable and reliable network connection.
• MQTT is based on a publish-subscribe model, and HTTP is based on request-response, so MQTT supports duplex communication.
• MQTT can push messages in real time, but HTTP requires polling for data updates.
• MQTT is stateful, but HTTP is stateless.
• MQTT can recover from abnormal disconnection, HTTP cannot achieve this goal.

2. How MQTT works

To understand how MQTT works, you first need to master the following concepts:

• publish-subscribe model,
• MQTT Broker、
• MQTT client,
• theme,
• QoS。

1. Publish-subscribe model

MQTT is based on the publish-subscribe model, which decouples the sender (publisher) and receiver (subscriber) of the message, and introduces the role of an intermediate agent to complete the routing and distribution of messages.

Publishers and subscribers don't need to know each other's existence, the only connection between them is a consistent agreement on the message, such as what topic the message will use, what fields the message will contain, and so on. This makes MQTT communication more flexible, because we can dynamically increase or decrease subscribers and publishers at any time.

By publishing and subscribing, we can easily implement message broadcasting, multicasting and unicasting.

For more information on the MQTT publish-subscribe model, we can refer to the introduction to the publish-subscribe model in the MQTT series of blogs: https://www.emqx.com/zh/blog/mqtt-5-introduction-to-publish-subscribe-model

2、MQTT Broker

Acts as an intermediary between publishing clients and subscribing clients, forwarding all received messages to matching subscribing clients. So sometimes we also directly refer to the server as Broker.

MQTT Broker is a key component responsible for processing client requests, including operations such as connection establishment, disconnection, subscription and unsubscription, and is also responsible for message forwarding. An efficient and powerful MQTT Broker can easily handle massive connections and million-level message throughput, thereby helping IoT service providers focus on business development and quickly build reliable MQTT applications.

3. MQTT client

A device or application that connects to an MQTT server using the MQTT protocol. It can be either a publisher, a subscriber, or both.

Any application or device running an MQTT client library is an MQTT client. For example, instant messaging applications using MQTT are clients, various sensors that use MQTT to report data are clients, and various MQTT testing tools are also clients, such as MQTTX client tools.

4. Theme

The MQTT protocol forwards messages according to topics. Topics are used to identify and distinguish different messages, which is the basis of MQTT message routing.

Publishers can specify the subject of the message when publishing, and subscribers can choose to subscribe to the topics they are interested in to receive related messages.

Topics are hierarchically differentiated by /, similar to URL paths. For example:

chat/room/1

sensor/10/temperature

sensor/+/temperature

MQTT 主题支持以下两种通配符：+ 和 #。

• +: Indicates a single-level wildcard, for example, a/+ matches a/x or a/y.
• #: Indicates multi-level wildcards, for example, a/# matches a/x, a/b/c/d.

注意：Wildcard topics can only be used for subscription, not for publishing.

For more details about MQTT topics, please refer to the article Understanding MQTT Topics and Wildcards Through Cases: https://www.emqx.com/zh/blog/advanced-features-of-mqtt-topics

5、QoS

MQTT defines three QoS levels to ensure message reliability in different network environments. Each message can independently set its own QoS when published.

• QoS 0: Messages are delivered at most once. If the current client is unavailable, it will lose this message (the message may be lost);
• QoS 1: The message is delivered at least once, and the message can be guaranteed to arrive, but may be repeated;
• QoS 2: The message is only delivered once, and the message is guaranteed to arrive and will not be repeated.

The larger the QoS, the higher the complexity of message transmission. We need to choose the appropriate QoS according to the actual scenario.

For more details about MQTT QoS, please refer to the article Introduction to MQTT QoS 0, 1, 2: https://www.emqx.com/zh/blog/introduction-to-mqtt-qos

3. MQTT workflow

Now that we understand the basic components of MQTT, let's take a look at its general workflow:

1. The client uses the TCP/IP protocol to establish a connection with the Broker, and can choose to use TLS/SSL encryption to achieve secure communication. The client provides authentication information and specifies the session type (Clean Session or Persistent Session).
2. Clients can either publish messages to specific topics, or subscribe to topics to receive messages. When a client publishes a message, it will send the message to the MQTT Broker; and when the client subscribes to the message, it will receive the message related to the subscribed topic.
3. MQTT Broker receives published messages and forwards these messages to clients that have subscribed to corresponding topics. It ensures reliable delivery of messages based on QoS class and stores messages for disconnected clients based on session type.

See the official documentation for more information.

– If you are hungry for knowledge, be humble if you are foolish.