EMQ million-level MQTT message service (TLS Docker Golang)

attach:

Meow's blog: w-blog.cn
EMQ official address: http://emqtt.com/
EMQ Chinese documentation: http://emqtt.com/docs/v2/guide.html

1. TLS certificate verification

In order to ensure security. We often use HTTPS to ensure that requests are not tampered with, and use TLS encryption as MQTT to ensure transmission security

The default TLS encrypted port used by EMQ is port 8883, and the default certificate is in etc/certs in the EMQ directory:

the corresponding configuration file is in emq.conf, you can modify your port and configuration file path

listener.ssl.external = 8883
listener.ssl.external.keyfile = etc/certs/key.pem
listener.ssl.external.certfile = etc/certs/cert.pem

PS: When linking, note that you need to modify tcp:// -> ssl:// from the previous link prefix.

2. Deploy EMQ using Docker

Using Docker to deploy EMQ is very convenient for version upgrades, port management and single-node multi-EMQ, etc., which basically brings no additional overhead for performance.

However, the official does not provide a Docker image that can be used directly, but the provided GIT can be packaged and compressed by yourself. The author made it public because of its use as follows (versions 2.3.5 and above will be maintained):

registry.cn-hangzhou.aliyuncs.com/sunmi-base/sunmi-emq:2.3.5
registry.cn-hangzhou.aliyuncs.com/sunmi-base/sunmi-emq:2.3.6
registry.cn-hangzhou.aliyuncs.com/sunmi-base/sunmi-emq:2.3.7

Of course, the more important point is about configuration. EMQ provides a way to affect configuration files through environment variables. You can refer to the arrangement files of daocker-composer and Kubernetes, and modify the configuration files through environment variables. Authorization and custom authentication statements and the mysql authentication plug-in enabled by default

version: '2'
services:
  emq:
    image: 'registry.cn-hangzhou.aliyuncs.com/sunmi-base/sunmi-emq:2.3.7'
    ports:
      - '31883:1883'
      - '31083:18083'
      - '38883:8883'
    environment:
      - EMQ_MQTT__ALLOW_ANONYMOUS=false
      - EMQ_AUTH__MYSQL__USERNAME=emq
      - EMQ_AUTH__MYSQL__PASSWORD=Emq666
      - EMQ_AUTH__MYSQL__DATABASE=emq
      - "EMQ_AUTH__MYSQL__SERVER=xxxxxx:3306"
      - "EMQ_AUTH__MYSQL__AUTH_QUERY=select password from mqtt_user where username = '%u' limit 1"
      - "EMQ_AUTH__MYSQL__SUPER_QUERY=select is_superuser from mqtt_user where username = '%u' limit 1"
      - "EMQ_AUTH__MYSQL__ACL_QUERY=select allow, ipaddr, username, clientid, access, REPLACE(topic,'$user','%u') from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"
      - "EMQ_LOADED_PLUGINS=emq_auth_mysql,emq_recon,emq_modules,emq_retainer,emq_dashboard"
    restart: always

The following is the K8S orchestration file:

apiVersion: extensions/v1beta1                  # K8S对应的API版本
kind: Deployment                                # 对应的类型
metadata:
  name: emq-deployment
  labels:
    name: emq-deployment
spec:
  replicas: 1                                   # 镜像副本数量
  template:
    metadata:
      labels:                                   # 容器的标签 可和service关联
        app: emq
    spec:
      containers:
        - name: emq                        # 容器名和镜像
          image: registry.cn-hangzhou.aliyuncs.com/sunmi-base/sunmi-emq:2.3.6
          imagePullPolicy: Always
          env:                                  # 环境变量
          - name: EMQ_MQTT__ALLOW_ANONYMOUS
            value: "false"
          - name: EMQ_AUTH__MYSQL__USERNAME
            value: "emq"
          - name: EMQ_AUTH__MYSQL__PASSWORD
            value: "Emq666"
          - name: EMQ_AUTH__MYSQL__DATABASE
            value: "emq"
          - name: EMQ_AUTH__MYSQL__SERVER
            value: "xxxxxx:3306"
          - name: EMQ_AUTH__MYSQL__AUTH_QUERY
            value: "select password from mqtt_user where username = '%u' limit 1"
          - name: EMQ_AUTH__MYSQL__SUPER_QUERY
            value: "select is_superuser from mqtt_user where username = '%u' limit 1"
          - name: EMQ_AUTH__MYSQL__ACL_QUERY
            value: "select allow, ipaddr, username, clientid, access, REPLACE(topic,'$user','%u') from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"
          - name: EMQ_LOADED_PLUGINS
            value: "emq_auth_mysql,emq_recon,emq_modules,emq_retainer,emq_dashboard"
---
apiVersion: v1
kind: Service
metadata:
  name: emq-service                          # 名称
  labels:
    name: emq-service
spec:
  type: NodePort                                  # 开发端口的类型
  selector:                                       # service负载的容器需要有同样的labels
    app: emq
  ports:
  - name: emq-service-1883-30111
    port: 1883                                    # 通过service来访问的端口
    targetPort: 1883                              # 对应容器的端口
    nodePort: 30111
  - name: emq-service-8883-30112
    port: 8883                                    # 通过service来访问的端口
    targetPort: 8883                              # 对应容器的端口
    nodePort: 30112
  - name: emq-service-18083-30113
    port: 18083                                    # 通过service来访问的端口
    targetPort: 18083                              # 对应容器的端口
    nodePort: 30113

PS: TCP link optimization needs to be done on the host

3. Goalng client

The author here uses the gobot library based on the library under the https://github.com/eclipse/paho.mqtt.golang paho system. The examples are as follows:

package main

import (
  "gobot.io/x/gobot"
  "gobot.io/x/gobot/platforms/mqtt"
  "fmt"
  "time"
)

func main() {
  mqttAdaptor := mqtt.NewAdaptor("tcp://0.0.0.0:1883", "pinger")

  work := func() {
    mqttAdaptor.On("hello", func(msg mqtt.Message) {
      fmt.Println(msg)
    })
    mqttAdaptor.On("hola", func(msg mqtt.Message) {
      fmt.Println(msg)
    })
    data := []byte("o")
    gobot.Every(1*time.Second, func() {
      mqttAdaptor.Publish("hello", data)
    })
    gobot.Every(5*time.Second, func() {
      mqttAdaptor.Publish("hola", data)
    })
  }

  robot := gobot.NewRobot("mqttBot",
    []gobot.Connection{mqttAdaptor},
    work,
  )

  robot.Start()
}

Use mqttAdaptor.Publish to send messages and mqttAdaptor.On to subscribe to messages. If there is user authentication, you can use the following methods:

    mqttAdaptor = mqtt.NewAdaptorWithAuth(
        "EMQ.host",
        "EMQ.clientID",
        "EMQ.userName",
        "EMQ.passWordActive",
    )

You can also configure the reconnection rule to automatically reconnect when disconnected (not enabled by default, it is strongly recommended to enable it as a server)

mqttAdaptor.SetAutoReconnect(true)

And message cleanup mechanism (default disconnect cleanup message)

mqttAdaptor.SetCleanSession(false)

You can also specify to connect using a TLS certificate

mqttAdaptor.SetUseSSL(true)
# 下面可以指定证书(如果EMQ使用了标准的CA证书下面就不用配置了)
mqttAdaptor.SetClientKey(`/client/client-key.pem`)
mqttAdaptor.SetClientCert(`/client/client-cert.pem`)

4. Summary

There are still many details to pay attention to in the use of EMQ and MQTT, and you can go further by paying attention to details

Note: The author has limited ability and I hope that everyone can point out the wrong places, and I hope to communicate more!