[RabbitMQ] Basic concepts of MQ, introduction and installation of RabbitMQ

The basic concept of MQ

MQ overview

The full name of MQ is Message Queue (message queue), which is a container for storing messages during message transmission. It is mostly used for communication between distributed systems.

Generally, our distributed systems communicate in two ways:

• The first method:
  System A accesses system B directly through remote calls

• The second type:
  

  The sender is called the producer and the receiver is called the consumer

  System A relies on a third party, and the third party sends data to system B to achieve indirect communication. Message queues belong to this method

Advantages and Disadvantages of MQ

MQ has three advantages:

• application decoupling
• Asynchronous speed up
• Asynchronous speed up

Application Decoupling Aspects

When the user clicks the button to place an order, he accesses the order system, and then the order system needs to access the inventory, payment, and logistics systems. Here he has two ways to access. One is a direct remote call, so that the order system and the three systems on the right will be coupled together, which will cause some problems.

For example, if our inventory system is down, then our order system will also fail to work:

Assuming that when using the order system now, there is another X system called, then we have to modify the code of the order system, which is obviously very troublesome when the demand increases and changes frequently.

The more coupled the system, the lower the fault tolerance and the lower the maintainability.

And if we use MQ, when the user places an order to access the order system, the order system only needs to send a message to MQ. At this time, it can give the user feedback that the order is successful. The system on the right only needs to take out the order message data from MQ, and then process it in its own system. And if the inventory system is down at this time, the order system will have no effect.

Asynchronous speed up

Time-consuming for an order placing operation: 20 + 300 + 300 + 300 = 920ms After
the user clicks the order button, he needs to wait 920ms to get the order response, which is too slow!

After the user clicks the order button, he only needs to wait 25ms to get the order response (20 + 5 = 25ms).
Improve user experience and system throughput (the number of requests processed per unit time).

Shaving peaks and filling valleys

After using MQ, limit the speed of consuming messages to 1000. In this way, the data generated during the peak period will inevitably be backlogged in MQ, and the peak will be "cut" off, but because of the backlog of messages, a period of time after the peak period Within a certain period of time, the speed of consuming news will still be maintained at 1000 until the backlog of news is consumed, which is called "filling the valley".

After using MQ, the system stability can be improved.

Disadvantages of MQ:

• Reduced system availability
  The more external dependencies the system introduces, the worse the system stability will be. Once MQ goes down, it will affect the business. How to ensure the high availability of MQ?
• Increased system complexity
  The addition of MQ has greatly increased the complexity of the system. In the past, there were synchronous remote calls between systems, but now asynchronous calls are made through MQ. How to ensure that messages are not consumed repeatedly? How to deal with message loss? So to ensure the order of message delivery?
• Consistency problem
  System A finishes processing the business and sends message data to systems B, C, and D through MQ. If systems B and C process successfully, system D fails. How to ensure the consistency of message data processing?

Common MQ products

At present, there are many MQ products in the industry, such as RabbitMQ, RocketMQ, ActiveMQ, Kafka, ZeroMQ, MetaMq, etc., and there are also cases of directly using Redis as a message queue. Consider your own needs and MQ product features comprehensively.

Introduction to RabbitMQ

AMQP, that is Advanced Message Queuing Protocol（高级消息队列协议）, is a network protocol, an open standard of the application layer protocol, designed for message-oriented middleware. The client and message middleware based on this protocol can transmit messages, and it is not limited by different client/middleware products, different development languages ​​and other conditions. In 2006, the AMQP specification was released. Analogy HTTP.

The process of AMQP is: the producer publishes the message to the switch, and the switch sends the message to different queues for storage through routing rules, and then the consumer monitors and takes the corresponding message from the queue for consumption. ,

In 2007, RabbitMQ 1.0 developed by Rabbit Technology Company based on the AMQP standard was released. RabbitMQ is developed in Erlang language. The Erlang language was designed by Ericson, a language specially developed for the development of highly concurrent and distributed systems, and is widely used in the telecommunications field.

The basic structure of

RabbitMQ is as follows: Related concepts in RabbitMQ:

• Broker: The application for receiving and distributing messages, RabbitMQ Server is Message Broker
• Virtual host: Designed for multi-tenancy and security factors, the basic components of AMQP are divided into a virtual group, similar to the namespace concept in the network. When multiple different users use the services provided by the same RabbitMQ server, multiple vhosts can be divided, and each user creates an exchange/queue in his own vhost, etc.
• Connection: TCP connection between publisher/consumer and broker
• Channel: If a Connection is established every time RabbitMQ is accessed, the overhead of establishing a TCP Connection when the message volume is large will be huge and the efficiency will be low. Channel is a logical connection established inside the connection. If the application supports multi-threading, usually each thread creates a separate channel for communication. AMQP method includes the channel id to help the client and message broker identify the channel, so the channels are completely isolated of. As a lightweight Connection, Channel greatly reduces the overhead of the operating system to establish a TCP connection
• Exchange: The message arrives at the first stop of the broker, according to the distribution rules, matches the routing key in the query table, and distributes the message to the queue. Commonly used types are: direct (point-to-point), topic (publish-subscribe) and fanout (multicast)
• Queue: The message is finally sent here and waits for the consumer to pick it up
• Binding: Virtual connection between exchange and queue, the binding can contain routing key. Binding information is stored in the query table in the exchange for the basis of message distribution

RabbitMQ provides 6 working modes:

• simple mode
• work queues
• Publish/Subscribe publish and subscribe mode
• Routing routing mode
• Topics theme mode
• RPC remote call mode (remote call, not too MQ; no introduction for now).

Official website corresponding mode introduction: https://www.rabbitmq.com/getstarted.html

Finally, let's talk about JMS:

JMS Java 消息服务（JavaMessage Service）应用程序接口is an API for message-oriented middleware in the Java platform

• JMS is one of the JavaEE specifications, analogous to JDBC
• Many message middleware implement the JMS specification, for example: ActiveMQ.
• RabbitMQ does not officially provide JMS implementation packages, but the open source community has

RabbitMQ installation

1. Install dependent environment

Online installation depends on the environment:

yum install build-essential openssl openssl-devel unixODBC unixODBC-devel make gcc gcc-c++ kernel-devel m4 ncurses-devel tk tc xz

2. Install Erlang

upload

erlang-18.3-1.el7.centos.x86_64.rpm
socat-1.7.3.2-5.el7.lux.x86_64.rpm
rabbitmq-server-3.6.5-1.noarch.rpm

# 安装
rpm -ivh erlang-18.3-1.el7.centos.x86_64.rpm

If the following error occurs

It means that the gblic version is too low. We can view the gblic version of the current machine

strings /lib64/libc.so.6 | grep GLIBC

The current highest version is 2.12, which requires 2.15. So you need to upgrade glibc

• Use yum to update and install dependencies

  sudo yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make -y
  

• download rpm package

  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/glibc-utils-2.17-55.el6.x86_64.rpm &
  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/glibc-static-2.17-55.el6.x86_64.rpm &
  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/glibc-2.17-55.el6.x86_64.rpm &
  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/glibc-common-2.17-55.el6.x86_64.rpm &
  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/glibc-devel-2.17-55.el6.x86_64.rpm &
  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/glibc-headers-2.17-55.el6.x86_64.rpm &
  wget http://copr-be.cloud.fedoraproject.org/results/mosquito/myrepo-el6/epel-6-x86_64/glibc-2.17-55.fc20/nscd-2.17-55.el6.x86_64.rpm &
  

• Install the rpm package

  sudo rpm -Uvh *-2.17-55.el6.x86_64.rpm --force --nodeps
  

• After the installation is complete, check the glibc version and find that the glibc version has reached 2.17

  strings /lib64/libc.so.6 | grep GLIBC
  

3. Install RabbitMQ

# 安装
rpm -ivh socat-1.7.3.2-5.el7.lux.x86_64.rpm

# 安装
rpm -ivh rabbitmq-server-3.6.5-1.noarch.rpm

4. Open the management interface and configure

# 开启管理界面
rabbitmq-plugins enable rabbitmq_management
# 修改默认配置信息
vim /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.5/ebin/rabbit.app 
# 比如修改密码、配置等等，例如：loopback_users 中的 <<"guest">>,只保留guest

5. start

service rabbitmq-server start # 启动服务
service rabbitmq-server stop # 停止服务
service rabbitmq-server restart # 重启服务

• set configuration file

cd /usr/share/doc/rabbitmq-server-3.6.5/

cp rabbitmq.config.example /etc/rabbitmq/rabbitmq.config

6. Configure virtual hosts and users

user role

After RabbitMQ is installed, it can be accessed http://ip地址:15672; it comes with guest/guest user name and password; if you need to create a custom user; then you can also log in to the management interface, as follows:

Role description :

1. Super administrator (administrator)

You can log in to the management console, view all the information, and perform operations on users and policies.

2. Monitoring

You can log in to the management console, and you can view the relevant information of the rabbitmq node (number of processes, memory usage, disk usage, etc.)

3. Policymaker

You can log in to the management console and manage policies at the same time. But you cannot view the relevant information of the node (the part marked by the red box in the figure above).

4. Ordinary managers (management)

You can only log in to the management console, but you cannot see node information or manage policies.

5. Others

Unable to log in to the management console, usually ordinary producers and consumers.

Virtual Hosts configuration

Like mysql has the concept of a database and can specify user permissions for operations such as libraries and tables. RabbitMQ also has similar permission management; in RabbitMQ, virtual message server Virtual Host can be used, and each Virtual Host is equivalent to a relatively independent RabbitMQ server, and each Virtual Host is isolated from each other. Exchange, queue, and message cannot communicate with each other. Equivalent to mysql's db. Virtual Name generally starts with /.

Create Virtual Hosts

Set Virtual Hosts permissions