[RabbitMQ] Introduction and installation of RabbitMQ [Original]

Introduction and installation of RabbbitMQ

1-1 Overview of Message Queuing

1. Queue

Queue is a common data structure, and its biggest feature is First In First Out. As the most basic data structure, queues are widely used, such as the well-known Redis basic data type List, which The underlying data structure is the queue.

2. Message queue

Message (Message) refers to the data transmitted between applications. Messages can be very simple, like just text strings, JSON, or complex, like embedded objects.

A message queue ( Messaeg Queue) is a distributed message container that uses a queue (Queue) as the underlying storage data structure and can be used to solve the communication between different processes and applications, also known as message middleware.

In essence, the message queue is a middleware of a queue structure, that is to say, after the message is put into this middleware, it can be returned directly, and it does not need to be processed immediately by the system, and another program will read the data and sequentially Process successively.

Currently used message queues are ActiveMQ, RabbitMQ, Kafka, RocketMQetc.

3. Application scenarios

There are five commonly used scenarios for message queues:

Newsletter
Asynchronous processing
Service decoupling
Traffic clipping

A. Messaging

The main function of message queue is to send and receive messages, and it has an efficient communication mechanism inside, so it is very suitable for message communication.

A peer-to-peer chat system can be developed based on message queues, or a broadcast system can be developed for broadcasting messages to a large number of recipients.

B. Asynchronous processing

Generally written programs are executed sequentially (synchronous execution), such as a user registration function, the execution sequence is as follows:

Write user registration data
Send registration email
Send SMS notification of successful registration
update statistics

According to the above execution sequence, all executions must be completed before returning to success, but in fact, after the first step is successfully executed, other steps can be executed asynchronously, and the subsequent logic can be sent to the message queue, and then executed asynchronously by other programs. As follows:

Using the message queue for asynchronous processing can return results faster, speed up the response speed of the server, and improve the performance of the server.

C. Service Decoupling

In the system, communication between applications is very common. Generally, applications are called directly. For example, application A calls the interface of application B. At this time, the relationship between applications is strongly coupled.

If App B is unavailable, App A will also be affected.

A message queue is introduced between application A and application B for service decoupling. If application B hangs, it will not affect the use of application A.

After using the message queue, the producer does not care who the consumer is, and the consumer also does not care who the sender is. This is decoupling. The message queue is often used to resolve the call dependencies between services.

D. Traffic clipping

For a high-concurrency system, at peak access times, sudden traffic flows to the application system like a flood, especially for some high-concurrency write operations, which can cause the database server to be paralyzed at any time and cannot continue to provide services.

The introduction of message queues can reduce the impact of burst traffic on the application system. The message queue is like a "reservoir", which intercepts the upstream flood and reduces the flood peak flow entering the downstream river, so as to achieve the purpose of reducing flood disasters.

The most common example of this is the seckill system. Generally, the instant traffic of seckill activities is very high. If all traffic flows to the seckill system, it will overwhelm the seckill system. By introducing a message queue, the burst traffic can be effectively buffered, and the “peak cut and valley fill” can be achieved. " effect.

1-2. RabbitMQ overview

1 Overview

RabbitMQIt is Erlangan implemented message queue server developed in language AMQP协议. Compared with other message queues of the same type, the biggest feature is that it guarantees a considerable single-machine throughput and at the same time, the delay is very good.

RabbitMQVarious clients are supported, such as: Python, Ruby, .NET, Java, JMS, , C, PHP, ActionScript, XMPP, STOMPetc.

RabbitMQOriginally originated from entry systems for store-and-forward messages in distributed systems.

Features of RabbitMQ:

Reliability: RabbitMQ provides a variety of techniques to trade off performance and reliability. These techniques include persistence mechanisms, delivery confirmation, publisher confirmation, and high availability mechanisms
Flexible routing: messages are routed through switches before reaching the queue. RabbitMQ provides multiple built-in switch types for typical routing logic
Clustering: Multiple RabbitMQ servers in the same LAN can be aggregated together and used as an independent logical broker
Federation: For servers, it requires more loose and unreliable links than clusters. RabbitMQ provides a federation model for this
High Availability Queues: Queues can be mirrored across multiple machines in the same cluster to ensure messages remain safe if some of the hardware fails
Multiprotocol: Messaging that supports multiple message protocols
Wide range of clients: supports a wide variety of clients
Visual management tool: RabbitMQ comes with an easy-to-use visual management tool, which can help you monitor every link of the message broker
Tracing: RabbitMQ provides support for tracing abnormal behavior, which can find the problem
Plug-in system: RabbitMQ comes with a variety of plug-ins to extend, and even write plug-ins to use

2. AMQP protocol

RabbitMQ is a message queue server that implements the AMQP protocol. Here we will introduce the following AMQP.

AMQP, the Advanced Message Queuing Protocol, is an open standard for application layer protocols. It is designed for message-oriented middleware. It supports client applications and messaging middleware brokers that meet the requirements. ) to communicate with each other.

Message brokers (message brokers) receive messages from publishers (publishers), also known as producers (producers), and send the received messages to consumers (consumers) that process messages according to established routing rules.

Since AMQP is a network protocol, publishers, consumers, and message brokers in this process can exist on different devices.

The working process of AMQP 0-9-1 is as follows:

Messages are sent by publishers to exchanges, which are often likened to post offices or mailboxes. The switch then distributes the received message to the bound queue according to the routing rules. Finally, the AMQP broker will deliver the message to the consumer who subscribes to this queue, or the consumer can obtain it on demand.

The publisher (publisher) can specify various message attributes (message meta-data) to the message when publishing the message. Some properties may be used by message brokers, while others are completely opaque and can only be used by the application receiving the message.

From a security point of view, the network is unreliable, and the application receiving the message may fail while processing the message.

For this reason, the AMQP module includes a concept of message acknowledgements: when a message is delivered from the queue to the consumer (consumer), the consumer will notify the message broker (broker), which can be automatic or Can be executed by the developer of the application that handles the message. When "message acknowledgment" is enabled, the message broker will not completely remove the message from the queue until it receives an acknowledgement from the consumer.

In some cases, such as when a message cannot be successfully routed, the message may be returned to the publisher and discarded. Alternatively, if the message broker performs a deferred operation, the message is put into a so-called dead letter queue. At this point, the message publisher can choose certain parameters to handle these special cases.

Queues, exchanges and bindings are collectively referred to as AMQP entities.

3. Related concepts

RabbitMQIt has its own set of core concepts, and the understanding of these concepts is very important. Only by understanding these core concepts can it be possible to establish RabbitMQa comprehensive understanding.

A. Producer, Producer

The producer connects to the RabbitMQ server, and then sends the message to the queue of the RabbitMQ server, and is the sender of the message.

A message can generally contain 2 parts:

Message body: The message body can also be called payload. In practical applications, the message body is generally a data with a business logic structure, such as a JSON string.
Label

B. Consumer, Consumer

The consumer connects to the RabbitMQ server and subscribes to the queue, which is the receiver of the message.

When a consumer consumes a message, it only consumes the payload of the message.

C. Broker

Broker is the service node of message middleware.

For RabbitMQ, a RabbitMQ Broker can simply be regarded as a RabbitMQ service node, or a RabbitMQ service instance.

D. Queue, queue

Queue, Queue, is an object used to store messages inside RabbitMQ, and it is a structure that actually stores messages.

On the production side, the producer's message is finally sent to the specified queue, and the consumer also obtains the message by subscribing to a certain queue.

Messages in RabbitMQ can only be stored in queues, and the characteristics of queues are first-in, first-out.

Messages received by the queue must be forwarded by Exchange.

The message data in one queue may come from multiple exchanges, and the message data in one exchange may also be pushed to multiple queues, and the relationship between them is many-to-many.

E. Exchange

Exchange, a message exchange, is used to receive messages from producers and forward messages to bound queues according to routing keys.

The message sent by the producer, each message will have a routing key (RoutingKey), which is a simple string, and the message is first forwarded to the queue through Exchange according to the binding rules.

The message data in one queue may come from multiple exchanges, and the message data in one exchange may also be pushed to multiple queues, and the relationship between them is many-to-many.

The exchange takes a message and routes it to one or more queues. Which routing algorithm it uses is determined by the switch type and rules called bindings.

There are 4 types of Exchange Types:

fanout: fan-type switch, this type does not handle routing keys (RoutingKey), similar to broadcasting, all messages sent to the switch will be sent to all queues bound to the switch. Sending messages of this type is the fastest.
direct: Directly connected to the switch, the mode handles the routing key (RoutingKey), and a queue with an exact routing key matching is required to receive messages from the switch. This mode is used the most.
topic: topic switch, which matches routing keys with a pattern.
headers: header switch, generally rarely used.

F. Binding

BindingIt is an operation whose function is to establish a rule for Exchangeforwarding messages from to to. When binding with , you need to specify one . The Binding operation is generally used for the routing work mode and the topic work mode.QueueExchangeQueueBindingKeyRabbitMQ

G. vhosts

Vhosts, virtual hosts, Virutal hostalso called virtual hosts, Virtual Hosthave a set of different ands under one , and Exchnagedifferent Queueands do not affect each other.Virtual hostExchnageQueue

Application isolation and permission division Virtual hostare the smallest granular permission unit division in RabbitMQ.

If you want to make an analogy, you can compare it Virtual hostto MySQLthe database in China. Usually, when we use MySQLit, we specify different databases for different projects. Similarly, when we use RabbitMQit, we can specify different databases for different applications Virtual host.

In RabbitMQ, users can only control permissions at the granularity of virtual hosts. Therefore, if group A needs to be disabled from accessing group B's switches/queues/bindings, a virtual host must be created for A and B each.

Every RabbitMQ server has a default virtual host "/".

There can be multiple vhosts on a RabbitMQ server, and the user and permission settings are attached to the vhosts.

H. Connection

ConnectionIs RabbitMQone of the internal objects, the concept of partial physical, is a physical connection, used to manage each to RabbitMQthe TCPnetwork connection.

Producers, consumers and brokers are connected through Connection.

I. Channel

Channel, Channel is RabbitMQthe most important interface to deal with. It is a logical concept. Multiple Channels can be created in a Connection.

Most of the operations related to RabbitMQ are done in Channelthis interface, including defining Queue, defining Exchange, binding , Queueand Exchangepublishing messages.

You can refer to the figure:

2-1. Installation

Only the installation of Windows and installation under CentOS7 are introduced here. The official website also provides installation under Ubuntu, Mac and even docker.

The code of the RabbitMQ server is written in the erlang language, so the erlang language needs to be installed first.

Note: The version of RabbitMQ depends on the version of Erlang. For details, please refer to: https://www.rabbitmq.com/which-erlang.html

1. Windows installation

A. Install erlang

First download erlang through the official website: official website download address

Download the exe file and install it

After installation, you need to set the environment variables:

My Computer - Right-click Properties - Advanced System Settings - Environment Variables - User Variables/System Variables Create a new variable:

The variable name is: ERLANG_HOME, and the variable value is the installation directory of erlang

It also needs to be added to Path: %ERLANG_HOME%\bin

Then open the command line and enter erl. If the version information of erlang is displayed, the installation is successful:

B. Install RabbitMQ

Download RabbitMQ through the official website: official website download address

Double click to install

C. Install Web Management [optional]

This step is not necessary. RabbitMQ also provides a web management tool. As a plug-in of RabbitMQ, the web management tool is equivalent to a background management page, which is convenient for viewing in the browser.

Go to the sbin directory, open the command line and enter:

./rabbitmq-plugins.bat enable rabbitmq_management

After the installation is successful, the browser can enter http://localhost:15672 to access the management page

The default account and password are guest

Note: Generally, a new administrator user will be created. The default guest is not used. The guest user can only access it under localhost. If it is accessed from other machines in the intranet, an error will be reported when logging in:

2. Centos 7 installation

A. Install erlang

Regarding installing erlang, RabbitMQ official website provides 4 ways to install:

I use the first method here, according to the installation method on Github

Create a new file: /etc/yum.repos.d/rabbitmq_erlang.repo

# vim /etc/yum.repos.d/rabbitmq_erlang.repo
[rabbitmq_erlang]
name=rabbitmq_erlang
baseurl=https://packagecloud.io/rabbitmq/erlang/el/7/$basearch
repo_gpgcheck=1
gpgcheck=1
enabled=1
# PackageCloud's repository key and RabbitMQ package signing key
gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey
       https://dl.bintray.com/rabbitmq/Keys/rabbitmq-release-signing-key.asc
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

[rabbitmq_erlang-source]
name=rabbitmq_erlang-source
baseurl=https://packagecloud.io/rabbitmq/erlang/el/7/SRPMS
repo_gpgcheck=1
gpgcheck=0
enabled=1
# PackageCloud's repository key and RabbitMQ package signing key
gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey
       https://dl.bintray.com/rabbitmq/Keys/rabbitmq-release-signing-key.asc
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

After saving, install yum

yum install erlang

Then open the command line and enter erl. If the version information of erlang is displayed, the installation is successful:

Note: Not directly yum install erlang, it will cause the version of erlang to be very low, and there will be version conflicts when rabbitMQ is installed later

B. Install other dependencies

In addition to erlang, RabbitMQ also needs to be installed: socat and logrotate

yum install -y socat logrotate

C. Install RabbitMQ

There are two ways to install:

rpm install
yum install

The first way: rpm installation

Download RabbitMQ, the specific rpm package link can refer to: official website download page

wget https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.8.14/rabbitmq-server-3.8.14-1.el7.noarch.rpm

Then rpm install:

rpm -ivh rabbitmq-server-3.8.14-1.el7.noarch.rpm

The second way: yum installation

Create a new file: /etc/yum.repos.d/rabbitmq_server.repo

# vim /etc/yum.repos.d/rabbitmq_server.repo
[rabbitmq_server]
name=rabbitmq_server
baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/7/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

[rabbitmq_server-source]
name=rabbitmq_server-source
baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/7/SRPMS
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

Then yum install:

yum install -y rabbitmq_server rabbitmq-server

Use rabbitmqctl to verify that the installation was successful:

>>> rabbitmqctl version
3.8.14

D. start

# -detached为可选参数，表示后台开启
rabbitmq-server -detached

If you want to close it, you can use:

rabbitmqctl stop

E. Install Web Management [optional]

rabbitmq-plugins enable rabbitmq_management

After the installation is complete, you can access it through localhost:15672

The default account and password are guest

Here's how to add an admin user (root here):

# 添加用户
rabbitmqctl add_user root 123456

# 赋予管理员权限
rabbitmqctl set_user_tags root administrator

# 查看用户列表
rabbitmqctl list_users