MQ called the Message Queue, the message queue (MQ)
It is an application communication method according to the application. Application to communicate via a message (data for the application program) read out of the queue, without the need for a dedicated connection to link them.
It is usually a software program to some minor calls between each other, in order to carry out normal work, then call each other every program, there are ways to send information sent by the two, one is sent directly to another small program the other is to send me the information I do put a temporary intermediate conversion software in this intermediate conversion software is MQ.
This in-between between the two software called middleware: software work between two or more software, just like an intermediary, nexus, common middleware, rabbitmq (responsible for communication between the two components a), tomcat (explain java program), memcache (do site cache), redis (do site cache).
OSI layer 4 layer transmission scheme: tcp (high reliability) / udp (untrusted), then at this level, it identifies the ports, I can recognize a packet transmission over the data, the data packet is port in four to resolve.
MQ is like to work in co-udp inside, but then, this is a sender. He sent a message of it, to put a middleware, the middleware from the client to go to get the news, so it, to decouple the relationship between the sender and a couple of clients. With this middleware later, I sent a message after the end into the middleware, you can do other things, called decoupling (also called loose coupling can).
Refers to message passing communication between a program in the data message transmitted by, rather than through direct calls to communicate with each other, is a technique commonly called directly, such as remote procedure calls. It refers to the application queue by the queue to communicate. Removed using a queue and transmission requirements received concurrently executing applications.
It refers to the application queue by the queue to communicate. Removed using a queue and transmission requirements received concurrently executing applications.
AMQP 即 Advanced Message Queuing Protocol
Advanced Message Queuing Protocol, is an open standard application-layer protocol for message-oriented middleware design. Message middleware mainly for decoupling between the components, sender of the message without knowing the user's presence information, and vice versa.
AMQP main feature is a message for the queue, the routing (including point and publish / subscribe), reliability and safety.
Rabbitmq concept
Belonging to a popular open source system message queue. Belonging to the AMQP (Advanced Message Queuing Protocol) to achieve a standard. It is an open standard application-layer protocol for message-oriented middleware design. In a distributed system for store and forward messages, in terms of ease of use, scalability, high availability, and so doing well.
RabbitMQ Features:
Written in Erlang
Support persistence: when we learn redis, and it stores the message in two ways, first memory (provisional), the second disk (persistent). It can also work through two ways, one end of the message there is a temporary memory a persistence to disk.
Support HA: He let us take this availability is not keepalived, lvs then take it they have their own high-availability clusters. Like galera, he was a program mariadb cluster.
Providing development client terminal C #, erlang, java, perl, python, ruby etc.
What is the coupling, decoupling
A coupling
1, the coupling means between two or more systems or two forms of movement by interaction affect each well together phenomenon.
2, in software engineering, is the degree of coupling between objects dependencies between objects. The higher the coupling between objects, higher maintenance costs, so the design should minimize the object class and the coupling between the member.
3, Classification: coupling between the coupling, as well as hardware and software between the software modules. Coupling is a measure of the program structure of the linkages between the various modules. It depends on the complexity of the interface between each module, the calling module and what information the way through the interface.
Second, decoupling
1, decoupling, literally means to decouple the relationship.
2, in software engineering, i.e. to reduce the degree of coupling can be understood as decoupling, dependent relationship must exist between the coupling module, coupled to the theoretical absolute zero is impossible, but can be identified by conventional methods to reduce the degree of coupling to a minimum.
3, the design of the core idea: to minimize coupling of code, if the code is coupled found, it is necessary to take decoupling technology. Let data model, business logic and views between the three lower display coupled to one another, the association relies to a minimum, and thus will not affect the situation as a whole. A principle feature of the code is not written in the function code B, if needed interaction between the two, through the interface, through a message, or even the introduction of the framework, but in short, is not to write directly cross.
RabbitMQ in terms of concept
Broker: Message Queue Server is simply an entity.
I deployed a rabbitmq service, this server is called Broker.
Exchange: Message switch that specifies what the message according to the rules, which are routed to a queue.
Not among the network switch. I decided to put my message how the message queue up.
Queue: vector message queue, each message will be put into one or more queues.
Binding: binding, its role is to exchange routing and queue bind in accordance with the rules.
Routing Key: routing keywords, exchange of messages delivered in accordance with this keyword.
Just like you go to the train station you go to apply for a ticket refund or altered tickets business, you go to the ticket hall after each window labeled with a different identity, there is the ticket window, the window has altered tickets, refund window there, there are soldiers window, then identifies the window is similar to Routing Key, you are like a message, after you see the keywords that you want routed behind the corresponding queue.
vhost: web hosting, where a broker can offer more vhost, as a different user privilege separation.
I have a message queue server, he can only give not only provide a service message queue, you can give a number of service providers, so it is easy to provide more confusion, so I just use the concept of a logical division of resources, these services are separated into different virtual hosts where this is the case, these messages will not string together.
producer: news producer, is the program delivered the message.
Who made the news
consumer: consumer news, is that the program accepts messages.
Who will read the message
channel: message channels, each client's connection can be established multiple channel, each channel representing a session task.
RabbitMQ works
MQ consumption - a model representative of typical producer, end to continue to write messages in the message queue, and the other end can be read or subscribe to messages in the queue. MQ is a concrete follow AMQP protocol implementations and products. In the project, without some return immediately extracted and time-consuming operation, made asynchronous processing, and this processing asynchronous requests greatly saving server response time, thereby increasing the throughput of the system.
(1) Message Queuing clients connect to the server to open a channel.
(2) a declaration client exchange, and provision of the relevant property, what kind of queue I declare this switch bearer.
(3) The client declares a queue, and set the associated property.
(4) The client uses routing key, establish a good relationship between the exchange and the binding queue.
(5) the delivery message to the client exchange.
After (6) exchange receives the message, according to the message and key binding has been set, route messages, the message delivered to one or more queue.
Rabbitmq 的 metadata
Metadata can be persistent in the RAM Disc or RabbitMQ from this angle can be divided into two nodes in the cluster:. RAM Node and Disk Node.
What is metadata?
A file, disk he wants us to keep them, there are two ways, he will put this document is divided into two types, the first one is information about the file, the second is the actual data files, then about a file information is metadata. The metadata includes the name of the file owner, size, modification time to create, format. Metadata is usually stored in the inode inside. This is really specific data files stored in block inside.
RAM only the Node metadata stored in the RAM
Disk node metadata will be persisted to disk.
Single-node system is not any choice, allowing only disk node, or because there is no data redundancy, once restarted will lose all the configuration information, but in a clustered environment can choose which nodes are RAM node. Declare (declare) in the cluster Creating exchange queue binding, such operations until all nodes will return to complete the creation of:
If it is necessary to modify memory data memory node,
If it is necessary to wait for the write to disk disk node, the node where excessive speed will be greatly slow down.
Some scenes exchang queue fairly constant, change very little, and that even if all disc node, there is no effect if used Rabbitmq do RPC. (RPC: Remote Procedure Call- remote procedure call), RPC or RPC-like scene this serious problem the frequent destruction create a temporary queue, the disk reading and writing skills quickly become a performance bottleneck. Therefore, in most cases, we try to create a Node for the RAM Node. Here there is a problem, metadata in order to restart the cluster may need to restore the cluster metadata persisted to disk, it requires planning RabbitMQ cluster RAM Node and Disc Node.
As long as there is a node Disc Node will be able to provide the conditions for the cluster metadata written to the disk, RabbitMQ and indeed such requirements: a cluster as long as one disk node can, others can be a RAM node cluster nodes to join or withdraw from certain at least to notify a disk node in the cluster.
If the cluster disk node are dawdle away, do not change the metadata of the cluster. Disclaimer exchange queue modify user permissions, add users, etc. These changes can not be recovered after a node restart.
A situation requires that all disk node to be the case in order to operate online that add or remove nodes .RAM node will start when connected to the predefined disk node to download the latest cluster metadata. If you have two disk node (d1 d2), when a RAM node join you just tell d1, and just this RAM node restart when d1 does not start, restart will fail. so when adding RAM nodes, all the disk node information to tell it will RAM node disk node information is persisted to disk for subsequent startup can find what you want.
Rabbitmq cluster deployment
First, preparation
(1) Conditions: Prepare three linux system to ensure good source configuration, and epel source
(2) three machines can still resolve each other
(3) setting can keyless landing
Second, the installation process:
(1) All node execute:
Download and install rabbitmq package erlang:
yum install -y erlang rabbitmq-server.noarch
Start rabbitmq:
systemctl start rabbitmq-server.service
It is set to boot from Kai rabbitmq:
systemctl enable rabbitmq-server.service
View rabbitmq start status:
systemctl status rabbitmq-server.service
Check listening port:
netstat -lantp | grep 5672
rabbitmq profile:
/etc/rabbitmq/rabbitmq.config
(2) node1: Modify guest password is admin (default user: guest password: guest)
rabbitmqctl change_password guest admin
(3) node1: openstack add a user, and the password is admin. And set permissions and administrator, if you do not set the administrator, this user will not be landing on the graphical interface.
node1：
rabbitmqctl add_user openstack admin
rabbitmqctl set_permissions openstack ".*" ".*" ".*"
rabbitmqctl set_user_tags openstack administrator
(4) Each node: edit rabbittmq variable file
vim /etc/rabbitmq/rabbitmq-env.conf
RABBITMQ_NODE_PORT=5672
ulimit -S -n 4096
RABBITMQ_SERVER_ERL_ARGS="+K true +A30 +P 1048576 -kernel inet_default_connect_options [{nodelay,true},{raw,6,18,<<5000:64/native>>}] -kernel inet_default_listen_options [{raw,6,18,<<5000:64/native>>}]"
RABBITMQ_NODE_IP_ADDRESS = current host IP
View rabbitmq plug
/usr/lib/rabbitmq/bin/rabbitmq-plugins list
(5) All node open web management page of rabbitmq
rabbitmq-plugins enable rabbitmq_management
Restart:
systemctl restart rabbitmq-server.service
View status:
systemctl status rabbitmq-server.service
(6) using a browser landing
web page login: ip: 15672
Port number for this service: 5672
rabbitmq cluster configuration
(1) node1 erlang.cookie sent to another node cluster configuration
rabbitmqctl status
scp /var/lib/rabbitmq/.erlang.cookie node2 hostname: /var/lib/rabbitmq/.erlang.cookie
scp /var/lib/rabbitmq/.erlang.cookie node3 hostname: /var/lib/rabbitmq/.erlang.cookie
(2) node2 node3 and stop the application, and in a manner ram added node1 node restart after application
node2 and restart node3 respectively rabbitmq:
systemctl restart rabbitmq-server.service
node2 and node3 respectively stop app:
rabbitmqctl stop_app
node3 were added node2 and node1:
rabbitmqctl join_cluster --ram rabbit @ node1 hostname
node2 and restart respectively node3 app:
rabbitmqctl start_app
(3) node1 check the cluster status
rabbitmqctl cluster_status
(4) landed verification: http: // node1IP: 15672 / # / guest / admin
Other commands:
(1) add an administrator:
Add user names and passwords:
rabbitmqctl add_user mqadmin mqadmin
Set this user as an administrator:
rabbitmqctl set_user_tags mqadmin administrator
Add to this user with permission:
rabbitmqctl set_permissions -p / mqadmin ".*" ".*" ".*"
(2) change the node type (type of memory or disk type)
rabbitmqctl stop_app
rabbitmqctl change_cluster_node_type disc 或 rabbitmqctl change_cluster_node_type ram
rabbitmqctl start_app
(3) is removed from the cluster nodes (nodes or reset)
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl start_app
rabbitmqctl cluster_status
(4) removal of other nodes in the cluster from a node
rabbitmqctl forget_cluster_node rabbit@node3
rabbitmqctl reset
rabbitmqctl start_app
rabbitmqctl cluster_status
1. To ensure that there is at least one disk cluster node types to prevent data loss, when you change the node type with particular attention.
2. If the entire cluster to be stopped, and should ensure that the last one down out of the first node is started, if not will have to use the command to move it out of the cluster forget_cluster_node
3. If the cluster nodes almost simultaneously in an uncontrolled manner down in the case where a command to reset the node node uses force_boot