Message Queue: rabbitMQ

What is rabbitMQ?

rabbitMQ a messaging middleware is based AMQP protocol, which provides reliable message transfer between applications. On ease of use, scalability, high availability, outstanding. And the use of decoupling between the messaging middleware conducive to the application , the producer (clients) do not need to know the existence of consumers (server) is. And the two ends can use different languages, providing significant flexibility.

Installation rabbitmq reference links centos7 on: https://www.cnblogs.com/liaojie970/p/6138278.html

 

After rabbitmq installed, remote connection rabbitmq server, then you need to configure permissions

1 . First create a user on Server RabbitMQ 
[root @ RabbitMQ ~] # rabbitmqctl add_user ADMIN 123456
 
2 while also configuring permissions to allow access from the outside:. 
[Root @ RabbitMQ ~] # . Rabbitmqctl set_permissions -p / ADMIN "* . "*". "" * "
 
3 . assign administrator privileges to the user (optional) 
[root @ RabbitMQ ~] # rabbitmqctl set_user_tags ADMIN administrator
 
4 . list all user commands: 
[root @ RabbitMQ ~] # rabbitmqctl list_users

When the client connection authentication parameters need to be configured as follows:

credentials = pika.PlainCredentials('admin','123456')    # 用户名和密码
connection = pika.BlockingConnection(
        pika.ConnectionParameters("10.0.0.124",5672,"/",credentials)
    )
channel = connection.channel()

 

rabbitMQ basic concepts

• the Exchange: Producer can only send messages to the exchange. The exchange is responsible for sending messages to queues. Exchange must know exactly how to handle it receives a message that is being sent to a particular queue or many quenes, or abandoned, these rules are defined by exchange type. The main type has a direct, topic, headers, fanout. Specific use a different type for different scenarios.
• queue:  carrier message queue, the message. Receiving a message from the exchange, and then taken out of the consumer. exchange and queue can be one to many, to bind them by routingKey.
• Producer:  the producer, the source of the message, the message must be sent to the exchange. Rather than directly to the queue
• Consumer:  Consumers, directly from the message queue for consumption, rather than from the exchange.

From the above it can be seen that substantially works Rabbitmq producer sends a message to the exchange. The routingKey rabbitMQ responsible to exchange messages sent from the corresponding queue to the binding, which is responsible for doing the rabbitMQ. The consumer can only get messages from the queue. The basic effect is as follows: 

 

Persistent problems

1. message acknowledgment.

Here there will be a problem, if the consumer while on duty will take some time, this time if suddenly hung up, the message has not been completed, the message would not be lost, in order to prevent message loss, rabbitmq provides the message confirmation mechanism, Upon receiving the consumer, it will send a message after performing rabbitmq ack to tell it may be removed from the queue in the message. If you do not receive ack. Rabbitmq will re-send the message this provision, if there are other consumer online, you will receive this message and consumption. Message acknowledgment mechanism is turned on by default. If you want to turn off only need to set no_ack = true.

2. persistent queue

• In addition to the consumer we have to ensure that messages are not lost after rabbitMQ hung up. Here we need to make sure the message queue and queue messages have to be persistent.
• Then you need to set the queue lasting durable properties.

channel.queue_declare (Queue = task_queue, Durable = True)     # Durable persistent queue = True

3. news endurance of

If the message is persistent through delivery_mode attribute set to a value of 2.

channel.basic_publish(exchange='',
                      routing_key="task_queue",
                      body=message,
                      properties=pika.BasicProperties(
                         delivery_mode = 2,     # make message persistent （消息持久化）
                      ))

 

A simple model to send 

Messages can never be sent directly to where the rabbit MQ queue. Here we provide a through  to use the default exchange an empty string . This is a special exchange, it can send a message to a specified queue according routingKey . So our design looks like this:

Sending end Code: send.py

import pika
credentials = pika.PlainCredentials('admin','123456')

connection = pika.BlockingConnection(
        pika.ConnectionParameters("10.0.0.124",5672,"/",credentials)
    )

channel = connection.channel()

# 声明 queue
channel.queue_declare(queue="hello")

# n RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
channel.basic_publish ( 
        exchange = '' ,         # empty string indicates the default exchange. This is a special exchange, it can send a message to a specified queue according routingKey. 
        routing_key = ' Hello ' , 
        body = ' the Hello World ' 
    ) 

Print ( " [X] Sent 'the Hello World!' " ) 
Connection.close ()

Receiving end Code: receive.py

import pika
credentials = pika.PlainCredentials('admin','123456')

connection = pika.BlockingConnection(
        pika.ConnectionParameters("10.0.0.124",5672,"/",credentials)
    )

channel = connection.channel()


#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue='hello')

def callback(ch,method,properties,body):
    print("[x] Received %r" % body)


channel.basic_consume(
        callback,
        queue='hello',
        no_ack=True        # 默认为 False
    )

print("[*] Waiting for messages.To exit press CTRL+C")
channel.start_consuming()

 

Second, the work queue mode

A producer to send messages to the queue, a queue is shared with a plurality of consumers, each consumer access message is unique. In this mode, the default p RabbitMQ message sent sequentially distributed to the consumer (C), with similar load balancing, as shown below:

The principle of equitable distribution of messages (similar to load balancing)

If the order Rabbit just send the message to all consumers, regardless of consumer load, it is likely to occur, a machine configuration where the consumer is not high accumulation of a lot of message handling could not finish, while high consumer has been configured Easy. To solve this problem, can be in various consumer side, configure = 1 perfetch , meaning that when I tell RabbitMQ the consumer has not finished processing the current message, do not give me the news. ,As shown below:

Sending end Code: send.py

import pika
import sys

credentials = pika.PlainCredentials("admin","123456")

connection = pika.BlockingConnection(
        pika.ConnectionParameters("10.0.0.124",5672,"/",credentials)
    )

channel = connection.channel()

channel.queue_declare(queue="task_queue",durable=True)

msg = " ".join(sys.argv[1:]) or "Hello World!"
channel.basic_publish(
        exchange="",
        routing_key="task_queue",
        body=msg,
        properties=pika.BasicProperties(
            delivery_mode=2,    # make message persistent （使消息持久化）
        )
    )

print("[x] Send %r" % msg)
connection.close()

Receiving end Code: receive.py

import pika
import time
import random

credentials = pika.PlainCredentials("admin","123456")

connection = pika.BlockingConnection(
        pika.ConnectionParameters("10.0.0.124",5672,"/",credentials)
    )

channel = connection.channel()

channel.queue_declare(queue="task_queue",durable=True)

defthe callback (CH, Method, Properties, body):
     Print ( " [X]% R & lt Received " % body) 
    the time.sleep (the random.randint ( 1,50 ))
     Print ( " [X] the Done " ) 
    ch.basic_ack ( delivery_tag = method.delivery_tag)         # sends ACK 

channel.basic_qos (prefetch_count = 1)         # in various consumer side, configure perfetch = 1, meaning that RabbitMQ would not have told me this at the time the consumer has not finished processing the current message He sent me the news. 
channel.basic_consume ( 
        callback, 
        Queue = " task_queue " 
    )         # no_ack 默认为 False

print("[x] Waiting for messages.To exit press CTRL+C")
channel.start_consuming()