Work Queues RabbitMQ learning (2)

1. Round robin scheduling (Round-robin dispatching): i.e., sequentially assigned to the task assigned worker.

2. Reply message (Message acknowledgement): After the consumer has been processed, a message reply. Avoid Kill worker, message message.

3. Message persistence (Message durability): After RabbitMQ server stops, to ensure that message is not lost. It requires persistence and message queue

4. Fair scheduling (Fair dispatch): In order to make the worker does not appear there has been busy, very busy and some consistent state. Using  channel.BasicQos (0, 1, false) , to ensure that the worker confirming the completion of a task, will assign the next.

5. Code

 

 

 

 

In the first tutorial, we are talking about the transmit and receive queue in a designated message. Next we assign a time-consuming task for the worker among the plurality of work queues (Work Queue).

 

Work Queue The main idea is to avoid doing immediately and resource-intensive task that must wait for it to finish. We task packaged into a message, and sent to the queue. This is inside the worker is in fact a consumer, then will perform these tasks by them.

We will count the string . To make the program SLEEP . That the use of Thread.Sleep () . For example Hello ... it takes 3 seconds.

 

Here our producer called the NewTask. And our consumer called the worker.

 

They can make some changes on the basis of one of the get

 

Sending a message code modification

var message = GetMessage(args);var body = Encoding.UTF8.GetBytes(message);
var properties = channel.CreateBasicProperties();
properties.Persistent = true;

channel.BasicPublish(exchange: "",
                     routingKey: "task_queue",
                     basicProperties: properties,
                     body: body);

private static string GetMessage(string[] args)
{
    return ((args.Length > 0) ? string.Join(" ", args) : "Hello World!");
}

Receiving a code modification message

var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
    var body = ea.Body;
    var message = Encoding.UTF8.GetString(body);
    Console.WriteLine(" [x] Received {0}", message);

    int dots = message.Split('.').Length - 1;
    Thread.Sleep(dots * 1000);

    Console.WriteLine(" [x] Done");
};
channel.BasicConsume(queue: "task_queue", autoAck: true, consumer: consumer);

The news is still inside the automatic reply

The above change was made to simulate a real time-consuming task.

Round robin scheduling (Round-robin dispatching)

One benefit of using the task queue is the ability to easily perform concurrent tasks.

First, we try to run two worker. They can simultaneously taken from the queue to the message. So what specifically is it?

You need to open the three console program, a two-run worker program.

 

# shell 1 cd Worker
dotnet run# => [*] Waiting for messages. To exit press CTRL+C

 

# shell 2 cd Worker
dotnet run# => [*] Waiting for messages. To exit press CTRL+C

The third is used to publish new tasks.

# shell 3 cd NewTask
dotnet run "First message."
dotnet run "Second message.."
dotnet run "Third message..."
dotnet run "Fourth message...."
dotnet run "Fifth message....."

We look at the results of two worker:

 

By default, RabbitMQ polls sends each message. So, on average, each consumer will get the same number of messages. This way is called distributed message polling.

 

Note also that in, queue the message can only be sent to a worker in that two worker in the task will not be repeated, that this is a point to point the way.

 

Message reply (Message acknowledgement)

Do you want next, if a consumer being a long task (long task), and to complete the part is dead. So what happens then? In our current code, once RabbitMQ send a message to a consumer, then the RabbitMQ where the message will immediately be marked for deletion (deletion). In this case, if you kill a worker, we will lose this message worker being processed, we will lose all the messages have been assigned to the worker but not yet processed.

 

Note: By default, not will wait for the next complete each task will distribute a message in the consumer, there are probably a lot about the distribution strip. Specifically by setting.

 

However, we do not want to lose mission tasks. If a worker died, we wanted task will be sent to another worker.

 

To message will not be lost, RabbitMQ introduced message acknowledge. An ack is received at a Message, sent back after the consumer process, and it is marked for deletion RabbitMQ.

 

If a consumer has not sent a ack died. RabbitMQ will think it has not been fully processed, and re-queue it. If the online as well as other consumer, then the RabbitMQ will soon send it to another consumer. Even though the worker died suddenly, and no message will be lost.

 

Outdated message does not exist. RabbitMQ will retransmit the message, when the consumer die. Even if it took a long time when processing message, it does not matter (because there is no out of date)

 

Manual message acknowledgment (manual reply message) is enabled by default. In the previous example, we autoAck by setting it to true closed. Now, we have to manually open the message reply (ie autoAck set to false), and, once we finish a task, we will send a confirmation (a acknowledgment).

 

var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
    var body = ea.Body;
    var message = Encoding.UTF8.GetString(body);
    Console.WriteLine(" [x] Received {0}", message);

    int dots = message.Split('.').Length - 1;
    Thread.Sleep(dots * 1000);

    Console.WriteLine(" [x] Done");

    channel.BasicAck(deliveryTag: ea.DeliveryTag, multiple: to false );   // message reply 
}; 
channel.BasicConsume (Queue: " task_queue " , AUTOACK: to false , Consumer: Consumer); // set manually open the reply message

 

In this way, even if we killed a worker, our message will not be lost.

 

Acknowledgement received message channels must be the same. Otherwise it will be reported channel-level protocol exception.

 

So, if we forget the hair acknowledgement what will happen?

Forget BasicAck is a mistake often occurs, but the consequences are very serious. When you exit the client, messages will be retransmitted. But RabbitMQ will eat (consume) more and more memory, as it can not release any unacked messages.

You can print infested confirmed by message_unacknowledged message

sudo rabbitmqctl list_queues name messages_ready messages_unacknowledged

Windows on

rabbitmqctl.bat list_queues name messages_ready messages_unacknowledged

 

Message persistence (Message durability)

We have learned that when the consumer is killed, the task is not lost. But if we stop the RabbitMQ server, and our task will still be lost.

When you want to stop the server, messages are not lost, you need to mark the message queue and are persistent (durable).

 

First, we mark queue is durable

channel.QueueDeclare(queue: "hello",
                     durable: true,  //标记queue为durable
                     exclusive: false,
                     autoDelete: false,
                     arguments: null);

 

Although the above code itself is correct, but at present it will not take effect. Because we have a hello before the queue is already defined, it is not durable. RabbitMQ will not allow you to use different parameters to redefine an existing queue, and will complain.

Here, we first declare a queue directly with different names. as follows

channel.QueueDeclare(queue: "task_queue",
                     durable: true,  //标记queue为durable
                     exclusive: false,
                     autoDelete: false,
                     arguments: null);

 

Which, QueueDeclare needs to be applied to producer and consumer of code.

Now, we mark messages as persistent (permanent). By setting IBasicProperties.SetPersistent to true.

var properties = channel.CreateBasicProperties();
properties.Persistent = true;  //设置message是persistent

 

Fair scheduling (Fair dispatch)

You may have noticed that the above schedule as we still do not want to work. It may be two worker has been a very busy, it has been a very busy (task execution time is not the same).

 

This is because RabbitMQ will be distributed when the message enter a queue. It will not look at a consumer unfinished queue, it is only the first of several blind to distribute the first of several consumer.

 

 

In order to change behavior, we can use BasicQos, and prefetchCount = 1. This will tell RabbityMQ to only give each worker a message. Or, RabbitRQ confirmed in the worker process and does not distribute a new message before. It can be said, RabbitMQ will be distributed under a busy worker.

channel.BasicQos(0, 1, false);

Note the size of the queue

If all your worker is busy, indicating your queue is full. You should make this a focus, and or you can add more worker or some other strategies.

 

Code

NewTask.cs

using System;using RabbitMQ.Client;using System.Text;
class NewTask
{
    public static void Main(string[] args)
    {
        var factory = new ConnectionFactory() { HostName = "localhost" };
        using(var connection = factory.CreateConnection())  //创建连接
        using(var channel = connection.CreateModel())   //创建channel
        {
            channel.QueueDeclare(queue: "task_queue",  //声明一个durable的queue
                                 durable: true,
                                 exclusive: false,
                                 autoDelete: false,
                                 arguments: null);

            var message = GetMessage(args);   //取得message
            var body = Encoding.UTF8.GetBytes(message);

            var properties = channel.CreateBasicProperties();  //设置message是persistent
            properties.Persistent = true;

            channel.BasicPublish(exchange: "",  //发送
                                 routingKey: "task_queue",
                                 basicProperties: properties,
                                 body: body);
            Console.WriteLine(" [x] Sent {0}", message);
        }

        Console.WriteLine(" Press [enter] to exit.");
        Console.ReadLine();
    }

    private static string GetMessage(string[] args)
    {
        return ((args.Length > 0) ? string.Join(" ", args) : "Hello World!");
    }
}

Worker.cs

using System;using RabbitMQ.Client;using RabbitMQ.Client.Events;using System.Text;using System.Threading;
class Worker
{
    public static void Main()
    {
        var factory = new ConnectionFactory() { HostName = "localhost" };
        using(var connection = factory.CreateConnection())  //建立连接
        using(var channel = connection.CreateModel())  //建立通道(channel)
        {
            channel.QueueDeclare (queue: " task_queue " ,   // statement queue is Durable 
                                 Durable: to true , 
                                 exclusive: false , 
                                 AutoDelete: false , 
                                 arguments The: null ); 

            channel.BasicQos (prefetchSize: 0 , prefetchCount: 1 , , Ltd. Free Join : false );   // set fair scheduling policy (fair dispatch) 

            Console.WriteLine ( "[*] Waiting for messages. " ); 

            Var Consumer = new new EventingBasicConsumer (Channel);   // callback 
            consumer.Received + = (Model, EA) => 
            { 
                var body = ea.Body;
                 var Message = Encoding.UTF8. the GetString (body); 
                Console.WriteLine ( " [X] Received {0} " , Message); 

                int DOTS = message.Split ( ' . ' .Length -) . 1 ;   // simulating real traffic takes some time 
                the Thread.Sleep ( * DOTS 1000 );

                Console.WriteLine ( " [X] the Done " ); 

                channel.BasicAck (deliveryTag: ea.DeliveryTag, Multiple: to false );   // message reply 
            ;} 
            channel.BasicConsume (Queue: " task_queue " ,   // send message and set manually reply open 
                                 AUTOACK: to false , 
                                 Consumer: Consumer); 

            Console.WriteLine ( " Press [Enter] to Exit. " ); 
            Console.ReadLine (); 
        } 
    } 
}

 

Reference website: RabbitMQ