Exactly when to use MQ

Disclaimer: The following article comes from the way the architect, the author Shen Jian 58

Components from the business of introducing any bullying. The introduction of a component, the first question to answer is, this component to solve the problem.

What is MQ

Message bus (Message Queue), hereinafter called the MQ, a communication mechanism across the process for the delivery message downstream. Voice-over: usually not on the same server both processes.

In the Internet architecture, MQ is often used as a "downstream decoupling":

The message sender only rely MQ, a consumer who is not concerned;
Message consumer only rely MQ, does not care who the sender is;

Vo: the sender and the consumer, not dependent on each other logically and physically.

When not in use MQ?

When a caller needs to be concerned about the results of the message is not normally used MQ, using RPC calls.

ret = PassportService::userAuth(name, pass);

switch(ret){

    case(YES) : return YesHTML();

    case(NO) : return NoHTML();

    case(JUMP) : return 304HTML():

    default : return 500HTML();

}

As illustrated, the upstream Passport service calls, the results of different treatments, different logic operations may go processing branches (successful login, the login fails, execution errors, etc.), i.e., "the processing result of the strong dependence", RPC calls should be used at this time. Voice-over: the vast majority of cases, you should use the RPC.

At this point if forced to use MQ it?

If forced to use MQ communication, the caller can not directly tell the user login success and or failure, will have to wait for another MQ notification callback. So to play, not only makes coding complexity, also introduce the risk of message loss, adding a layer of a multi-center, superfluous.

What MQ when using it?

The following four typical scenario, you should use MQ.

A typical scenario: data-driven task dependencies

What is task-dependent : For chestnuts, Internet companies often make some statistics task in the morning, there is a certain dependencies between these tasks, such as:

task3 task2 output required as input;
task2 task1 output required as input;

这样的话，tast1, task2, task3之间就有任务依赖关系，必须task1先执行，再task2执行，载task3执行。

对于这类需求，通常怎么实现呢？常见的玩法是，crontab人工排执行时间表。

如上图，手动设定如下：

task1，0:00执行，经验执行时间为50分钟；
task2，1:00执行（为task1预留10分钟buffer），经验执行时间也是50分钟；
task3，2:00执行（为task2预留10分钟buffer）；

crontab手动排表有什么坏处呢？

如果有一个任务执行时间超过了预留buffer的时间，将会得到错误的结果，因为后置任务不清楚前置任务是否执行成功，此时要手动重跑任务，还有可能要调整排班表；
总任务的执行时间很长，总是要预留很多buffer，如果前置任务提前完成，后置任务不会提前开始；
如果一个任务被多个任务依赖，这个任务将会称为关键路径，排班表很难体现依赖关系，容易出错；
如果有一个任务的执行时间要调整，将会有多个任务的执行时间要调整；

无论如何，采用“crontab排班表”的方法，各任务严重耦合，谁用过谁痛谁知道。画外音：用过的，痛过的，请留言。

应该如何优化呢？采用MQ解耦。