When I returned to my hometown during the holiday, my father was repairing the fence and called Peng Xiaobei to help move the bricks. Sure enough, it's better not to run around during the holidays. He is singing and playing in the air conditioner at home. Doesn't he smell good?
Peng Xiaobei was tired without moving twice. This won't work! Thinking of the usual boss's teaching to him, big data technology should be used to drive business development. So he found a shady place to start technical selection.
Technology selection can be simply divided into three steps: demand understanding, proposal, implementation and optimization.
1. Need understanding
Moving bricks refers to the transportation of bricks, from point A to point B. Expect the sooner the better (how is this so familiar?). The factors that affect the speed of moving bricks are:
1. The distance between points A and B;
2. The number of bricks ;
3. The number of porters ;
4. The influence of abnormal weather;
5. Brick moving process.
2. Propose a plan
Peng Xiaobei analyzed these 5 factors. Among them, distance and weather are force majeure, and the number of emmm bricks seems to be force majeure. After all, his father will kill him. It seems that the number of porters and the brick-moving process can be optimized.
So the first plan came out: add a porter, change from single-threaded to multi-threaded, and increase the efficiency by 50%!
The optimization effect is very obvious, but there will be a problem: the distance between A and B is constant, and two people have to run around .
So the second optimized version was released: optimized brick-moving process ~~ serial execution tasks. Since there is no need to move, the efficiency is faster than two people running around, and the efficiency is increased by 30% again.
Three, landing and optimization
So Peng Xiaobei called to Peng Xiaobao to move bricks together. A lot of bricks were moved in a short while, and the efficiency was high. But the brick-moving system broke down after a while: the weather was too hot, Peng Xiaobao was tired and hot, and it was on strike. It was not easy to give popsicles! So back to the original single-threaded era , Peng Xiaobei is about to collapse! At this time, the boss's words linger again: use big data technology to drive business! Suddenly, an idea flashed: it can be solved by the principle of kafka (MQ ) !
Peng Xiaobei turned out the previous study notes, which was very clear.
Kafka (MQ) optimizes the information transmission process as follows:
Upstream and downstream decoupling
Synchronous to asynchronous
Peak shaving and valley filling
Increased reliability
The final plan is released
A message middleware is added between Peng Xiaobei and Peng Xiaobao. The producer Peng Xiaobei puts the bricks in the MQ, and Peng Xiaobao pulls the bricks from the MQ and stacks them on B. A short strike or outbreak by either party will not affect the brick handling work of the entire chain, perfect!
You can do more at the same time:
Producers only need to keep moving bricks to MQ to achieve maximum capacity (maximum throughput)
Put the disordered bricks and order codes of A ground in MQ, the writing speed is faster (ordered writing)
Consumers move bricks on demand, and are not afraid of moving too many bricks at one time from upstream (peak cutting and filling valley)
The producer puts the bricks in the MQ, instead of handing it to the consumer, the two do their own things (business decoupling)
Peng Xiaobei and Peng Xiaobao finally finished moving the bricks, and happily went to report their findings to their father. After his dad listened, he showed him a picture:
