Article Directory
1. Introduction to reconciliation model
Operation logic of reconciliation system
- When users place an order through the online mall, an electronic order will be generated and stored in the order database;
- After that, the logistics will generate a delivery order for delivery to the user, and the delivery order is stored in the delivery order database;
- In order to prevent missed deliveries or repeated deliveries, the reconciliation system will also check for abnormal orders every day.
The model diagram is as follows, and the
code abstraction is as follows:
while(存在未对账订单){
// 查询未对账订单
pos = getPOrders();
// 查询派送单
dos = getDOrders();
// 执行对账操作
diff = check(pos, dos);
// 差异写入差异库
save(diff);
}
2. Use parallel optimization reconciliation system
If the single-threaded system is changed to multi-threaded, in the
following code, two threads T1 and T2 are created to execute two operations, query unreconciled order getPOrders () and query delivery single getDOrders () in parallel. Perform reconciliation operations check () and difference write save () in the main thread. However, it should be noted that the main thread needs to wait for the execution of threads T1 and T2 to execute the two operations check () and save (). For this purpose, we call T1.join () and T2.join () to achieve the wait. When the T1 and T2 threads exit, the main thread calling T1.join () and T2.join () will be awakened from the blocking state , so that the subsequent check () and save () will be executed.
while(存在未对账订单){
// 查询未对账订单
Thread T1 = new Thread(()->{
pos = getPOrders();
});
T1.start();
// 查询派送单
Thread T2 = new Thread(()->{
dos = getDOrders();
});
T2.start();
// 等待 T1、T2 结束
T1.join();
T2.join();
// 执行对账操作
diff = check(pos, dos);
// 差异写入差异库
save(diff);
}
3. Use CountDownLatch to achieve thread waiting
The shortcomings of the above code: a new thread is created every time in the while loop, and creating a thread is a time-consuming operation. Therefore, it is best to create a thread that can be recycled, and the thread pool can solve this problem.
After the thread pool is optimized: we first create a thread pool with a fixed size of 2, and then reuse it in the while loop. Everything looks smooth, but one problem seems to be unsolved, that is how the main thread knows when the two operations getPOrders () and getDOrders () are executed. The previous main thread waited for threads T1 and T2 to exit by calling the join () method of threads T1 and T2, but in the thread pool scheme, the thread would not exit at all, so the join () method has become invalid.
// 创建 2 个线程的线程池
Executor executor = Executors.newFixedThreadPool(2);
while(存在未对账订单){
// 查询未对账订单
executor.execute(()-> {
pos = getPOrders();
});
// 查询派送单
executor.execute(()-> {
dos = getDOrders();
});
/* ？？如何实现等待？？*/
// 执行对账操作
diff = check(pos, dos);
// 差异写入差异库
save(diff);
}
The solution is to set a counter, when the calculator is 0, it means the thread execution is completed.
// 创建 2 个线程的线程池
Executor executor = Executors.newFixedThreadPool(2);
while(存在未对账订单){
// 计数器初始化为 2
CountDownLatch latch = new CountDownLatch(2);
// 查询未对账订单
executor.execute(()-> {
pos = getPOrders();
latch.countDown();// 计数器减1
});
// 查询派送单
executor.execute(()-> {
dos = getDOrders();
latch.countDown(); // 计数器减1
});
// 等待两个查询操作结束
latch.await(); // 等待计数器为0
// 执行对账操作
diff = check(pos, dos);
// 差异写入差异库
save(diff);
}
4. Further optimize performance
The two query operations getPOrders () and getDOrders () can also be used to reconcile the check () and save () operations in parallel.
Producer-Consumer
Two query operations are producers, and reconciliation operations are consumers. Since it is a producer-consumer model, you need to have a queue to hold the data produced by the producer, and consumers consume data from this queue.
The order query operation inserts the order query result into the order queue, and the dispatch order query operation inserts the dispatch order into the dispatch order queue. There is a one-to-one correspondence between the elements of the two queues. The advantage of the two queues is that the reconciliation operation can output one element from the order queue and one element from the dispatch single queue at a time, and then perform reconciliation operations on these two elements so that the data will not be messed up.
Thread T1 and thread T2 can only be executed together when they have finished producing one piece of data , that is, thread T1 and thread T2 have to wait for each other and have the same pace ; at the same time, when both threads T1 and T2 have produced one piece of data At the same time, it should also be able to notify the thread T3 to perform reconciliation operations.
5. Use CyclicBarrier to achieve thread synchronization
The difficulty of the above scheme: one is that threads T1 and T2 must be in the same pace, and the other is to be able to notify thread T3 .
// 订单队列
Vector<P> pos;
// 派送单队列
Vector<D> dos;
// 执行回调的线程池
Executor executor = Executors.newFixedThreadPool(1);
final CyclicBarrier barrier =
new CyclicBarrier(2, ()->{
executor.execute(()->check());
});
void check(){
P p = pos.remove(0);
D d = dos.remove(0);
// 执行对账操作
diff = check(p, d);
// 差异写入差异库
save(diff);
}
void checkAll(){
// 循环查询订单库
Thread T1 = new Thread(()->{
while(存在未对账订单){
// 查询订单库
pos.add(getPOrders());
// 等待
barrier.await();
}
}
T1.start();
// 循环查询运单库
Thread T2 = new Thread(()->{
while(存在未对账订单){
// 查询运单库
dos.add(getDOrders());
// 等待
barrier.await();
}
}
T2.start();
}
First, a CyclicBarrier with a counter initial value of 2 is created. When creating a CyclicBarrier, a callback function is also passed in. When the counter decreases to 0, this callback function is called.
- Thread T1 is responsible for querying the order. When it finds one, it calls barrier.await () to decrement the counter and wait for the counter to become 0;
- Thread T2 is responsible for querying the delivery order. When it finds one, it also calls barrier.await () to decrement the counter by 1 while waiting for the counter to become 0;
- When both T1 and T2 call barrier.await (), the counter will decrease to 0. At this time, T1 and T2 can execute the next statement and call the barrier callback function to perform the reconciliation operation.
- It is worth mentioning that the CyclicBarrier counter has an automatic reset function. When it drops to 0, it will automatically reset the initial value you set. This function is very convenient to use.
6. Summary
- CountDownLatch is mainly used to solve the scenario where one thread waits for multiple threads;
- CyclicBarrier is a group of threads waiting for each other;
- The counter of CountDownLatch cannot be recycled, that is to say, once the counter decreases to 0, and then a thread calls await (), the thread will pass directly;
- CyclicBarrier's counter can be recycled, and has the function of automatic reset. Once the counter decreases to 0, it will automatically reset to the initial value you set, and you can also set a callback function.
