divide and conquer
A very efficient way to handle large amounts of data. The famous MapReduce also adopts the idea of divide and conquer. Simply put, if you want to process 1000 data, but you do not have the ability to process 1000 data, then you can only process 10 of them, then process 100 times in stages, and combine the results of 100 times. That is the final desired result of processing the original 1000 data.
Fork/Join framework
In Java, after using fork(), the system has an additional execution branch (thread), and the index needs to wait for the execution of the execution branch to complete before it is possible to get the final result, so join means waiting.
If you use fork() to open threads for processing without any scruples, it is likely to cause the system to open too many threads and seriously affect performance. All JDKs provide a ForkJoinPool thread pool. The fork() method does not rush to start threads, but submits them to the ForkJoinPool thread pool for processing to save system resources.
ForkJoinPool thread pool
flow chart
Note: Due to the optimization of the thread pool, there is not a one-to-one relationship between the number of tasks submitted and the number of threads. In most cases, a physical thread actually needs to handle multiple logical tasks. Therefore, each thread must have a task queue. Therefore, in the actual execution process, this situation may be encountered: thread A has completed its own tasks, and thread B has a bunch of tasks waiting to be processed. At this time, thread A will "help" thread B, Take a task from the task queue of thread B and process it to achieve a balance as much as possible. The picture below shows this spirit of mutual help. It is worth noting that when a thread master and apprentice help others, it always starts to get data from the bottom of the task queue, and when a thread tries to perform its own task, it starts from the opposite top, so this behavior is beneficial to avoid data races .
