stream parallel custom spliterator spliterator

1. It is very simple to implement parallel jdk1.8 streams, as long as the parallel() method is added to the stream. The parallel principle of this method is to use the fork/join recursive method. When we use parallelism in development, one of the most concerned issues is shared variables. A commonly used method is to clone this variable every time it is used, and by modifying the cloned variable without affecting the original variable, it takes up more memory in exchange for performance. Of course, this is not what I want to discuss here. This time I will talk about how to divide data or tasks when streams are parallel.
1. We do a simple calculation of the number of words in a string, and calculate the number of words in the string. There is no corresponding processing in the operation method provided by the existing stream. Here is a modification of the processing, changing the String into a Stream, the code The following is as follows:
insert image description here
the logic of character processing is placed in the CounterWord class, because it is necessary to save whether the last character is empty. We use the reduce method of the stream to do the reduction statistics. The following is the code of the CounterWord class:
insert image description here
the result is found to be 7, which is correct, and then we use parallel to execute this code: the
insert image description here
result is found to be 11, the reason is that the fork/join method is used in parallel, and some words in the string are split when splitting Split into two words, so the word count turns out to be more than normal. We want to constrain its segmentation strategy.
2. The Spliterator interface is for customers to customize the spliterator. Look at the inheritance structure of its class, and you will find a large number of its implementations. It has the following abstract interfaces:
tryAdvance(Consumer<? super T> action), consume it if there are remaining elements in the stream, and return true at the same time, otherwise return false, indicating that there are no elements in the stream to process.
trySplit(), the logic method for processing the split strategy, returns a Spliterator.
estimateSize(), evaluates the remaining size that still needs to be divided, just evaluates, not accurate.
characteristics(), some characteristics of the splitter.
The following is the implementation code of segmentation:
insert image description here
insert image description here
insert image description here
insert image description here
here we need to distinguish between a custom segmenter and a processor that calculates the number of words. Both are recursive operations, the front is to split the string, and the back is to count words. In other words, the front splitter will split into several string data sources in parallel, and the subsequent counters will also recursively calculate each data source in parallel.

Guess you like

Origin blog.csdn.net/u013326684/article/details/102711168