Implement batch processing using skip + limit of Java8 Stream

1. Generally, data will be added to a temporary collection during batch processing. When the amount of data reaches a certain size, the next step will be performed. When the amount of data is insufficient, additional judgment is required;

2. If you use the skip + limit in Java8's Stream, you can operate the collection conveniently and quickly, among which: (1)
skip(x): return the new value composed of the remaining elements after discarding the first x elements in the stream Stream; if the number of elements contained in the original stream is less than x, an empty stream is returned.
(2) limit(x): Perform a truncation operation on a Stream to obtain its first x elements; if the number of elements contained in the original stream is less than x, then obtain all its elements;

3. The example code is as follows:

public static void main(String[] args) {

        List<Integer> list = new ArrayList<>();
        for (int i = 10; i < 36; i++) {
            list.add(i);
        }

        int limit = 10;
        for (int offset = 0; offset < list.size(); offset += limit) {
            List<Integer> subList = list.stream()
                                        .skip(offset)
                                        .limit(limit)
                                        .collect(Collectors.toList());
            System.out.println(subList);
        }
}

Output result:

[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
[20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[30, 31, 32, 33, 34, 35]

3. It can also be seen from the results that when skip is used in conjunction with the limit operation, limit is the maximum size that can be retrieved, and there is no need to judge the size of the last batch of data when the size is not enough.

Improved version

int j = 0, size = list.size(), batchSize = 100;
while (j < size) {
    batchList = list.stream().skip(j).limit(Math.min(j + batchSize, size) - j).collect(Collectors.toList());
    j += batchSize;
}