Learn to use java8 skip and limit

Sometimes it is necessary to batch process data, especially when using mybatis for batch insertion.

skip+limit learning

1. The skip(long n) method skips the first n (non-negative) elements and returns the remaining stream, which may be an empty stream

2. limit(long maxSize): The method truncates its previous maxSize element; if the number of elements contained in the original stream is less than maxSize, then get all its elements;

skip

    public static void main(String[] args) {
        skip();
    }

    // skip(long n) 方法跳过前 n （非负）个元素，返回剩下的流，有可能为空流。
    public static void skip() {
        List<String> names = Arrays.asList("ling", "lian", "yan", "feng", "yun", "wen", "tian", "lang", "xiu");
        String skipFirstName = names.stream().skip(0).findFirst().orElse("");
        System.out.println("skipFirstName: " + skipFirstName);
        String skipLastName = names.stream().skip(names.size() - 1).findFirst().orElse("");
        System.out.println("skipLastName: " + skipLastName);
        names = Collections.singletonList("yan");
        // 超出报错
        String penultName = names.stream().skip(names.size() - 2).findFirst().orElse("");
        System.out.println("reduce penultName: " + penultName);
    }

If the third exceeds, an error will be reported. 

limit:

     public static void main(String[] args) {
        limit();
    }
 

    // limit(long n): 方法截断其前个n元素；若原流中包含的元素个数小于n，那就获取其所有的元素；
    public static void limit() {
        List<String> names = Arrays.asList("ling", "lian", "yan", "feng", "yun", "wen", "tian", "lang", "xiu");
        List<String> limit1 = names.stream().limit(5).collect(Collectors.toList());
        System.out.println("inner size: " + limit1);
        List<String> limit2 = names.stream().limit(25).collect(Collectors.toList());
        System.out.println("over size: " + limit2);
    }

result:

inner size: [ling, lian, yan, feng, yun]
over size: [ling, lian, yan, feng, yun, wen, tian, lang, xiu]

skip+limit batch processing

    public static void main(String args[]) {
        List<Integer> datas = new ArrayList<>();
        for (int i = 0; i < 26; i++) {
            datas.add(i);
        }
        skipData2(datas, 5);
        System.out.println("===========");
        skipData2(datas, 6);
    }

    public static void skipData1(List<Integer> datas, int maxSize){
        int dataSize = datas.size();
        // 向上取整
//        int loop =  dataSize / maxSize + (dataSize % maxSize != 0 ? 1 : 0);
        int loop =(int) Math.ceil(dataSize * NumberUtils.DOUBLE_ONE / maxSize);
        List<Integer> skipData;
        for (int i = 0; i < loop; i++) {
            skipData = ListUtils.emptyIfNull(datas).stream().skip(i * maxSize).limit(maxSize).collect(Collectors.toList());
            System.out.println(skipData);
        }
    }

    public static void skipData2(List<Integer> datas, int maxSize){
        List<Integer> skipData;
        for (int offset = 0, listSize = datas.size(); offset < listSize; offset += maxSize) {
            skipData = datas.stream()
                    .skip(offset)
                    .limit(maxSize)
                    .collect(Collectors.toList());
            System.out.println(skipData);
        }
    }

result:

[0, 1, 2, 3, 4]
[5, 6, 7, 8, 9]
[10, 11, 12, 13, 14]
[15, 16, 17, 18, 19]
[20, 21, 22, 23, 24]
[25]
===========
[0, 1, 2, 3, 4, 5]
[6, 7, 8, 9, 10, 11]
[12, 13, 14, 15, 16, 17]
[18, 19, 20, 21, 22, 23]
[24, 25]

The first one is more in line with the idiomatic way of pagination.

application:

    When mybatis uses foreach for batch insertion, because the default executor type is Simple, a new prepared statement will be created for each statement, that is, an PreparedStatementobject will be created. In our project, the batch insert method will be used continuously, and because MyBatis cannot use caching for the contained statements, the sql statement will be re-parsed every time the method is called. Moreover, PreparedStatement is very long and contains many placeholders, which is especially time-consuming for the mapping of placeholders and parameters.

        Therefore, it is more appropriate to use foreach to insert 20-50 rows at a time, and the time consumption is acceptable. At this time, the above batch data is used.

Summarize:

        Using skip and limit is suitable for batch processing of data.