Previously, paging query was connected to ES. As the system usage time and data volume continue to increase, the following error will be reported. The key information is that Result window is too large, from + size must be less than or equal to: [10000] but was [100001]
When the index is very, very large (tens of millions or billions), it is impossible to do deep paging according to from + size , because the deeper the paging, the easier it is to OOM. Even if there is no OOM, it consumes a lot of CPU and memory resources. In the later version 2.x, the official has added the limit index.max_result_window:10000 as a protection measure, that is, the default from + size cannot exceed 10,000. (from indicates which row to start from, size indicates how many pieces of data to query, from defaults to 0, and size defaults to 10)
solution:
Option 1: Limit the query to 10,000 items at the interface level, and the business end will prompt the user to continue filtering data.
Integer queryTotal = pageQuery.getPageIndex() * pageQuery.getPageSize();
if (queryTotal > 10000) {
throw new BusinessException(String.format("只能查询前[%s]条数据, 建议缩小查询范围", 10000));
}
Option 2: Use other deep paging strategies: scroll or scroll after [The disadvantage is that you cannot turn pages up and down at will, choose according to the scenario]
Option 3: Set when creating the index [not recommended]
"settings":{
"index":{
"max_result_window":1000000
}
}