Recently at work, we encountered a requirement. Party A requested to directly export all the work order information in use in a business module from the database. To achieve this goal, I wrote a SQL query and asked the DBA to assist in exporting the data. Although the number of work orders is not large, only more than 3,000, each work order contains a large amount of information. The DBA performed multiple export operations. Unfortunately, each export attempt resulted in the operating platform becoming stuck and unresponsive.
In order to overcome this problem, we decided to use MySQL's paging technology, specifically using the LIMIT and OFFSET keywords to split the export operation into multiple batches. Usually, we use some open source plug-ins such as pagehelper to implement page paging in our projects, and rarely write paging logic in SQL ourselves. But in this requirement, we had to understand and use the paging function of MySQL in depth.
LIMITIn this article, we will explore in detail the and keywords in MySQL OFFSETand how to handle paginated queries through performance optimization to meet business needs.
What is pagination query?
Paginated queries are a technique for splitting large data sets into manageable chunks for page-by-page display in a user interface. This is very common in web applications, mobile applications, and report generation, and it helps improve performance and improves user experience since the entire data does not need to be loaded at once.
Pagination keywords
LIMITkeywords
LIMITKeywords are used to limit the number of rows returned in the result set. Its basic syntax is as follows:
SELECT * FROM 表名 LIMIT 行数;
For example, to select information about the 10 most recently created tickets from a table named mark_info, you can execute the following query:
SELECT * FROM mark_info ORDER BY CREATE_TIME DESC LIMIT 10;
OFFSETkeywords
OFFSETKeywords are used to specify which row in the result set to start returning data from. Usually, it is LIMITused together with to achieve pagination effect. Its syntax is as follows:
SELECT * FROM 表名 LIMIT 行数 OFFSET 偏移量;
or
SELECT * FROM 表名 LIMIT 行数 , 偏移量;
The effect of these two writing methods is the same.
For example, to select the information of the latest 11th to 20th work orders created from the mark_info table, you can execute the following query
SELECT * FROM mark_info ORDER BY CREATE_TIME DESC LIMIT 10 OFFSET 10;
or
SELECT * FROM mark_info ORDER BY CREATE_TIME DESC LIMIT 10 , 10;
Example of paginated query
Suppose we have a bus_work_order_operate_infotable named which stores a large number of work order operation records. We hope to implement a paging function to display information about 10 work order operations on each page. Here is an example of how to perform a paginated query:
-- 第一页，显示最新的10个工单操作信息
SELECT * FROM bus_work_order_operate_info ORDER BY OPERATE_TIME DESC LIMIT 10;
-- 第二页，显示接下来的10个工单操作信息
SELECT * FROM bus_work_order_operate_info ORDER BY OPERATE_TIME DESC LIMIT 10 OFFSET 10;
-- 第三页，显示接下来的10个工单操作信息
SELECT * FROM bus_work_order_operate_info ORDER BY OPERATE_TIME DESC LIMIT 10 OFFSET 20;
-- 以此类推...
Performance optimization
In many practical application scenarios, we usually add limit, offset, order by clause, and appropriate index, and the efficiency is usually good. But when the offset is very large, a large amount of data needs to be skipped, which will cause big performance problems. Here is an optimized example:
Not optimized sql
SELECT * FROM bus_work_order_operate_info ORDER BY OPERATE_TIME DESC LIMIT 10 OFFSET 5000;
query analysis
Optimized sql
select T1.* from bus_work_order_operate_info T1 INNER JOIN (select ID FROM bus_work_order_operate_info ORDER BY OPERATE_TIME DESC LIMIT 10 OFFSET 5000 ) T2 ON T1.ID = T2.ID;
query analysis
limitThrough query analysis and comparison, the optimized SQL scans fewer data rows and the query may be more efficient, so we can consider using this method to optimize large offsets.
Summarize
MySQL's paging queries are a common requirement for processing large data sets. Understanding LIMITthe OFFSETusage of keywords can help you implement paging functions effectively. At the same time, performance optimization is also the key to ensuring efficient query execution. Through proper configuration and combination with other optimization strategies, you can easily cope with the challenges of paging queries and provide a better user experience.