mysql limit performance problem

Take a look at a sql:

SELECT m_id ,is_tax_paid FROM merchandise WHERE m_id > 10 AND last_update_time < NOW() ORDER BY m_id LIMIT (pageNum-1) * pageSize,pageSize

 The m_id and last_update_time of the merchandise table are uniquely indexed. Of course, this is not a combined index.

At first glance, there is no problem with this sql, but after running online for a while, there is a serious performance problem. A single query takes about 3 seconds and is recorded as slow sql.

The reason is that the Merchandise table is too large, and there are more than 100 million rows of data online. When the number of pages is too many, the limit paging of mysql needs to retrieve too much data. Specifically, it depends on how the B+ tree index of mysql searches the data. .

 

Knowing the reason, the improved method only needs to paginate by id. Each time you query, specify the size of the id, and then limit, such as:

SELECT m_id ,is_tax_paid  FROM merchandise WHERE m_id > ? AND last_update_time < ? ORDER BY m_id LIMIT pageSize

In each loop, find the largest m_id from the found results and pass it to the next sql query. The m_id above is preferably a unique index, a primary key index, preferably.

long updateBeginMid = 19541094l;
		while (true) {
			List<MerchandiseIsTaxPaid> midIsTaxPaidPairs = vipGoodsDao.listMidByLastUpdateTime(lastMigrateUpdateTime,updateBeginMid, BATCH_SIZE);
			doSomething(midIsTaxPaidPairs);
			if (CollectionUtils.isEmpty(midIsTaxPaidPairs) || midIsTaxPaidPairs.size() < BATCH_SIZE) {
				break;
			} else {
				int size = midIsTaxPaidPairs.size();
				MerchandiseIsTaxPaid merchandiseIsTaxPaid = midIsTaxPaidPairs.get(size - 1);
				updateBeginMid = merchandiseIsTaxPaid.getMerchandiseNo(); // Each query starts from the largest update id
			}
		}

 

 

 You can also refer here:

http://database.51cto.com/art/201005/200395.htm