basic concept
Streaming query means that after a successful query, an iterator is returned instead of a set, and the application fetches one query result from the iterator each time. The benefit of streaming queries is that they can reduce memory usage.
Without streaming query, when we want to fetch 10 million records from the database but do not have enough memory, we have to perform paging query. The efficiency of paging query depends on the table design. If the design is not good, we will not be able to perform efficient query. Paging query. Therefore, streaming query is a must-have function of a database access framework.
During the process of streaming query, the database connection remains open, so it should be noted that after executing a streaming query, the database access framework is not responsible for closing the database connection. The application needs to close it after fetching the data.
MyBatis streaming query interface
MyBatis provides an interface class called org.apache.ibatis.cursor.Cursor for streaming query. This interface inherits java.io.Closeable and java.lang.Iterable Interface, it can be seen from this:
1. Cursor can be closed;
2. Cursor is traversable.
In addition, Cursor also provides three methods:
1, isOpen(): Used to determine whether the Cursor object is open before fetching data. Cursor can only retrieve data when it is open;
2, isConsumed(): Used to determine whether all query results have been fetched.
3, getCurrentIndex(): Return how many pieces of data have been obtained
Because Cursor implements the iterator interface, in actual use, getting data from Cursor is very simple:
cursor.forEach(rowObject -> {...});
But the process of building Cursor is not simple
Let's take a practical example. Here is a Mapper class:
@Mapper
public interface FooMapper {
@Select(select * from foo limit #{limit})
Cursor<Foo> scan(@Param(limit) int limit);
}
The method scan() is a very simple query. By specifying the return value of the Mapper method as the Cursor type, MyBatis will know that this query method is a streaming query.
Then we write a SpringMVC Controller method to call Mapper (irrelevant code has been omitted):
@GetMapping(foo/scan/0/{limit})
public void scanFoo0(@PathVariable(limit) int limit) throws Exception {
try (Cursor<Foo> cursor = fooMapper.scan(limit)) { // 1
cursor.forEach(foo -> {}); // 2
}
}
In the above code, fooMapper is @Autowired. Note: The scan method is called at point 1 to obtain the Cursor object and ensures that it can be closed finally; the data is fetched from the cursor at point 2.
The above code seems to have no problem, but when executing scanFoo0(), an error will be reported:
java.lang.IllegalStateException: A Cursor is already closed.
This is because we mentioned earlier that the database connection needs to be maintained during the process of fetching data, and the Mapper method usually closes the connection after execution, so the Cusor is also closed.
Therefore, the idea to solve this problem is not complicated, just keep the database connection open. We have at least three options available. Follow the public account Java Technology Stack to get answers to Mybatis and more interview questions.
Option 1: SqlSessionFactory
We can use SqlSessionFactory to manually open the database connection and modify the Controller method as follows:
@GetMapping(foo/scan/1/{limit})
public void scanFoo1(@PathVariable(limit) int limit) throws Exception {
try (
SqlSession sqlSession = sqlSessionFactory.openSession(); // 1
Cursor<Foo> cursor =
sqlSession.getMapper(FooMapper.class).scan(limit) // 2
) {
cursor.forEach(foo -> { });
}
}
In the above code, in the first place we open a SqlSession (actually also represents a database connection) and ensure that it can be closed eventually; in the second place we use the SqlSession to obtain the Mapper object. This ensures that the Cursor object obtained is open.
Option 2: TransactionTemplate
In Spring, we can use TransactionTemplate to execute a database transaction, during which the database connection is also open. code show as below:
@GetMapping(foo/scan/2/{limit})
public void scanFoo2(@PathVariable(limit) int limit) throws Exception {
TransactionTemplate transactionTemplate =
new TransactionTemplate(transactionManager); // 1
transactionTemplate.execute(status -> { // 2
try (Cursor<Foo> cursor = fooMapper.scan(limit)) {
cursor.forEach(foo -> { });
} catch (IOException e) {
e.printStackTrace();
}
return null;
});
}
In the above code, we created a TransactionTemplate object at point 1 (no need to explain how transactionManager came here. This article assumes that readers are familiar with the use of Spring database transactions), executed database transactions at point 2, and the content of the database transaction It is a streaming query that calls the Mapper object. Note that the Mapper object here does not need to be created through SqlSession.
Option 3: @Transactional annotation
This is essentially the same as option 2, the code is as follows:
@GetMapping(foo/scan/3/{limit})
@Transactional
public void scanFoo3(@PathVariable(limit) int limit) throws Exception {
try (Cursor<Foo> cursor = fooMapper.scan(limit)) {
cursor.forEach(foo -> { });
}
}
It just adds a @Transactional annotation to the original method. This solution looks the simplest,but please note the pitfalls of using annotations in the Spring framework:only when called externally Valid. Calling this method in the current class will still report an error.
The above are three methods to implement MyBatis streaming query.