spring data jpa使用流式查询

思路

调用org.hibernate.query.Query.stream方法查询数据

代码样例

import static org.hibernate.annotations.QueryHints.READ_ONLY;
import static org.hibernate.jpa.QueryHints.HINT_FETCH_SIZE;
import org.hibernate.query.Query;

1. 使用HQL查询

Query<MyEntity> q = entityManager.createQuery(
	"select e from MyEntity e where field1 = :field1",
	MyEntity.class
)
.setParameter("field1", "test")
.setHint(HINT_FETCH_SIZE, Integer.MIN_VALUE) // 根据数据库实现设置，这里是MYSQL
.setHint(READ_ONLY, true)
.unwrap(Query.class);
Stream<MyEntity> stream= q.stream();
stream.close(); // 释放

2. 使用条件查询

CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<MyEntity> query = builder.createQuery(MyEntity.class);
Root<MyEntity> root = query.from(MyEntity.class);
query.where(builder .equal(root.get("field1"), "test"));
query.select(root);
Query<MyEntity> q = entityManager.createQuery(query)
	.setHint(HINT_FETCH_SIZE, Integer.MIN_VALUE)
	.setHint(READ_ONLY, true)
	.unwrap(Query.class);  
Stream<MyEntity> stream= q.stream();
stream.close(); // 释放

踩坑

1. 使用连接池，运行一段时间后报错

org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
…
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30001ms.

原因：stream.close()并不会释放数据库连接，导致数据库连接池用完
解决：在事务中执行查询（遍历过程需要保持连接），关闭流后提交事务并释放连接

2. 关闭stream前执行其他查询报错

Streaming result set com.mysql.cj.protocol.a.result.ResultsetRowsStreaming@b534a9f is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.

原因：如果需要在遍历stream的过程中使用其他查询，需要在新的连接中处理。
解决：使用propagation = Propagation.REQUIRES_NEW让遍历逻辑在新事务中执行即可。

直接使用jdbc

jdbcTemplate.query(con -> {
    
    
    String sql = "select * from t_my_entity where fd_field1 = ?";
    PreparedStatement preparedStatement = this.createPreparedStatement(con, sql, context);
    statement.setString(1, "test");
    statement.setFetchSize(Integer.MIN_VALUE);
    statement.setFetchDirection(ResultSet.FETCH_FORWARD);
    return preparedStatement;
}, resultSet -> {
    
    
    // 遍历处理结果
});