Cassandra 分页读取数据

文章目录

为什么要分页
方案选择

Token
JPA
PagingState

参考

为什么要分页

如果一个查询得到的记录数太大，一次性返回回来，那么效率非常低，并且很有可能造成内存溢出，使得整个应用都奔溃。所以，在数据量比较大的时候，分页还是非常有必要的。

方案选择

Token

Cassandra 提供了Token 函数来记录上次查询的最后一条数据，但是它需要多个primary key, 不适合我们项目。所以放弃之。

JPA

看了下JPA API, 有setMaxResult 和 setFirstResult，但是发下不work, 后来查了下，Cassandra对offset 还不support，所以这种方案不行。

       Query query;
        List<Person> personList;
        // Create and execute SELECT * query
        String cqlString = "Select p from Person p";
        // String cqlString = "Select p from Person p";
        query = entityManager.createQuery(cqlString).setMaxResults(100).setFirstResult(10);
        // This solution doesn't work
        for (int i =0; i<10; i++) {
            personList = query.getResultList();
            // Verify number of records and contents are correct
            Assert.assertEquals(100, personList.size());
            System.out.println("haofan for: " + personList.get(10).getPersonId());
        }

PagingState

Cassandra 2.0 has auto paging. Instead of using token function to create paging, it is now a built-in feature.
PagingState: the driver exposes a PagingState object that represents where we were in the result set when the last page was fetched.(可以记录上次最后一页的位置)

public String paging(Session session, String pageState) {
    Statement statement = new SimpleStatement("SELECT * FROM testkeyspace.\"PERSON\"");
    statement.setFetchSize(3);
    if (pageState != null) {
        statement.setPagingState( PagingState.fromString(pageState));
    }
    ResultSet rs = session.execute(statement);
    int remaining = rs.getAvailableWithoutFetching();
    System.out.println("remaining " + remaining);
    for (Row row : rs) {
        System.out.println("first" + row);
        if (--remaining == 0) {
            break;
        }
    }
    return rs.getExecutionInfo().getPagingState().toString();
}
How to use this function:
@Test
public void pagingTest() throws Exception {
    Cluster cluster = Cluster.builder().addContactPoint("localhost").withPort(9042).build();
    Session session = cluster.connect();

    String pageState = paging(session, null);

    paging(session, pageState);
}