Kafka Simple Consumer

最近想做一个基于kafka backoff 到一定时间前进行数据重建的功能,之前一直没用过simpleconsumer,翻阅代码看了下SimepleConsumer的几个接口,

getOffsetsBefore

其中用到这个方法PartitionOffsetRequestInfo(time,maxNumOffsets)

实际测试发现此方法返回 给定time之前生成的segment的文件名里的那个offset

static SimpleConsumer consumer = new SimpleConsumer("*.*.*.*", 9092, 1000000, 64*1024, "capillary");
public static void main(String[] args) {
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        Date date = new Date();
        try {
            date = sdf.parse("2017-06-16 14:00:00");
        } catch (ParseException e) {
            e.printStackTrace();
        }
        Long time = date.getTime();
        PartitionOffsetRequestInfo requestInfo = new PartitionOffsetRequestInfo(time,2);
        TopicAndPartition topicAndPartition = new TopicAndPartition("TEST_ALL_TOPIC",3);
        Map<TopicAndPartition, PartitionOffsetRequestInfo> requestInfoMap = new HashMap<TopicAndPartition, PartitionOffsetRequestInfo>();
        requestInfoMap.put(topicAndPartition,requestInfo);
        OffsetRequest request = new OffsetRequest(requestInfoMap, (short)0, "capillary");
        OffsetResponse response = consumer.getOffsetsBefore(request);
        long[] test_all_topics = response.offsets("TEST_ALL_TOPIC", 3);
        Map<TopicAndPartition, PartitionFetchInfo> requestFecthInfoMap = new HashMap<TopicAndPartition, PartitionFetchInfo>();
//        FetchRequest fetchRequest = new FetchRequest((short)0, 0, "capillary", 0, 0, 1,requestFecthInfoMap);
//        consumer.fetch();
        System.out.println( "timestamp:" + time + ", offset:" + test_all_topics[0]);
    }
 

猜你喜欢

转载自my.oschina.net/u/2433649/blog/968402