storm中如何使用BaseWindowedBolt的ack机制

版权声明:本文为博主原创文章,转载请注明出处。 https://blog.csdn.net/liuchuanhong1/article/details/79526067

最近有个需求,在WindowedBole中需要对每个tuple进行ack,如果一场则进行fail,如是,我按照普通Bolt的写法,却始终实现不了上面的功能,代码如下:

List<Tuple> tupleList = inputWindow.get();
        try {
            if (tupleList.isEmpty()) {
                return;
            }
             // batch save
            service.saveDetails(details);
            
            for (Tuple input : tupleList) {
                 //ack
                collector.ack(input);
            }
            
        } catch (Exception ex) {
            log.error("", ex);
            // fail
            for (Tuple input : tupleList) {
            	collector.fail(input);
            }
        }

通过测试发现每一个tuplewindow只会ack一次,具体的说就是,比如一个window窗口有100条数据,按照上面的ack方法,只会ack一条,而不会将100条数据都ack,通过查看官网,发现TupleWindow的ack还真有点不一样的地方,官网文档如下:

Guarentees

The windowing functionality in storm core currently provides at-least once guarentee. The values emitted from the bolts execute(TupleWindow inputWindow) method are automatically anchored to all the tuples in the inputWindow. The downstream bolts are expected to ack the received tuple (i.e the tuple emitted from the windowed bolt) to complete the tuple tree. If not the tuples will be replayed and the windowing computation will be re-evaluated.

也就是说,storm会自动的ack TupleWindow里面的所有tuples,如果想形成一个tuple树,需要在下游的bolt里面调用ack方法,如是新建了一个普通的Bolt,用来过滤和ack WindowBolt里面的tuple:

public void execute(Tuple input) {
        try {
        	this.collector.ack(input);
            filter();
             } catch (Exception ex) {
            log.error("", ex);
        }
    }
经过测试,发现,TupleWindow里面的每个tuple都会被ack了。



猜你喜欢

转载自blog.csdn.net/liuchuanhong1/article/details/79526067