flume1.5.0.1+hbase0.98.7+hadoop1.2.1遇到的奇怪问题。

如题:
hdfs:hadoop1.2.1搭建的
hbase:0.98.7版本
flume:1.5.0.1版本
flume拿到日志sink到hbase中去,遇到的问题是这样的:
往表中存入100条数据以后,就开始报错,是flume爆出来的:
2014-11-01 11:18:35,168 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:353)] Failed to commit transaction.Transaction rolled back.
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
	at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
	at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
	at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
	at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
	at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
	at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
	at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
	at java.lang.Thread.run(Thread.java:662)
2014-11-01 11:18:35,170 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:356)] Failed to commit transaction.Transaction rolled back.
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
	at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
	at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
	at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
	at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
	at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
	at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
	at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
	at java.lang.Thread.run(Thread.java:662)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
	at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
	at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
	at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
	at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
	at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
	at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
	at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
	at java.lang.Thread.run(Thread.java:662)

我翻阅了flume1.5.0.1和hbase0.98.7版本的源代码以后发现确实是由hbase的0.98.7版本的Increment没有setWriteToWAL方法,这种情况如何破?


找到了问题的替代解决方法:
之前的flume配置是:
agent0.sinks.log-sink0.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
替换成:
agent0.sinks.log-sink0.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
不知道是不是SimpleHbaseEventSerializer的bug(不确定),flume1.5.0.1的Simple***lizer在所引用的hbase版本中不存在的方法。不过这个类在实际使用中应该都会自己重写。
此处路略过了。

猜你喜欢

转载自qiwb.iteye.com/blog/2150946