Flume写数据换行

在Flume写入HDFS的时候会自动换行问题, 项目要求写入HDFS中数据不需要换行, 查看源码及配置如下:
BodyTextEventSerializer.java
  @Override
  public void write(Event e) throws IOException {
    out.write(e.getBody());
    if (appendNewline) {
      out.write('\n');
    }
  }
  // for legacy reasons, by default, append a newline to each event written out
  private final String APPEND_NEWLINE = "appendNewline";
  private final boolean APPEND_NEWLINE_DFLT = false;

  private final OutputStream out;
  private final boolean appendNewline;

  private BodyTextEventSerializer(OutputStream out, Context ctx) {
    this.appendNewline = ctx.getBoolean(APPEND_NEWLINE, APPEND_NEWLINE_DFLT);
    this.out = out;
  }

http://flume.apache.org/FlumeUserGuide.html中查看此配置项为:
Body Text Serializer

Alias: text. This interceptor writes the body of the event to an output stream without any transformation or modification. The event headers are ignored. Configuration options are as follows:

Property Name	Default	Description
appendNewline	true	Whether a newline will be appended to each event at write time. The default of true assumes that events do not contain newlines, for legacy reasons.
Example for agent named a1:

a1.sinks = k1
a1.sinks.k1.type = file_roll
a1.sinks.k1.channel = c1
a1.sinks.k1.sink.directory = /var/log/flume
a1.sinks.k1.sink.serializer = text
a1.sinks.k1.sink.serializer.appendNewline = false

猜你喜欢

转载自houshangxiao.iteye.com/blog/2212097
今日推荐