下载flume-ng
$ wget http://mirrors.hust.edu.cn/apache/flume/1.5.2/apache-flume-1.5.2-bin.tar.gz
解压,创建文件输出目录
创建flume的配置
$ vi conf/flume-conf.properties
agent1.sources = source1
agent1.sinks = sink1
agent1.channels = channel1
# Describe/configure source1
agent1.sources.source1.type = avro
agent1.sources.source1.bind = 10.111.131.68 # avro监听host
agent1.sources.source1.port = 44446 # avro监听port
agent1.sources.source1.channels = channel1
# Use a channel which buffers events in memory
agent1.channels.channel1.type = memory
agent1.channels.channel1.capacity = 1000
agent1.channels.channel1.transactionCapactiy = 100
# Bind the source and sink to the channel
# Describe sink1
agent1.sinks.sink1.type = file_roll # 文件输出
agent1.sinks.sink1.sink.directory = /home/hai/app-monitoring/flume/apache-flume-1.5.2-bin/flumed-logs # 文件输出目录
agent1.sinks.sink1.channel = channel1
|
拷贝
apache-flume-1.5.2-bin/tools/flume-ng-log4jappender-1.5.2-jar-with-dependencies.jar
到需要输出日志的应用的WEB-INF/lib下
修改log4j配置
扫描二维码关注公众号,回复:
554054 查看本文章
log4j.appender.flume=org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
log4j.appender.flume.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss}%5p %F %L %m%n
log4j.appender.flume.Hostname=10.111.131.68
log4j.appender.flume.Port=44446
log4j.rootCategory=DEBUG, console, flume
|
启动flume
bin/flume-ng agent -n agent1 -c conf -f conf/flume-conf.properties -Dflume.monitoring.type=http -Dflume.monitoring.port=34545
接下来就可以在日志输出目录查看到日志文件了
这个页面可以看到flume的计量信息
http://10.111.131.68:34545/metrics
$ ls flumed-logs/
1426141638517-1 1426141638517-14 1426141638517-19 1426141638517-23 1426141638517-28 1426141638517-32 1426141638517-37 1426141638517-7
1426141638517-10 1426141638517-15 1426141638517-2 1426141638517-24 1426141638517-29 1426141638517-33 1426141638517-38 1426141638517-8
1426141638517-11 1426141638517-16 1426141638517-20 1426141638517-25 1426141638517-3 1426141638517-34 1426141638517-4 1426141638517-9
|