Following the last time Flume read data to Oracle in real time. https://blog.csdn.net/sileiH/article/details/83023404
1. Preparation of construction materials
-
jdk 1.8
-
flume1.8
-
Two servers
2. Install jdk1.8, then decompress flume1.8, rename it to flume and put it in /app/
3. Edit the configuration file.
# 服务器A(192.168.3.129)
a1.sources = r2
a1.sinks = k2
a1.channels = c2
#################各Server运行错误日志###########################
# 配置读取的日志
a1.sources.r2.type = exec
a1.sources.r2.command = tail -F /app/flume/test/node_err_info
# Describe the sink
a1.sinks.k2.type = avro
a1.sinks.k2.hostname = 192.168.3.129
a1.sinks.k2.port = 9011
# Use a channel which buffers events in memory
a1.channels.c2.type = memory
# Bind the source and sink to the channel
a1.sources.r2.channels = c2
a1.sinks.k2.channel = c2
# 服务器B(192.168.3.130)
a1.sources = r2
a1.sinks = k2
a1.channels = c2
#################各Server运行错误日志###########################
# 配置读取的日志
a1.sources.r2.type = avro
a1.sources.r2.bind= 192.168.3.130
a1.sources.r2.port = 9011
# Describe the sink
a1.sinks.k2.type = logger
# Use a channel which buffers events in memory
a1.channels.c2.type = memory
# Bind the source and sink to the channel
a1.sources.r2.channels = c2
a1.sinks.k2.channel = c2
5. Execute flume.
bin/flume-ng agent --conf conf --conf-f conf/confA.properties -name a1 -Dflume.root.logger=INFO,console
bin/flume-ng agent --conf conf --conf-f conf/confB.properties -name a1 -Dflume.root.logger=INFO,console