flume three components
source collection
channel aggregation
sink output
The key is to write using the Flume profile
A configuration source
Channel configuration B
C Configuration sink
The above three components D string together
1. IP port to receive data
a1 agent name
r1 data source name
k1 sinks name
c1 channel name
# example.conf: A single-node Flume configuration
# Name the components on this agent
a1.sources = r1
a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind =hadoop000
a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
Start agent
flume-agent \
--name a1 \
--conf $FLUME_HOME/conf \
--conf-file $FLUME_HOME/conf/example.conf \
-Dflume.roog.logger=INFO,console
agent选型 ： exec source+ memory channel + logger skin
# Name the components on this agent
a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = exec a1.sources.r1.command=tail -f /home/hadoop/data/data.log a1.sources.r1.shell=/bin/sh -c # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1