1、创建flume监听的目录，如果不嫌创建，启动flume时会报错
2、配置flume
sources：目录
channels：内存
skin：hdfs
#定义agent名， source、channel、sink的名称 a4.sources = s1 a4.channels = c1 a4.sinks = s1 #具体定义source a4.sources.s1.type = spooldir a4.sources.s1.spoolDir = /root/logs #具体定义channel a4.channels.c1.type = memory a4.channels.c1.capacity = 10000 a4.channels.c1.transactionCapacity = 100 #定义拦截器，为消息添加时间戳 a4.sources.s1.interceptors = i1 a4.sources.s1.interceptors.i1.type = org.apache.flume.interceptor.TimestampInterceptor$Builder #具体定义sink a4.sinks.s1.type = hdfs a4.sinks.s1.hdfs.path = hdfs://ns1/flume/%Y%m%d a4.sinks.s1.hdfs.filePrefix = events- a4.sinks.s1.hdfs.fileType = DataStream #不按照条数生成文件 a4.sinks.s1.hdfs.rollCount = 0 #HDFS上的文件达到128M时生成一个文件 a4.sinks.s1.hdfs.rollSize = 134217728 #HDFS上的文件达到60秒生成一个文件 a4.sinks.s1.hdfs.rollInterval = 60 #组装source、channel、sink a4.sources.s1.channels = c1 a4.sinks.s1.channel = c1
3、
copy hadoop-common-x.x.x.jar、commons-configuration-x.x.jar、hadoop-auth-x.x.x.jar、hadoop-hdfs-x.x.x.jar到flume/lib下，flume把数据写如到hdfs时需要使用hadoop API
copy core-site.xml、hdfs-site.xml到flume/conf，flume需要知道hadoop的具体配置
4、启动flume，配置文件名称为a4.conf
bin/flume-ng agent -n a4 -c conf -f conf/a4.conf -Dflume.root.logger=INFO,console
此时只有文件被放入/root/logs就会被flume收集到，上传到hdfs