Flume的官网
http://flume.apache.org
Flume的部署
1、JDK的安装
2、安装包下载网址：apache-flume-1.5.0-bin.tar.gz
源码下载地址：apache-flume-1.5.0-src.tar.gz
3、将压缩包复制到系统中~/sw/目录下，
安装包解压：tar zxvf apache-flume-1.5.0-bin.tar.gz
源码解压： tar zxvf apache-flume-1.5.0-src.tar.gz
移动到~/md/ mv apache-flume-1.5.0-bin ../md/
移动到~/md/ mv apache-flume-1.5.0-src ../md/
将源码复制到安装bin目录下
4、设置环境变量
vi ~/.bash_profile
export FLUME_HOME =/home/a/md/apache-flume-1.5.0-bin
export PATH = $FLUME_HOME/bin:$PATH
source ~/.bash_profile
5、修改配置文件
~/md/apache-flume-1.5.0-bin 下查找flume-env.sh
6、验证flume是否安装完成
7、Flume的测试
~/md/apache-flume-1.5.0-bin/conf 创建agent1.conf
agent1.sources = source1
agent1.channels = channel1
agent1.sinks = sink1
# For each one of the sources, the type is defined
agent1.sources.source1.type = spooldir
agent1.sources.source1.spoolDir =/home/a/flume/aboutyunlog
# The channel can be defined as follows.
agent1.source1.source1.channels=channel1
agent1.source1.source1.fileHeader=false
# Each sink's type must be defined
agent1.sinks.sink1.type=hdfs
agent1.sinks.sink1.hdfs.path=hdfs://master:9000/flume/aboutyunlog
agent1.sinks.sink1.hdfs.fileType=DataStream
agent1.sinks.sink1.writeFormat=TEXT
agent1.sinks.sink1.hdfs.rollInterval=60
#Specify the channel the sink should use
agent1.sinks.sink1.channel=channel1
# Each channel's type is defined.
agent1.channels.channel1.type=file
agent1.channels.channel1.checkpointDir=/home/a/flume/aboutyun_check
agent1.channels.channel1.dataDirs=/home/a/flume/aboutyun_data
$FLUME_HOME下编写shell脚本文件
#!bin/bash
bin/flume-ng agent -n agent1 -c conf -f conf/agent1.conf -Dflume.root.logger=DEBUG,console >./agent1.log 2>&1 &
如果HDFS启动的话，在当前目录下
bash agent1.bash
jps查看进程是否有进程名称叫做Appcation的
如果想查看相关实时内容可以 Tail -f agent.log进行查看
cd ~/flume/aboutyunlong
echo fafsdfsafsa>> 1.txt
echo ffasfasfsaf>> 2.txt
ls 查看目录下是否包含1.txt.COMPELTED, 2.txt.COMPELTED.
再去HDFS上面查看数据
hdfs dfs -ls /flume/aboutyunlog
hdfs dfs -cat /flume/aboutyunlog/*
完结，暂时写的这里，有不对的地方请指正。