Installation node requirements:
- jdk1.8 more
- Enough memory space
- Adequate disk space
- Folder has read and write permissions
- hadoop
- zookeeper
1. Upload the installation package
put c:/apache-flume-1.8.0-bin.tar.gz
2. Extract
tar -xzvf apache-flume-1.8.0-bin.tar.gz -C /home/hadoop/apps
3. Modify the configuration file
cd /home/hadoop/apps/apache-flume-1.8.0-bin/conf
Modify the template name
mv flume-env.sh.template flume-env.sh
Edit
vi flume-env.sh
flume generally do not configure the environment variables, thus the installation is complete
4. Remote transmission flume
The use of general use to multiple machines
scp -r apache-flume-1.8.0-bin hadoop02:/home/hadoop/apps/
scp -r apache-flume-1.8.0-bin hadoop03:/home/hadoop/apps/
5. Use simple
Select the file storage path (can be arbitrarily set)
cd /home/hadoop/apps/apache-flume-1.8.0-bin/conf
Create a direct
vi example.conf
Add the following
# a1 当前这个agent的名字
# a1 的 source 别名
a1.sources = r1
# a1 的sink 别名
a1.sinks = k1
# a1 的通道 别名
a1.channels = c1
# 指定source 来自于端口的
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44455
# 指定channel
a1.channels.c1.type = memory
# 指定sink的类型 logger 控制台打印
a1.sinks.k1.type = logger
# 绑定 channel sink source
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
Start flume, to ensure that no error
../bin/flume-ng agent --conf conf --conf-file /home/hadoop/apps/apache-flume-1.8.0-bin/conf/example.conf --name a1 -Dflume.root.logger=INFO,console
Then we open a window in the current virtual machine
Installation for simulating message about telnet
sudo yum install -y telnet
44455 window above our connection set up after the installation is complete (if flume does not start properly, telnet will report the error connection refused)
telnet localhost 44455
Direct input message transmitted by the transport
hello world
Another window to see the received message
When closed only need to use ctrl + c terminate directly on it, and then kill -9 to kill the Application Process