Chapter 4 Flume monitoring of Ganglia
4.1 Ganglia installation and deployment
1) installation and service httpd php
[atguigu@hadoop102 flume]$ sudo yum -y install httpd php
2) Installation other dependencies
[atguigu@hadoop102 flume]$ sudo yum -y install rrdtool perl-rrdtool rrdtool-devel
[atguigu@hadoop102 flume]$ sudo yum -y install apr-devel
3) Installation ganglia
[atguigu@hadoop102 flume]$ sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[atguigu@hadoop102 flume]$ sudo yum -y install ganglia-gmetad
[atguigu@hadoop102 flume]$ sudo yum -y install ganglia-web
[atguigu@hadoop102 flume]$ sudo yum install -y ganglia-gmond
When installing encountered an error as follows:
Error:Cannot retrieve metalink for repository: epel. Please verify its path and try again
Solution:
will
- name=Extra Packages for Enterprise Linux 6 - $basearch
- # Baseurl = http: //download.fedoraproject.org/pub/epel/6/$basearch
change into:
- baseurl = http : //download.fedoraproject.org/pub/epel/6/$basearch
- #mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
4) modify the configuration file /etc/httpd/conf.d/ganglia.conf
[atguigu@hadoop102 flume]$ sudo vi /etc/httpd/conf.d/ganglia.conf
Modify red color configurations:
# Ganglia monitoring system php web frontend Alias /ganglia /usr/share/ganglia <Location /ganglia> Order deny,allow Deny from all Allow from all # Allow from 127.0.0.1 # Allow from ::1 # Allow from .example.com </Location>
5) modify the configuration file /etc/ganglia/gmetad.conf
[atguigu@hadoop102 flume]$ sudo vi /etc/ganglia/gmetad.conf
change into:
data_source "node01" 192.168.1.21
6) modify the configuration file /etc/ganglia/gmond.conf
[atguigu@hadoop102 flume]$ sudo vi /etc/ganglia/gmond.conf
change into:
cluster { name = "node01" owner = "unspecified" latlong = "unspecified" url = "unspecified" } udp_send_channel { #bind_hostname = yes # Highly recommended, soon to be default. # This option tells gmond to use a source address # that resolves to the machine's hostname. Without # this, the metrics may appear to come from any # interface and the DNS names associated with # those IPs will be used to create the RRDs. # mcast_join = 239.2.11.71 host = 192.168.174.121 port = 8649 ttl = 1 } udp_recv_channel { # mcast_join = 239.2.11.71 port = 8649 bind = 192.168.174.21 retry_bind = true # Size of the UDP buffer. If you are handling lots of metrics you really # should bump it up to e.g. 10MB or even higher. # buffer = 10485760 }
7) modify the configuration file / etc / selinux / config
[atguigu@hadoop102 flume]$ sudo vi /etc/selinux/config
change into:
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted
Screaming Tip:
SELinux must be restarted for this to take effect shut down, if at this time do not want to restart, can be temporary entry into force:
[atguigu@hadoop102 flume]$ sudo setenforce 0
5) Start ganglia
[atguigu@hadoop102 flume]$ sudo service httpd start
[atguigu@hadoop102 flume]$ sudo service gmetad start
[atguigu@hadoop102 flume]$ sudo service gmond start
6) Open a web browser page ganglia
http://192.168.174.21/ganglia
Screaming Tip
: If you have completed the above operations still appear insufficient permissions error, please modify var / lib right / / ganglia directory
limit:
[atguigu@hadoop102 flume]$ sudo chmod -R 777 /var/lib/ganglia
4.2 operating Flume Test Monitor
1) flume-env.sh in modification / opt / module / flume / conf directory configuration:
JAVA_OPTS="-Dflume.monitoring.type=ganglia
-Dflume.monitoring.hosts=192.168.174.21:8649
-Xms100m
-Xmx200m"
2) Start Flume task
[atguigu@hadoop102 flume]$ bin/flume-ng agent \
--conf conf/ \
--name a1 \
--conf-file job/flume-telnet-logger.conf \
-Dflume.root.logger==INFO,console \
-Dflume.monitoring.type=ganglia \
-Dflume.monitoring.hosts=192.168.1.102:8649
3) 发送数据观察 ganglia 监测图
[atguigu@hadoop102 flume]$ telnet localhost 44444
样式如图: