Ansible入门篇（七）：Hadoop自动化安装

Hadoop是一个由Apache基金会所开发的分布式系统基础架构，主要用于存储和计算。

Hadoop安装包下载

到Hadoop官网下载对应版本的安装包，本文以hadoop 2.7.5为例。

将下载好的hadoop-2.7.5.tar.gz上传到/opt/ansible/roles/files目录下，本文将所有tar安装包都进行了重命名操作，比如将hadoop-2.7.5.tar.gz解压后重命名为hadoop并打成hadoop.tar.gz包。

编写主机hosts

编辑/opt/ansible/hosts文件添加部署Zookeeper机器IP

[hadoop_servers]
172.18.18.120 namenode_active=true namenode_standby=false datanode=true
172.18.18.121 namenode_active=false namenode_standby=true datanode=true
172.18.18.122 namenode_active=false namenode_standby=false datanode=true

编写Hadoop配置文件

将hadoop-2.7.5.tar.gz解压后的配置文件hadoop-env.sh、core-site.xml、hdfs-site.xml及slaves修改成自己的配置并重命名为hadoop-env.sh.j2、core-site.xml.j2、hdfs-site.xml.j2及slaves.hadoop.j2，然后上传到/opt/ansible/roles/templates/ 目录下。由于配置文件较多，本文不进行内容展示。

编写参数配置main.yml

编辑/opt/ansible/roles/vars/main.yml文件，添加变量

BigdataDir: /opt/hzgc

AnsibleDir: /opt/ansible

编写执行文件hadoop.yml

编辑新建/opt/ansible/yml/hadoop.yml文件，添加内容如下：

- hosts: hadoop_servers
remote_user: root
roles:
- roles

tasks:
- name: install dependency package
yum: name={{ item }} state=present
with_items:
- openssh
- rsync
- name: delete {{BigdataDir}}/hadoop
file: path={{BigdataDir}}/hadoop/ state=absent
- name: copy and unzip hadoop
unarchive: src={{AnsibleDir}}/roles/files/hadoop.tar.gz dest={{BigdataDir}}
- name: create hadoop logs directory
file: dest={{BigdataDir}}/hadoop/logs mode=0775 state=directory
- name: install configuration file hadoop-env.sh.j2 for hadoop
template: src={{AnsibleDir}}/roles/templates/hadoop-env.sh.j2 dest={{BigdataDir}}/hadoop/etc/hadoop/hadoop-env.sh
- name: install configuration file core-site.xml.j2 for hadoop
template: src={{AnsibleDir}}/roles/templates/core-site.xml.j2 dest={{BigdataDir}}/hadoop/etc/hadoop/core-site.xml
- name: install configuration file hdfs-site.xml.j2 for hadoop
template: src={{AnsibleDir}}/roles/templates/hdfs-site.xml.j2 dest={{BigdataDir}}/hadoop/etc/hadoop/hdfs-site.xml
- name: install configuration file slaves.j2 for hadoop
template: src={{AnsibleDir}}/roles/templates/slaves.hadoop.j2 dest={{BigdataDir}}/hadoop/etc/hadoop/slaves
- name: change shell sbin file
file: dest={{BigdataDir}}/hadoop/sbin mode=0755 recurse=yes
- name: change shell bin file
file: dest={{BigdataDir}}/hadoop/bin mode=0755 recurse=yes
- name: start journalnode
shell: sh {{BigdataDir}}/hadoop/sbin/hadoop-daemon.sh start journalnode
become: true
become_method: su
become_user: root
when: datanode == "true"
- name: format active namenode hdfs
shell: sh {{BigdataDir}}/hadoop/bin/hdfs namenode -format -force
become: true
become_method: su
become_user: root
when: namenode_active == "true"
- name: start active namenode hdfs
shell: sh {{BigdataDir}}/hadoop/sbin/hadoop-daemon.sh start namenode
become: true
become_method: su
become_user: root
when: namenode_active == "true"
- name: format standby namenode hdfs
shell: sh {{BigdataDir}}/hadoop/bin/hdfs namenode -bootstrapStandby -force
become: true
become_method: su
become_user: root
when: namenode_standby == "true"
- name: stop active namenode hdfs
shell: sh {{BigdataDir}}/hadoop/sbin/hadoop-daemon.sh stop namenode
become: true
become_method: su
become_user: root
when: namenode_active == "true"
- name: format ZKFC
shell: sh {{BigdataDir}}/hadoop/bin/hdfs zkfc -formatZK -force
become: true
become_method: su
become_user: root
when: namenode_active == "true"
- name: stop journalnode
shell: sh {{BigdataDir}}/hadoop/sbin/hadoop-daemon.sh stop journalnode
become: true
become_method: su
become_user: root
when: datanode == "true"
- name: start namenode
shell: sh {{BigdataDir}}/hadoop/sbin/start-dfs.sh
become: true
become_method: su
become_user: root
when: namenode_active == "true"
tags:
- start hadoop

执行Hadoop安装部署及启动

ansible-playbook -i hosts yml/hadoop.yml

查看是否安装成功：jps

注：因为hadoop依赖zookeeper通信，所以在安装启动hadoop的过程中要保证zookeeper正常启动，zookeeper的自动化部署参考Zookeeper自动化安装

Ansible入门篇（七）：Hadoop自动化安装

猜你喜欢