Zero-based configuration Hadoop cluster installation and configuration in detail --Ambari

1. Preparations

1.1. System Environment

Host List:

IP addresses HostName description
192.168.610.153 ambari.server Ambari server
192.168.10.152  hadoop.namenode Hadoop NameNode server
192.168.10.151 hadoop.datanodeone Hadoop DataNode server
192.168.10.147  hadoop.datanodetwo Hadoop DataNode server

operating system:

  CentOS 7 Minimal Edition

Login User

  root

Environment Description:

  As used herein, the installation is yum-line installation, all machines need can access the Internet.

  CentOS clean version of the initial state of the card is disabled, you need to be performed after the system is installed as follows:

  1) into the / etc / sysconfig / network-scripts / directory, locate the file ifcfg-e ...... names such as ifcfg-ens33

  2) vi editor to change the file ONBOOT set to yes, and increase NM_CONTROLLED, DNS entries, the specific code as follows:

we / etc / sysconfig / network-scripts / ifcfg-ens33 

ONBOOT = yes
NM_CONTROLLED = no
DNS1 = 8.8.8.8    
DNS2 = 4.2.2.2  

  3) Save and exit, restart the network connection

service network restart

  4) ip addr command to view the local IP address, or use the command ping www.baidu.com see if you can access the Internet.

  5) Once configured, you can use the SSH tool to log this virtual machine operating from another computer, it is recommended to use XShell

1.2. Modify the machine name, configure the Hosts file

   Hostname modified for each host name reference table 1.1, as ambari.server

hostnamectl set-hostname ambari.server

  Edit each machine / etc / hosts file, add the following information at the end of the original file

192,168 . 10.153     ambari.server
 192,168 . 10,152     hadoop.namenode
 192,168 . 10,151     hadoop.datanodeone
 192,168 . 10,147     hadoop.datanodetwo

  Use the reboot command to restart after editing each machine

1.3. Ntp service open

  For each installation, open the npt service, and is set to boot

  1) install ntp

yum install -y ntp

  2) Set ntp boot

systemctl enable ntpd

  3) Start ntp service

systemctl start ntpd

  4) All machine settings Time Synchronization

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
ntpdate us.pool.ntp.org

1.4. Close Selinux and firewall

  Do the following for each machine:

  1) vi / etc / sysconfig / selinux and add the following code

# vi /etc/sysconfig/selinux
selinux
=disalbed

  2) turn off the firewall

systemctl stop firewalld
systemctl disable firewalld

1.5. Configuring SSH-free dense Login

  1) performs the following operations to generate the key in ambari.server

ssh-keygen -t rsa

  All options are by the Enter key, until the key is generated.

  Key files are stored in ~ / .ssh directory

    Private: id_rsa
    Public key: id_rsa.pub
  2) create the ~ / .ssh directory on the other machines other than ambari.server
mkdir ~/.ssh

  3) Run the following commands on ambari.server, sending the key to another server

scp ~/.ssh/id_rsa.pub hadoop.namenode:/root/.ssh/authorized_keys_from_ambari
scp ~/.ssh/id_rsa.pub hadoop.datanodeone:/root/.ssh/authorized_keys_from_ambari
scp ~/.ssh/id_rsa.pub hadoop.datanodetwo:/root/.ssh/authorized_keys_from_ambari

  4) Run the following commands on another except ambari.server of combined public key:

cd ~/.ssh
cat authorized_keys_from_ambari >> authorized_keys

  5) Run the following commands on all machines

chmod 700 ~/.ssh
chmod 600 ~/.ssh/*

Install JDK 1.6

  Current Ambari recommended openjdk1.8.0, need to do the same on each machine with the following command:

  1) Install JDK

yum -y install java-1.8.0-openjdk java-1.8.0-openjdk-devel

  2) into the installation directory / usr / lib / jvm / find java 1.8.0-openjdk-1.8.0-. ...... $ JAVA_HOME folder and it is for this directory, such as java-1.8.0-openjdk-1.8 .0.212.b04-0.el7_6.x86_64

JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/

  3) Use java -vsersion command to verify whether the JDK installation configuration is successful

1.7 Setting the UTF8 character set

Execute instructions per machine:

cat << EOF > /etc/locale.conf
LANG="en_US.UTF-8"
SUPPORTED="zh_CN.UTF-8:zh_CN:zh:zh_CN.gb18030:en_US.UTF-8:en_US:en"
SYSFONT="lat0-sun16"
EOF

2. Configure Ambari local download station

  Only performed on the following machine ambari.server

2.1. Download the local library

2.1.1. Download ready

  Pure CentOS wget not installed in the need to install package wget

yum install -y wget

  Ambari version of this article corresponding to 2.7.3, if need other versions, please https://docs.hortonworks.com/ lookup

 

 

 

 

 

2.1.2. Download the local library

  This operation is time-consuming, you can run on the server

  Execute the following code, download the local library:

wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari-2.7.3.0-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.1.0/HDP-3.0.1.0-centos7-rpm.tar.gz
wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-centos7-gpl.tar.gz

2.2. 配置本地下载站

  1)安装httpd服务

yum install -y httpd

  2)创建站点目录,将本地库压缩包解压至对应目录

mkdir /var/www/html/ambari
mkdir /var/www/html/ambari/HDP-UTILS
mkdir /var/www/html/ambari/HDP-GPL

tar -xvf ambari-2.7.3.0-centos7.tar.gz -C /var/www/html/ambari/
tar -xvf HDP-3.0.1.0-centos7-rpm.tar.gz -C /var/www/html/ambari/
tar -xvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/ambari/HDP-UTILS
tar -xvf HDP-GPL-3.1.0.0-centos7-gpl.tar.gz -C /var/www/html/ambari/HDP-GPL

  3)启动httpd服务

systemctl start httpd

  4)打开浏览器,访问本地下载站http://192.168.10.153/ambari/,若出现以下界面则配置成功

2.3. 配置YUM源

  1)在ambari.server上执行:
yum install -y yum-plugin-priorities
  2)修改Yum安装源的校验
vi /etc/yum/pluginconf.d/priorities.conf

  设置为以下内容

[main]
enabled=1
gpgcheck=0

  3)配置repo文件

  该步骤中baseurl和gpgkey地址需要和上一步对应

cat << EOF > /etc/yum.repos.d/ambari.repo
#VERSION_NUMBER=2.7.3.0-139
[ambari-2.7.3.0]
name=ambari Version - ambari-2.7.3.0
baseurl=http://192.168.10.153/ambari/ambari/centos7/2.7.3.0-139/
gpgcheck=1
gpgkey=http://192.168.10.153/ambari/ambari/centos7/2.7.3.0-139/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
EOF

cat << EOF > /etc/yum.repos.d/hdp.repo
#VERSION_NUMBER=3.0.1.0-187
[HDP-3.0.1.0-187]
name=HDP Version - HDP-3.0.1.0-187
baseurl=http://192.168.10.153/ambari/HDP/centos7/3.0.1.0-187/
gpgcheck=1
gpgkey=http://192.168.10.153/ambari/HDP/centos7/3.0.1.0-187/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[HDP-UTILS-1.1.0.21]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.21
baseurl=http://192.168.10.153/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=1
gpgkey=http://192.168.10.153/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
EOF

  4)更新本地资源池

yum clean list
yum update list
yum makecache

  更新完成后查看资源池

yum repolist

  显示如下内容则配置成功

已加载插件:fastestmirror, priorities
Loading mirror speeds from cached hostfile
 * base: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.163.com
 * updates: mirrors.163.com
4 packages excluded due to repository priority protections
源标识                                                           源名称                                                                                状态
HDP-3.0.1.0-187                                                  HDP Version - HDP-3.0.1.0-187                                                              197
HDP-UTILS-1.1.0.21                                               HDP-UTILS Version - HDP-UTILS-1.1.0.21                                                      16
ambari-2.7.3.0                                                   ambari Version - ambari-2.7.3.0                                                             13
base/7/x86_64                                                    CentOS-7 - Base                                                                       10,015+4
extras/7/x86_64                                                  CentOS-7 - Extras                                                                          419
updates/7/x86_64                                                 CentOS-7 - Updates                                                                       2,137
repolist: 12,797

 

 3. 安装启动Ambari

   该项中的所有操作均在ambari.server上执行

3.1. 安装Ambari-Server

  本次安装采用YUM远程安装,数据库采用默认数据库。安装命令如下。

yum install -y ambari-server

3.2. 配置Ambari-Server

  配置命令如下:

ambari-server setup

  配置命令中需要的各个选项如下,注意其中jdk选择自定义,$JAVA_HOME变量与上文1.6中保持一致:

 

Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? 
Customize user account for ambari-server daemon [y/n] (n)? 
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? 
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? 
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database ... OK


About to start PostgreSQL
Configuring local database...
Configuring PostgreSQL...
Restarting PostgreSQL
Creating schema and user...
done.
Creating tables...
done.
Extracting system views...
ambari-admin-2.7.3.0.139.jar
....
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

3.3 启动Ambari-Server

ambari-server start

  启动成功后在浏览器中输入地址:

  http://192.168.141.159:8080/#/login

  若出现如下界面则配置成功

  默认登录名密码均为admin

 

 

Guess you like

Origin www.cnblogs.com/zklight/p/11097290.html