Ambari 安装 HDP(Centos7,HDP-2.6.3.0-235,HDP-UTILS-1.1.0.21,ambari-2.6.0.0)

构建Ambari 安装HDP环境大致包含这几个步骤
(1) SSH免密登陆
(2) hosts配置（所有机器必须执行）
(3) 防火墙管理 （所有机器必须执行）
(4) 安装JDK （所有机器必须执行）
(5) 构建本地yum源(master 主机安装)
(6) 安装mysql (master 主机安装)
(7) 初始化Ambari和HDP库 (master 主机)
(8) 安装Ambari (master 主机)
(9) 配置Ambari 启动项
(10) 启动Ambari
(11) 安装HDP
http://blog.51cto.com/tryingstuff/2066561
https://blog.csdn.net/mawenwu1983/article/details/78983275
https://blog.csdn.net/qq_31246691/article/details/79005281
https://www.cnblogs.com/gispathfinder/p/9060604.html
https://my.oschina.net/ibobby/blog/1836716

centos1：192.168.1.4    2G 2核 60G
centos2：192.168.1.5     1G 1核 60G
centos3：192.168.1.6    1G 1核 60G

(1) SSH免密登陆

hadoop.001
[root@localhost etc]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:fquK7a5XfTlDvIv3DMsHITcM1KumYf6jTnk33vqJE84 [email protected]
The key's randomart image is:
+---[RSA 2048]----+
|         ...     |
|          . .    |
|           + .   |
|          . O    |
|        S. = =   |
|       .+.+ O    |
|       +++o=+*   |
|     o..ooo=EB.. |
|    o==o++oo***  |
+----[SHA256]-----+
[root@localhost etc]# cd ~/.ssh

[root@localhost etc]# cat id_rsa.pub >>authorized_keys
[root@localhost etc]# chmod 600 ~/.ssh
[root@localhost etc]# chmod 600 ~/.ssh/authorized_keys

先在从节点登录root执行命令
hadoop.002
[root@SY-002 ~]# mkdir ~/.ssh/
hadoop.003
[root@SY-003 ~]# mkdir ~/.ssh/

[root@localhost etc]# scp /root/.ssh/authorized_keys [email protected]:/root/.ssh/authorized_keys
[root@localhost etc]# scp /root/.ssh/authorized_keys [email protected]:/root/.ssh/authorized_keys
//测试ssh面密码登陆
ssh [email protected]

(2) hosts配置（所有机器必须执行）

[root@localhost ~]# find -name hosts
[root@localhost /]# cd etc
[root@localhost etc]# vi hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.4 hadoop.001
192.168.1.5 hadoop.002
192.168.1.6 hadoop.003
//重启所有机器

(3) 防火墙管理 （所有机器必须执行）

//关闭防火墙 需要在所有机器上进行操作
[root@localhost etc]# systemctl disable firewalld
[root@localhost etc]# systemctl stop firewalld

(4) 安装JDK （所有机器必须执行）

//安装jdk
[root@SY-001 ~]# tar -zxvf jdk-8u181-linux-x64.tar.gz -C /opt/java/
[root@SY-001 ~]# vi /etc/profile
export JAVA_HOME=/opt/java/jdk1.8.0_181
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
source /etc/profile

(5) 构建本地yum源(master 主机安装)

//制作本地镜像源
[root@localhost etc]# yum install httpd
[root@localhost etc]# firewall-cmd --add-service=http 
[root@localhost etc]# firewall-cmd --permanent --add-service=http
添加 Apache 服务到系统层使其随系统自动启动

[root@localhost etc]# systemctl start httpd.service
[root@localhost etc]# systemctl enable httpd.service
    安装本地源制作相关工具
[root@localhost etc]# yum install yum-utils createrepo
    在httpd网站根目录,默认是即/var/www/html/，创建目录ambari， 
并且将下载的压缩包解压到/var/www/html/ambari目录
HDP-UTILS需要先建立一个HDP-UTILS文件夹并将HDP-UTILS-1.1.0.21-centos7.tar.gz放到HDP-UTILS目录下再解压

[root@localhost html]# cd ambari/
[root@localhost ambari]# ls
ambari-2.6.0.0-centos7.tar.gz  HDP-2.6.3.0-centos7-rpm.tar.gz  HDP-UTILS-1.1.0.21-centos7.tar.gz
[root@localhost ambari]# tar -zxvf ambari-2.6.0.0-centos7.tar.gz 
[root@localhost ambari]# mkdir HDP-UTILS
[root@localhost ambari]# ls
ambari  ambari-2.6.0.0-centos7.tar.gz  HDP-2.6.3.0-centos7-rpm.tar.gz  HDP-UTILS  HDP-UTILS-1.1.0.21-centos7.tar.gz
[root@localhost ambari]# mv HDP-UTILS-1.1.0.21-centos7.tar.gz ./HDP-UTILS
[root@localhost ambari]# tar -zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz
    首先下载上面资源列表中的相应repo文件，修改其中的URL为本地的地址，相关配置如下
将上面的修改过的源放到/etc/yum.repos.d/下面
[root@localhost yum.repos.d]# yum clean all
[root@localhost yum.repos.d]# yum list update
[root@localhost yum.repos.d]# yum makecache
[root@localhost yum.repos.d]# yum repolist

(6) 安装mysql (master 主机安装)

//安装mysql
Ambari安装会将安装等信息写入数据库，建议使用MariaDB数据库，也可以不安装而使用默认数据库PostgreSQL
[root@localhost ~]# yum install mariadb-server
[root@localhost ~]# systemctl start mariadb
[root@localhost ~]# systemctl enable mariadb
#数据库初始化设置
[root@SY-001 ~]# mysql_secure_installation
    #首先是设置密码，会提示先输入密码
    Enter current password for root (enter for none):<–初次运行直接回车
    #设置密码
    Set root password? [Y/n] <– 是否设置root用户密码，输入y并回车或直接回车
    New password: <– 设置root用户的密码
    Re-enter new password: <– 再输入一次你设置的密码
    #其他配置
    Remove anonymous users? [Y/n] <– 是否删除匿名用户，回车
    Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,
    Remove test database and access to it? [Y/n] <– 是否删除test数据库，回车
    Reload privilege tables now? [Y/n] <– 是否重新加载权限表，回车
    #初始化MariaDB完成，接下来测试登录，输入密码能正常登陆就完成了

(7) 初始化Ambari和HDP库 (master 主机)

[root@SY-001 ~]# mysql -uroot -p
        create database ambari character set utf8 ;  
        CREATE USER 'ambari'@'%'IDENTIFIED BY 'Ambari-123';
        GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
        FLUSH PRIVILEGES;
    
    create database hive character set utf8 ;  
    CREATE USER 'hive'@'%'IDENTIFIED BY 'Hive-123';
    GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
    FLUSH PRIVILEGES;
    
    
    create database oozie character set utf8 ;  
    CREATE USER 'oozie'@'%'IDENTIFIED BY 'Oozie-123';
    GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
    FLUSH PRIVILEGES;
    [root@SY-001 ~]# yum install mysql-connector-java

(8) 安装Ambari (master 主机)

//安装Ambari
[root@SY-001 ~]# yum install ambari-server

(9) 配置Ambari 启动项

[root@SY-001 ~]# ambari-server setup
下面是配置执行流程，按照提示操作

1.检查SELinux是否关闭，如果关闭不用操作

Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
2.提示是否自定义设置。输入：y

Customize user account for ambari-server daemon [y/n] (n)? y
3.ambari-server 账号。输入：ambari

Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
4.检查防火墙，如果关闭则不用操作

Checking firewall status...
Redirecting to /bin/systemctl status  iptables.service
5.设置JDK。输入：3

Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[] Custom JDK
==============================================================================
Enter choice (1): 3
6.如果上面选择3自定义JDK,则需要设置JAVA_HOME。输入：/opt/java/jdk1.8.0_152

WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/java/jdk1.8.0_152
Validating JDK on Ambari Server...done.
Completing setup...
7.数据库配置。选择：y

Configuring database...
Enter advanced database configuration [y/n] (n)? y
8.选择数据库类型。输入：3

Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
==============================================================================
Enter choice (3): 3
9.设置数据库的具体配置信息，根据实际情况输入，如果和括号内相同，则可以直接回车。

Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (Ambari-123): 
10.提示必须安装MySQL JDBC，回车结束ambari配置

WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
Press <enter> to continue.
11.将Ambari数据库脚本导入到数据库

如果使用自己定义的数据库，必须在启动Ambari服务之前导入Ambari的sql脚本

#用Ambari用户（上面设置的用户）登录mysql
mysql -u ambari -p
use ambari；
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

(10) 启动Ambari

 [root@SY-001 ~]# ambari-server start

(11) 安装HDP
参考:https://blog.csdn.net/mawenwu1983/article/details/78983275
问题：
本文总结在使用ambari时产生的异常，以及如何解决的。
如果发生了异常，在界面上不能直观的看出异常的原因，那么我一般通过查看日志的方法解决。
通过下面的命令查看

vim /var/log/ambari-server/ambari-server.log1


若该日志文件没有异常信息，可在其他日志文件里查找




1、异常一

异常发生在add host并confirm host的时候，会卡住不动，然后看日志会发现异常信息：



Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap1





1.1 具体的异常



七月 2018 14:29:08,339  INFO [ambari-client-thread-22654] BootStrapImpl:108 - BootStrapping hosts ambari.slave1.com:
09 七月 2018 14:29:08,340  INFO [Thread-4609] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/3
09 七月 2018 14:29:08,340  INFO [Thread-4609] BSRunner:372 - Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap
09 七月 2018 14:29:08,343 ERROR [Thread-4609] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/3/ambari.slave1.com.done (没有那个文件或目录)
09 七月 2018 14:29:08,344  WARN [Thread-4609] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/3/sshKey
09 七月 2018 14:40:28,033  INFO [ambari-client-thread-22676] BootStrapImpl:108 - BootStrapping hosts amabri.slave1.com:
09 七月 2018 14:40:28,034  INFO [Thread-4635] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/4
09 七月 2018 14:40:28,035  INFO [Thread-4635] BSRunner:372 - Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap
09 七月 2018 14:40:28,036 ERROR [Thread-4635] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/4/amabri.slave1.com.done (没有那个文件或目录)
09 七月 2018 14:40:28,037  WARN [Thread-4635] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/4/sshKey
09 七月 2018 14:44:02,863  INFO [ambari-client-thread-22659] BootStrapImpl:108 - BootStrapping hosts ambari.slave1.com:
123456789101112



1.2 解决办法

通过查看，/var/run/ambari-server目录下确实没有bootstrap文件夹，手动添加即可



mkdir bootstrap1

但是，如果只创建该文件夹的话，ambari依然报错，大概是没有权限相关的，就不贴出详细的异常了

为ambari用户添加权限即可



chown -R ambari:ambari /var/run/ambari-server/bootstrap/

sudo chown -R ambari /var/run/ambari-server

问题2

 ambari注册主机的时候，ambari-agent出现如下错误：
    NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:579)
    NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions.
    解决方案 在ambari-agent的配置文件ambari-agent.ini 在 ［security］增加一项
    
    [security]
    force_https_protocol=PROTOCOL_TLSv1_2

所有的机器都必须执行