这里以四台节点搭建
| IP | HostName | OS |
|---|---|---|
| 192.168.8.5 | h5（主） | CentOS 7.5 |
| 192.168.8.6 | h6（从） | CentOS 7.5 |
| 192.168.8.7 | h7（从） | CentOS 7.5 |
| 192.168.8.8 | h8（从） | CentOS 7.5 |
一、节点基本配置
1.yum 源，这里用阿里的（全部节点）
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo yum makecache
2.编辑 hosts 文件（全部节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/configure_network_names.html
vim /etc/hosts 192.168.8.5 h5 192.168.8.6 h6 192.168.8.7 h7 192.168.8.8 h8
3.关闭防火墙（全部节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/install_cdh_disable_iptables.html
systemctl disable firewalld
systemctl stop firewalld
4.关闭 SELinux（全部节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/install_cdh_disable_selinux.html
vim /etc/selinux/config SELINUX=disabled # sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config && setenforce 0
5.安装 NTP 服务（全部节点，其中一台为服务器端，其余为客户端）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/install_cdh_enable_ntp.html
这里以 h5 为 ntp 服务器，其它节点向 h5 同步时间。
具体搭建参考：https://www.cnblogs.com/jhxxb/p/10579816.html
6.安装 Python2.7（使用 HUE 的节点，不使用 HUE 不用安装）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/install_python_27.html
CentOS7 自带的就是 Python2.7，不用安装
yum install python27 source /opt/rh/python27/enable python --version
二、安装 Cloudera Manager 和 CDH
1.安装 CDH 的 yum 源（全部节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/configure_cm_repo.html
curl -o /etc/yum.repos.d/cloudera-manager.repo https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/cloudera-manager.repo rpm --import https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPM-GPG-KEY-cloudera
2.安装 JDK（全部节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_ig_jdk_installation.html
必须安装在 /usr/java/ 目录中，这里使用 CDH 源中带的 JDK
yum install oracle-j2sdk1.8
yum 下载安装可能会很慢，可以下载后本地安装
https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/
yum install /home/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm -y
3.安装 Cloudera Manager Server（全部节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/install_cm_server.html
主节点
yum install cloudera-manager-daemons cloudera-manager-server -y
从节点（在集群部署的时候会自动安装，但下载很慢，可以先提前本地安装）
yum install cloudera-manager-daemons cloudera-manager-agent -y
同样 yum 下载安装可能会很慢，可以下载后本地安装
4.安装数据库，这里选用 MySQL（主节点，也可以放在其它节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_mysql.html
MySQL 数据库安装参考：https://www.cnblogs.com/jhxxb/p/11156902.html
安装完成后先修改配置
vim /etc/my.cnf
my.cnf 文件
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock transaction-isolation = READ-COMMITTED # Disabling symbolic-links is recommended to prevent assorted security risks; # to do so, uncomment this line: symbolic-links = 0 key_buffer_size = 32M max_allowed_packet = 32M thread_stack = 256K thread_cache_size = 64 query_cache_limit = 8M query_cache_size = 64M query_cache_type = 1 max_connections = 550 #expire_logs_days = 10 #max_binlog_size = 100M #log_bin should be on a disk with enough free space. #Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your #system and chown the specified folder to the mysql user. log_bin=/var/lib/mysql/mysql_binary_log #In later versions of MySQL, if you enable the binary log and do not set #a server_id, MySQL will not start. The server_id must be unique within #the replicating group. server_id=1 binlog_format = mixed read_buffer_size = 2M read_rnd_buffer_size = 16M sort_buffer_size = 8M join_buffer_size = 8M # InnoDB settings innodb_file_per_table = 1 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 64M innodb_buffer_pool_size = 4G innodb_thread_concurrency = 8 innodb_flush_method = O_DIRECT innodb_log_file_size = 512M [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid sql_mode=STRICT_ALL_TABLES
然后再启动，新建 CDH 所需要的库：https://www.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_mysql.html#concept_dsg_3mq_bl
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; -- 也可以直接用 root 用户 GRANT ALL ON scm.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON amon.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON rman.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON hue.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON metastore.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON sentry.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON nav.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON navms.* TO 'cdh6'@'%' IDENTIFIED BY '123456'; GRANT ALL ON oozie.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
最后安装 MySQL 驱动
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_mysql.html#cmig_topic_5_5_3
yum install wget -y # 下载 wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz # 解压 tar -zxf mysql-connector-java-5.1.46.tar.gz # 放到指定目录 mkdir -p /usr/share/java/ cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar # 删除 rm -rf mysql-connector-java-5.1.46 mysql-connector-java-5.1.46.tar.gz
5.设置 Cloudera Manager 数据库（主节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/prepare_cm_database.html
格式：/opt/cloudera/cm/schema/scm_prepare_database.sh <databaseType> <databaseName> <databaseUser>
当 MySQL 或 MariaDB 与 Cloudera Manager Server 位于同一节点时：/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
当 MySQL 或 MariaDB 在其它节点上时：/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h db01.example.com --scm-host cm01.example.com scm scm
6.安装 CDH 和其他软件（主节点）
https://www.cloudera.com/documentation/enterprise/6/6.3/topics/install_software_cm_wizard.html
systemctl start cloudera-scm-server tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log # 看到如下日志就可以访问 WEB 端了：http://<server_host>:7180，用户名和密码默认都是 admin INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
子节点启动
systemctl start cloudera-scm-agent
http://wangshubing.com/2019/01/24/CentOS7.5,CDH6%E5%AE%89%E8%A3%85%E9%83%A8%E7%BD%B2/