CDH集群搭建（待完善）

其他 2019-03-31 14:11:06 阅读次数: 0
版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/answer100answer/article/details/88900308

文章目录

背景

以hadoop用户在hadoop1~4四台机器上安装了java、hadoop、zookeeper、hbase，随着安装组件的增加成本越大。故选用cloudera管理集群。

需求

不损坏原有hadoop用户的hadoop组件。
改动：

  1. 配置从/etc/profile移动到~/.bash_profile下。/etc/profie下保持初始状态的即可。
  2. 停掉hadoop、zk、hbase等进程。避免端口冲突。
    注意：上述4台机器均做改动。

说明：为了方便权限，以root用户安装CDH。

准备

文件

  • cloudera-manager-el6-cm5.8.0_x86_64.tar.gz

环境

  1. java环境。
[root@hadoop1 ~]# java -version
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
  1. ssh免密。
[root@hadoop1 ~]# ssh hadoop4
Last login: Sat Mar 30 12:32:16 2019 from 172.24.36.245
[root@hadoop4 ~]#
  1. mysql
[root@hadoop1 ~]# mysql -uroot -pwhb123456
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.7.25 MySQL Community Server (GPL)

Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

并授权所有用户登录

mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' identified by 'whb123456';
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql -uroot-pxxxx进入mysql命令行，创建以下数据库：

create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database Reports DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database Oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

安装

主节点：

tar -zxvf cloudera-manager-el6-cm5.8.0_x86_64.tar.gz -C /opt/

[root@hadoop1 opt]# chown -R root:root cloudera/
[root@hadoop1 opt]# chown -R root:root cm-5.8.0/

cp  mysql-connector-java-5.1.47-bin.jar /opt/cm-5.8.0/share/cmf/lib/

所有节点创建用户

[root@hadoop1 ~]# useradd --system --home=/opt/cloudera-manager/cm-5.8.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

新建scm用户并授权；

GRANT ALL PRIVILEGES ON *.* TO 'scm'@'%' IDENTIFIED BY 'Hobe1991!' WITH GRANT OPTION;

这里会报错没解决

JAVA_HOME=/tools/java-current
Verifying that we can write to /opt/cm-5.8.0/etc/cloudera-scm-server
Sat Mar 30 15:37:05 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2019-03-30 15:37:05,721 [main] ERROR com.cloudera.enterprise.dbutil.DbProvisioner  - Exception when creating/dropping database with user 'root' and jdbc url 'jdbc:mysql://localhost/?useUnicode=true&characterEncoding=UTF-8'
java.sql.SQLException: Your password does not satisfy the current policy requirements
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2491)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2449)
	at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
	at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
	at com.cloudera.enterprise.dbutil.DbProvisioner.executeSql(DbProvisioner.java:286)
	at com.cloudera.enterprise.dbutil.DbProvisioner.doMain(DbProvisioner.java:95)
	at com.cloudera.enterprise.dbutil.DbProvisioner.main(DbProvisioner.java:110)
2019-03-30 15:37:05,729 [main] ERROR com.cloudera.enterprise.dbutil.DbProvisioner  - Stack Trace:
java.sql.SQLException: Your password does not satisfy the current policy requirements
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2491)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2449)
	at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
	at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
	at com.cloudera.enterprise.dbutil.DbProvisioner.executeSql(DbProvisioner.java:286)
	at com.cloudera.enterprise.dbutil.DbProvisioner.doMain(DbProvisioner.java:95)
	at com.cloudera.enterprise.dbutil.DbProvisioner.main(DbProvisioner.java:110)
--> Error 1, ignoring (because force flag is set)
Creating SCM configuration file in /opt/cm-5.8.0/etc/cloudera-scm-server
Executing:  /tools/java-current/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.8.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.8.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Sat Mar 30 15:37:06 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2019-03-30 15:37:06,771 [main] INFO  com.cloudera.enterprise.dbutil.DbCommandExecutor  - Unable to login using supplied username/password.
2019-03-30 15:37:06,774 [main] ERROR com.cloudera.enterprise.dbutil.DbCommandExecutor  - Error when connecting to database.
java.sql.SQLException: Access denied for user 'scm'@'localhost' (using password: YES)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871)
	at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1714)
	at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1224)
	at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2199)
	at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2230)
	at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2025)
	at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:778)
	at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
	at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
	at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java:243)
	at com.cloudera.enterprise.dbutil.DbCommandExecutor.main(DbCommandExecutor.java:137)
2019-03-30 15:37:06,777 [main] ERROR com.cloudera.enterprise.dbutil.DbCommandExecutor  - Exiting with exit code 8
--> Error 8, ignoring (because force flag is set)
All done, your SCM database is configured correctly!

scp -r /opt/cm-5.8.0 hadoop3:/opt/

mkdir -p /opt/cloudera/parcel-repo
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo

猜你喜欢

转载自blog.csdn.net/answer100answer/article/details/88900308
今日推荐
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
GCC 14.1 发布
面壁智能发布 Eurux-8x22B 开源大模型 —— 堪称「理科状元」
开源日报 | 谷歌扶持鸿蒙上位；开源Rabbit R1；Docker加持的安卓手机；微软的焦虑和野心；海尔电器把开放平台关了
中国码农的“35岁魔咒”
蘭雅 CorelDRAW 插件 2024.5.1 国际劳动节版，免费下载
Arc Browser for Windows 1.0 正式 GA
90后程序员开发视频搬运软件、不到一年获利超 700 万，结局很刑！
周排行
Java自定义时间格式
同步整形电路
在开发中最最最常用的字符串的属性大集合
Linux 查看端口占用并杀掉
Java基础四：ArrayList
多线程之死锁就是这么简单
mysql 基础命令集
awk 命令详解
Centos6.3编译安装nginx+php步骤
OCR （Optical Character Recognition，光学字符识别）
每日归档
更多
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)
2024-05-03(19)
2024-05-02(0)
2024-05-01(4)
2024-04-30(1)
2024-04-29(40)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022