FusionInsight HD针对不同服务提供了Shell脚本，供开发维护人员在不同场景下登录其对应的服务维护客户端完成对应的维护任务。
1、前提条件
2、获取客户端软件包
登录FusionInsight Manager系统，单击“服务管理”，在菜单栏中单击“下载客户端”，弹出“客户端类型”信息提示框。“客户端类型”勾选“完整客户端”。
3、安装客户端软件
将客户端软件包上传至客户端服务器，解压软件包：
[root@hdp02 ~]# mkdir /opt/client
[root@hdp02 ~]# cd /opt/client
[root@hdp02 client]# tar -xf FusionInsight_Services_Client.tar
[root@hdp02 client]# sha256sum -c FusionInsight_Services_ClientConfig.tar.sha256
FusionInsight_Services_ClientConfig.tar: OK
[root@hdp02 client]# tar -xf FusionInsight_Services_ClientConfig.tar进入客户端目录，进行安装。这里将客户端安装至/opt/huawei/client目录：
[root@hdp02 client]# cd FusionInsight_Services_ClientConfig
[root@hdp02 client]# mkdir -p /opt/huawei/client
[root@hdp02 FusionInsight_Services_ClientConfig]# ./install.sh /opt/huawei/client/
[18-09-20 10:20:58]: Pre-install check begin...
[18-09-20 10:20:58]: Checking necessary files and directory.
[18-09-20 10:20:58]: Checking NTP service status.
[18-09-20 10:20:58]: Error: Network time protocol(NTP) not running. Please start NTP first.
[root@hdp02 FusionInsight_Services_ClientConfig]# ./install.sh /opt/huawei/client/
[18-09-20 10:31:50]: Pre-install check begin...
[18-09-20 10:31:50]: Error: "/opt/huawei/client/" is not empty.
[root@hdp02 FusionInsight_Services_ClientConfig]# ./install.sh /opt/huawei/client/
[18-09-20 10:32:11]: Pre-install check begin...
[18-09-20 10:32:11]: Checking necessary files and directory.
[18-09-20 10:32:11]: Checking NTP service status.
[18-09-20 10:32:11]: Checking "/etc/hosts" config.
[18-09-20 10:32:11]: Pre-install check is complete.
[18-09-20 10:32:11]: Precheck on components begin...
[18-09-20 10:32:11]: Precheck Loader begin ...
[18-09-20 10:32:11]: Checking java environment.
[18-09-20 10:32:11]: Precheck on components is complete.
[18-09-20 10:32:11]: Deploy "dest_hosts" begin ...
[18-09-20 10:32:11]: Warning: "hwd02" already exists in "/etc/hosts", it will be overwritten.
[18-09-20 10:32:11]: Warning: "hwd04" already exists in "/etc/hosts", it will be overwritten.
[18-09-20 10:32:11]: Warning: "hwd05" already exists in "/etc/hosts", it will be overwritten.
[18-09-20 10:32:11]: Warning: "hwd01" already exists in "/etc/hosts", it will be overwritten.
[18-09-20 10:32:11]: Warning: "hwd03" already exists in "/etc/hosts", it will be overwritten.
[18-09-20 10:32:11]: Deploy "dest_hosts" is complete.
[18-09-20 10:32:11]: Install public library begin ...
[18-09-20 10:32:11]: Install Fiber begin ...
[18-09-20 10:32:11]: Deploy Fiber client to /opt/huawei/client//Fiber
[18-09-20 10:32:11]: Create Fiber env file "/opt/huawei/client//Fiber/component_env".
[18-09-20 10:32:11]: Create Fiber conf file "/opt/huawei/client//Fiber/conf/fiber.xml"
[18-09-20 10:32:11]: Fiber installation is complete.
[18-09-20 10:32:11]: Public library installation is complete.
[18-09-20 10:32:11]: Install components client begin ...
[18-09-20 10:32:11]: Install HBase begin ...
[18-09-20 10:32:11]: Copy /opt/client/FusionInsight_Services_ClientConfig/HBase/FusionInsight-HBase-1.3.1.tar.gz to /opt/huawei/client//HBase.
/opt/client/FusionInsight_Services_ClientConfig/HBase
[18-09-20 10:32:11]: Copy HBase config files to "/opt/huawei/client//HBase/hbase/conf"
[18-09-20 10:32:11]: Create HBase env file "/opt/huawei/client//HBase/component_env".
[18-09-20 10:32:11]: Setup fiber conf started.
[18-09-20 10:32:12]: HBase installation is complete.
[18-09-20 10:32:12]: Install HDFS begin ...
[18-09-20 10:32:12]: Copy /opt/client/FusionInsight_Services_ClientConfig/HDFS/FusionInsight-Hadoop-2.7.2.tar.gz to /opt/huawei/client/HDFS.
/opt/client/FusionInsight_Services_ClientConfig/HDFS
[18-09-20 10:32:12]: Copy Hadoop config files to "/opt/huawei/client/HDFS/hadoop/etc/hadoop"
[18-09-20 10:32:12]: Create Hadoop env file "/opt/huawei/client/HDFS/component_env".
[18-09-20 10:32:12]: HDFS installation is complete.
[18-09-20 10:32:12]: Install Hive begin ...
[18-09-20 10:32:12]: Deploy Hive client directory to /opt/huawei/client//Hive.
/opt/client/FusionInsight_Services_ClientConfig/Hive
[18-09-20 10:32:12]: start to generate /opt/huawei/client//Hive/Beeline/conf/beeline_hive_jaas.conf for beeline...
[18-09-20 10:32:12]: generated /opt/huawei/client//Hive/Beeline/conf/beeline_hive_jaas.conf
[18-09-20 10:32:12]: Deploy HCatalog client to /opt/huawei/client//Hive
[18-09-20 10:32:13]: URI is 'jdbc:hive2://192.168.110.157:24002,192.168.110.158:24002,192.168.110.159:24002/\;serviceDiscoveryMode=zooKeeper\;zooKeeperNamespace=hiveserver2\;sasl.qop=auth-conf\;auth=KERBEROS\;principal=hive/[email protected]'.
[18-09-20 10:32:13]: Create Hive env file "/opt/huawei/client//Hive/component_env".
/opt/huawei/client/Hive/Beeline
[18-09-20 10:32:13]: Setup fiber conf started.
[18-09-20 10:32:13]: Hive installation is complete.
[18-09-20 10:32:13]: Install JDK begin ...
[18-09-20 10:32:13]: Decompress jdk.tar.gz to /opt/huawei/client//JDK.
/opt/client/FusionInsight_Services_ClientConfig/JDK
[18-09-20 10:32:19]: Create JRE env file "/opt/huawei/client//JDK/component_env".
[18-09-20 10:32:19]: JDK installation is complete.
[18-09-20 10:32:19]: Warning: /opt/client/FusionInsight_Services_ClientConfig/JDK/VERSION not exist.
[18-09-20 10:32:19]: Install KrbClient begin ...
[18-09-20 10:32:19]: Copy /opt/client/FusionInsight_Services_ClientConfig/KrbClient/FusionInsight-kerberos-1.15.2.tar.gz to /opt/huawei/client//KrbClient.
/opt/client/FusionInsight_Services_ClientConfig/KrbClient
[18-09-20 10:32:19]: Copy KRB config files to "/opt/huawei/client//KrbClient/kerberos/conf"
[18-09-20 10:32:19]: Copy security script files to "/opt/huawei/client//KrbClient/kerberos/bin"
[18-09-20 10:32:19]: Create KRB env file "/opt/huawei/client//KrbClient/component_env".
[18-09-20 10:32:19]: KrbClient installation is complete.
[18-09-20 10:32:19]: Install Loader begin ...
Install loader client successfully.
[18-09-20 10:32:19]: Loader installation is complete.
[18-09-20 10:32:19]: Install Oozie begin ...
/opt/client/FusionInsight_Services_ClientConfig/Oozie/install.sh: line 48: local: can only be used in a function
[18-09-20 10:32:19]: Copy /opt/client/FusionInsight_Services_ClientConfig/Oozie/install_files to /opt/huawei/client//Oozie.
/opt/client/FusionInsight_Services_ClientConfig/Oozie/install.sh: line 69: [: -eq: unary operator expected
[18-09-20 10:32:20]: Copy Oozie config files to "/opt/huawei/client//Oozie/oozie-client-4.2.0/conf"
[18-09-20 10:32:20]: Create Oozie env file "/opt/huawei/client//Oozie/component_env".
/opt/client/FusionInsight_Services_ClientConfig/Oozie/install.sh: line 109: [: -eq: unary operator expected
[18-09-20 10:32:20]: Oozie installation is complete.
[18-09-20 10:32:20]: Install Yarn begin ...
[18-09-20 10:32:20]: Copy Yarn config files to "/opt/huawei/client/Yarn/config"
[18-09-20 10:32:20]: Yarn installation is complete.
[18-09-20 10:32:20]: Install ZooKeeper begin ...
[18-09-20 10:32:20]: Copy /opt/client/FusionInsight_Services_ClientConfig/ZooKeeper/FusionInsight-Zookeeper-3.5.1.tar.gz to /opt/huawei/client//ZooKeeper.
/opt/client/FusionInsight_Services_ClientConfig/ZooKeeper
[18-09-20 10:32:20]: Create Zookeeper env file "/opt/huawei/client//ZooKeeper/component_env".
[18-09-20 10:32:20]: Copy zookeeper config files to /opt/huawei/client//ZooKeeper/zookeeper/conf
[18-09-20 10:32:20]: ZooKeeper installation is complete.
[18-09-20 10:32:20]: Components client installation is complete.4、验证安装
配置客户端环境变量：
[root@hdp02 ~]# source /opt/huawei/client/bigdata_env 设置kinit认证：
[root@hdp02 client]# kinit admin
Password for [email protected]: #输入admin用户登录密码（与登录集群的用户密码一致）
[root@hdp02 client]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [email protected]
Valid starting Expires Service principal
09/21/2018 09:43:33 09/22/2018 09:43:31 krbtgt/[email protected]验证hdfs：
[root@hdp02 client]# hdfs dfs -ls /
Found 12 items
drwxrwxrwx - hdfs hadoop 0 2018-09-19 13:36 /app-logs
drwxrwx--- - hive hive 0 2018-09-19 13:40 /apps
drwxr-xr-x - hdfs hadoop 0 2018-09-19 13:36 /datasets
drwxr-xr-x - hdfs hadoop 0 2018-09-19 13:36 /datastore
drwxr-x--- - flume hadoop 0 2018-09-19 13:36 /flume
drwx------ - hbase hadoop 0 2018-09-21 10:01 /hbase
drwxrwxrwx - mapred hadoop 0 2018-09-19 13:36 /mr-history
drwxrwxrwt - spark2x hadoop 0 2018-09-19 13:36 /spark2xJobHistory2x
drwxrwxrwt - spark hadoop 0 2018-09-19 13:36 /sparkJobHistory
drwx--x--x - admin supergroup 0 2018-09-19 13:36 /tenant
drwxrwxrwx - hdfs hadoop 0 2018-09-19 13:36 /tmp
drwxrwxrwx - hdfs hadoop 0 2018-09-19 13:39 /user5、批量配置客户端
这里的批量配置客户端是指将客户端软件安装在nfs服务器上，其他客户端通过挂载nfs共享到指定的位置，然后配置ntp服务和hosts文件即可。
5.1 NFS服务器安装客户端
- NTP配置
[root@onas FusionInsight_Services_ClientConfig]# vi /etc/ntp.conf server 192.168.120.140 key 1 prefer burst iburst minpoll 4 maxpoll 4 restrict -4 default kod notrap nomodify nopeer noquery restrict -6 default kod notrap nomodify nopeer noquery restrict 127.0.0.1 restrict ::1 tinker step 0 tinker panic 0 disable monitor disable kernel enable auth keys /etc/ntp/ntpkeys trustedkey 1 requestkey 1 controlkey 1 --启动ntp服务 [root@onas FusionInsight_Services_ClientConfig]# systemctl start ntpd [root@onas FusionInsight_Services_ClientConfig]# systemctl enable ntpd - HOSTS配置
[root@onas ~]# vi /etc/hosts 192.168.110.158 hwd02 192.168.110.160 hwd04 192.168.110.161 hwd05 192.168.110.157 hwd01 192.168.110.159 hwd03 192.168.120.140 casserver -
安装客户端
[root@onas FusionInsight_Services_ClientConfig]# ./install.sh /u02/huawei/client [18-09-21 11:41:48]: Pre-install check begin... [18-09-21 11:41:48]: Checking necessary files and directory. [18-09-21 11:41:48]: Checking NTP service status. [18-09-21 11:41:48]: Checking "/etc/hosts" config. <.......> [18-09-21 11:41:56]: Components client installation is complete. - 验证测试
[root@onas ~]# source /u02/huawei/client/bigdata_env
[root@onas ~]# kinit candon
Password for [email protected]:
[root@onas ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [email protected]
Valid starting Expires Service principal
09/21/2018 11:44:05 09/22/2018 11:44:01 krbtgt/[email protected]
[root@onas ~]# hdfs dfsadmin -report
Configured Capacity: 717607747591 (668.32 GB)
Present Capacity: 715191830197 (666.07 GB)
DFS Remaining: 713422566437 (664.43 GB)
DFS Used: 1769263760 (1.65 GB)
DFS Used%: 0.25%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 05.2 NFS客户端安装配置
- NTP服务配置
[root@hdp03 ~]# vi /etc/ntp.conf server 192.168.120.140 key 1 prefer burst iburst minpoll 4 maxpoll 4 restrict -4 default kod notrap nomodify nopeer noquery restrict -6 default kod notrap nomodify nopeer noquery restrict 127.0.0.1 restrict ::1 tinker step 0 tinker panic 0 disable monitor disable kernel enable auth keys /etc/ntp/ntpkeys trustedkey 1 requestkey 1 controlkey 1 --启动ntp服务 [root@hdp03 ~]# systemctl start ntpd [root@hdp03 ~]# systemctl enable ntpd - HOSTS配置
[root@hdp03 ~]# vi /etc/hosts 192.168.110.158 hwd02 192.168.110.160 hwd04 192.168.110.161 hwd05 192.168.110.157 hwd01 192.168.110.159 hwd03 192.168.120.140 casserver - 挂载NFS共享
[root@hdp03 ~]# mount onas:/u02 /u02 - 验证测试
[root@hdp03 ~]# kinit candon
Password for [email protected]:
[root@hdp03 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [email protected]
Valid starting Expires Service principal
09/21/2018 12:38:02 09/22/2018 12:37:59 krbtgt/[email protected]
[root@hdp03 ~]# beeline
Connecting to jdbc:hive2://192.168.110.157:24002,192.168.110.158:24002,192.168.110.159:24002/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;sasl.qop=auth-conf;auth=KERBEROS;principal=hive/[email protected]
Debug is true storeKey false useTicketCache true useKeyTab false doNotPrompt false ticketCache is null isInitiator true KeyTab is null refreshKrb5Config is false principal is null tryFirstPass is false useFirstPass is false storePass is false clearPass is false
Acquire TGT from Cache
Principal is [email protected]
Commit Succeeded
Connected to: Apache Hive (version 1.2.1)
Driver: Hive JDBC (version 1.2.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1 by Apache Hive
0: jdbc:hive2://192.168.110.157:21066/> show databases;
+----------------+--+
| database_name |
+----------------+--+
| default |
| hivedb |
+----------------+--+
2 rows selected (0.74 seconds)
0: jdbc:hive2://192.168.110.157:21066/>