CDH5.10启用Kerberos

前提

已经安装好了Cloudera Manager和CDH5.10.0

Kerberos server已经部署好了（服务搭建过程见 Kerberos服务部署），在CDH节点已经安装了kerberos client。

CDH添加Kerberos服务过程

创建Cloudera Manager Principal

在KDC server主机上，创建一个名为cloudera-scm的principal，并将其密码设为1234。执行命令：

~]# kadmin.local

Authenticating as principal root/[email protected] with password.
kadmin.local:  addprinc -pw cloudera-scm-1234 cloudera-scm/[email protected]
WARNING: no policy specified for cloudera-scm/[email protected]; defaulting to no policy
Principal "cloudera-scm/[email protected]" created.

通过CDH Wizard来启用Kerberos

在CM界面管理 -》安全 -》状态 -》启用kerberos

点击continue，进入下一页进行配置，要注意的是：这里的『Kerberos Encryption Types』必须跟KDC实际支持的加密类型匹配（即kdc.conf中的值）。
点击continue，进入下一页，这一页中可以不勾选『Manage krb5.conf through Cloudera Manager』。
点击continue，进入下一页，输入Cloudera Manager Principal（就我们之前创建的cloudera-scm/[email protected] ）的username和password。
点击continue，进入下一页，导入KDC Account Manager Credentials。
点击continue，进入下一页，restart cluster并且enable Kerberos。

大功告成！

看看现在KDC database中有哪些principals

# kadmin.local 
Authenticating as principal root/[email protected] with password.
kadmin.local:  listprincs 
HTTP/[email protected]
K/[email protected]
cloudera-scm/[email protected]
hbase/[email protected]
hbase/[email protected]
[email protected]
hdfs/[email protected]
[email protected]
hive/[email protected]
httpfs/[email protected]
hue/[email protected]
kadmin/[email protected]
kadmin/[email protected]
kafka/[email protected]
kafka_mirror_maker/[email protected]
krbtgt/[email protected]
mapred/[email protected]
oiteboy/[email protected]
oozie/[email protected]
sentry/[email protected]
solr/[email protected]
spark/[email protected]
yarn/[email protected]
zookeeper/[email protected]

创建HDFS超级用户

# kadmin.local
Authenticating as principal root/[email protected] with password.
kadmin.local: addprinc [email protected]
WARNING: no policy specified for [email protected]; defaulting to no policy
Enter password for principal "[email protected]":
Re-enter password for principal "[email protected]":

Principal "[email protected]" created.

确认Kerberized Hadoop Cluster可以正常使用

1、确认HDFS可以正常使用

hdfs/[email protected]是CM自动生成的，我们并不知道其密码，这可以通过生成keytab来进行验证。

生成hdfs的keytab文件

# kadmin.local
ktadd -norandkey -k /root/hdfs.keytab hdfs/[email protected]

验证keytab文件是否生效

]# klist -kt /root/hdfs.keytab 
Keytab name: FILE:/root/hdfs.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
  13 07/03/2018 10:08:10 hdfs/[email protected]
  13 07/03/2018 10:08:10 hdfs/[email protected]
  13 07/03/2018 10:08:10 hdfs/[email protected]

根据keytab获取KDC的ticket

# kinit -kt keytab/hdfs.keytab hdfs/[email protected]

查看ticket缓存

# klist -e
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hdfs/[email protected]

Valid starting       Expires              Service principal
07/06/2018 11:24:46  07/07/2018 11:24:46  krbtgt/[email protected]
	renew until 07/11/2018 11:24:46, Etype (skey, tkt): aes128-cts-hmac-sha1-96, aes128-cts-hmac-sha1-96

查看hdfs上的文件

]# hdfs dfs -ls /
Found 6 items
drwx------   - hbase hbase               0 2018-07-03 09:59 /hbase
drwxr-xr-x   - hdfs  supergroup          0 2018-07-04 14:57 /lts
drwxr-xr-x   - hdfs  supergroup          0 2018-07-04 15:25 /outer
drwxrwxr-x   - solr  solr                0 2018-07-03 14:19 /solr
drwxrwxrwt   - hdfs  supergroup          0 2018-07-03 13:57 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2018-07-03 11:42 /user

2、确认可以正常提交MapReduce job
获取了hdfs的证书后，提交一个PI程序，如果能正常提交并成功运行，则说明Kerberized Hadoop cluster在正常工作。

3、确认其他组件(Zookeeper/HBase/Hue/Oozie等)可以正常运行

常见问题

参考 Troubleshooting Authentication Issues

参考链接：

为CDH 5集群添加Kerberos支持；
CDH禁用kerberos；
+ Configuring Authentication in Clouera Manager
+ Understanding Kerberos
+ Instlling Kerberos
+ Troubleshooting Authentication Issues
+ Configuring YARN for Long-running Applications

前提