When the amount of platform users is low, we may not care about the lack of cluster security features, because there are few users, the team is small, it is relatively easy to control, and developers also know each other directly. At this time, only need to do a good job within the team or the enterprise through a series of administrative management methods to manage the cluster security issues. But do n’t forget that our platform is positioned as a single big data to support all applications within the enterprise. As the saying goes, a hundred people, all kinds of. When the number of platform users reaches a certain number, their quality will inevitably be uneven, and the big data platform is no longer a small team. At this time, it is difficult to rely on the team consciously or simply through the rules and regulations to play an effective role. As an enterprise-level platform, security issues should not be underestimated. For details, please refer to: Build an Enterprise Platform Security Solution .
1. Install Kerberos
1. Overall description
Software version
Operating system: CentOs 6.8
CDH version: Hadoop 2.6.0-cdh5.9.0
JDK version: jdk1.8.0_211
Running user: root
Cluster host role division
netcloud01 as the master node, install
other nodes of kerberos Server as slave nodes, install kerberos client
2. Configure host
Add the host name to the / etc / hosts file.
$ cat / etc / hosts
172.16.240.103 netcloud01
172.16.240.104 netcloud02
172.16.240.105 netcloud03
Note: Please use lowercase for hostname, otherwise some errors will occur when integrating kerberos.
3. Install Kerberos
Install krb5, krb5-server, and krb5-client on netcloud01.
> yum install krb5-server -y
# execute the following installation when commands such as klist cannot be found
> yum install -y krb5-server krb5-workstation pam_krb5
Install krb5-devel and krb5-workstation on other nodes (netcloud02, netcloud03)
> yum install krb5-devel krb5-workstation -y
4. Modify the configuration file
The kdc service involves three configuration files:
/etc/krb5.conf
/var/kerberos/krb5kdc/kdc.conf
/var/kerberos/krb5kdc/kadm5.acl
1) Edit the configuration file /etc/krb5.conf. The default installation file contains multiple sample items.
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = NETCLOUD.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
default_tgs_enctypes = aes256-cts
default_tkt_enctypes = aes256-cts
permitted_enctypes = aes256-cts
udp_preference_limit = 1
kdc_timeout = 3000
[realms]
NETCLOUD.COM = {
kdc = netcloud01
admin_server = netcloud01
}
[domain_realm]
.netcloud.com = NETCLOUD.COM
netcloud.com = NETCLOUD.COM
Description:
[logging]: indicates the printing position of the server-side logs
[libdefaults]: the default configuration of each connection, you need to pay attention to the following key small configurations
default_realm = 0HKJ.COM: set the default realm of the Kerberos application. If you have multiple realms, just add additional statements to the [realms] section.
ticket_lifetime: indicates the time limit for the certificate to take effect, generally 24 hours.
renew_lifetime: indicates the maximum time period that the voucher can be renewed, usually one week. After the credentials expire, subsequent access to the security-certified service will fail.
clockskew: The clock deviation is the tolerance of the time stamp of the ticket that does not fully meet the clock of the host system. Normally, set the clock skew to 300 seconds (5 minutes). This means that from the perspective of the server, the deviation between the time stamp of the ticket and it can be within 5 minutes before and after.
udp_preference_limit = 1: Prohibit the use of udp to prevent an error in Hadoop
[realms]: List the realm used.
kdc: represents the position of kdc. The format is machine: port
admin_server: represents the location of admin. The format is machine: port
default_domain: represents the default domain name
[appdefaults]: you can set some configuration for specific applications, override the default configuration.
2) Modify /var/kerberos/krb5kdc/kdc.conf, The file contains Kerberos configuration information. For example, the location of KDC, the realms of Kerbero's admin, etc. The configuration files on all Kerberos machines need to be synchronized. Only the basic configurations required are listed here. Detailed reference: krb5conf
[kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88
[realms]
NETCLOUD.COM = {
#master_key_type = aes256-cts
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
max_renewable_life = 7d
max_life = 1d
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
}
Description:
0HKJ.COM: Realms set. The name is random. Kerberos can support multiple realms, which will increase the complexity. It is case sensitive and generally uses all capitals for identification. This realms has nothing to do with the host of the machine.
master_key_type: and supported_enctypes use aes256-cts by default. JAVA uses aes256-cts authentication method requires the installation of JCE package, see the following instructions. For simplicity, you can not use the aes256-cts algorithm, so you don't need to install JCE.
acl_file: The user authority marked with admin needs to be created by the user. The file format is: Kerberos_principal permissions [target_principal] [restrictions]
supported_enctypes: supported verification methods.
admin_keytab: Keytab for verification by KDC.
About AES-256 encryption:
For systems using centos5.6 and above, AES-256 is used for encryption by default. This requires JCE to be installed on all nodes in the cluster. If you are using JDK1.6, go to the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JDK / JRE 6 page, if it is JDK1.7, go to Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JDK / JRE 7 download. The downloaded file is a zip package. After unpacking, the two files inside (local_policy.jar, US_export_policy.jar ) into the following directory: $ JAVA_HOME / jre / lib / security.
The above step must be done, otherwise it will report zk, namenode and other encryption methods that do not support the default tkt.
3) In order to be able to add and delete principals from the Kerberos database without directly accessing the KDC console, please indicate to the Kerberos management server which principals are allowed to perform what operations. This is done by creating /var/lib/kerberos/krb5kdc/kadm5.acl.
$ cat /var/kerberos/krb5kdc/kadm5.acl
reads as follows:
*/[email protected] *
If the second part of the principal's name is admin, then the principal has administrator rights
5. Synchronize configuration files
Copy /etc/krb5.conf in kdc to other servers in the cluster.
$ scp -r /etc/krb5.conf netcloud02: /etc/krb5.conf
$ scp -r /etc/krb5.conf netcloud03: /etc/krb5.conf to
build the database
run the initial database command on netcloud01. Where -r specifies the corresponding realm.
$ kdb5_util create -r NETCLOUD.COM -s
When loading random data appears, open another terminal to execute the CPU-consuming commands such as cat / dev / sda> / dev / urandom to speed up random number collection. This command will create the principal database in the / var / kerberos / krb5kdc / directory.
If you encounter a prompt that the database already exists, you can delete all the relevant files of the principal in the / var / kerberos / krb5kdc / directory. The default database name is principal. You can use -d to specify the database name.
7. Start the Kerberos service
Run on the netcloud01 node:
Chkconfig --level 35 krb5kdc ON $
$ chkconfig kadmin --level 35 ON
$ krb5kdc Service Start
$ Service Start kadmin
8, built kerberos administrator
about kerberos management, you can use kadmin.local or kadmin, as for the use of which, depending on the account And access rights:
If you have root access to the kdc server, but do not have a kerberos admin account, use kadmin.local.
If you do not have root access to the kdc server, but use the kerberos admin account, use kadmin to create a remote management administrator
on netcloud01 :
Enter the password twice manually
$ kadmin.local -q "addprinc root/admin"
Password: netcloud @ 123
can also be used without manually entering the password
$ echo -e "root \ nroot" | kadmin.local -q "addprinc root / admin"
extract the key and store it in the local keytab file /etc/krb5.keytab . This file is owned by the super user, so it must be root to execute the following commands in the kadmin shell:
kadmin.local -q "ktadd kadmin / admin"
# View the generated keytab
klist -k /etc/krb5.keytab
9. Common commands of kerberos
Enter kadmin
kadmin.local / kadmin to
create a database
kdb5_util create -r NETCLOUD.COM -s
start kdc service
service krb5kdc start
start kadmin service
service kadmin start
modify the current password
kpasswd
test keytab availability
kinit -k -t / var / kerberos / krb5kdc / keytab /root.keytab root/[email protected]
view keytab
klist -e -k -t /etc/krb5.keytab
clear cache
kdestroy login
through keytab file
kinit -kt / var / run / cloudera-scm-agent / process / ***-HIVESERVER2 / hive.keytab hive / node2
kadmin mode: principle addprinc -randkey root/[email protected] to
generate random key
principle addprinc -pw **** admin/[email protected] View principal
listprincs
modify the password of admin / admin cpw
-pw xxxx admin / admin
add / delete principle
addprinc / delprinc admin / admin
directly to keytab
ktadd -k /etc/krb5.keytab host/[email protected]
set password policy (policy)
addpol -maxlife "90 days" -minlife "75 days" -minlength 8 -minclasses 3 -maxfailure 10 -history 10 user
add user with password policy
addprinc -policy user hello/[email protected]
modify user's password policy
modprinc -policy user1 hello/[email protected]
delete password policy
delpol [-force] user
modify password policy
modpol -maxlife "90 days" -minlife "75 days" -minlength 8 -minclasses 3 -maxfailure 10 user
Second, CDH enables Kerberos
Operations in CM
Click Enable Kerberos on the CM interface. When enabling, you need to confirm several things:
1. KDC is installed and running
2. Configure KDC to allow renewable tickets with non-zerolifetime (three options of kdc_tcp_ports, max_life and max_renewable_life have been added when the kdc.conf file was modified before)
3. In Cloudera Manager Install openldap-clients on the Server
4. Create a principal for Cloudera Manager to enable it to create other principals in the KDC, which is the Kerberos administrator account created above
Enable Keberos
Check all
First line selection: MIT KDC
Fill in the second line: master node
Fill in the third line: NETCLOUD.COM
Fill in the fourth line: aes256-cts
Fifth line selection: 5 days
Note: Uncheck
If you cannot see clearly, you can read the following text:
First line: root/[email protected]
The second line: netcloud @ 123 (here is the password we just set, it is recommended to set to netcloud @ 123)
This step is also the default, directly check Yes , and then restart the entire cluster of services, complete the configuration of Kerberos in CDH
PS : Possible problems
1. If we have a problem in configuring Kerberos , we must determine whether the information we just configured is correct, because if it is wrong, it will cause CDH to fail to apply the Kerberos configuration correctly .
2. If, after configuring Kerberos , the ZooKeeper service fails to start and reports an error, or other services fail to start and report an error:
java.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly:
javax.security.auth.login.LoginException: No supported encryption types listed in default_tkt_enctypes
It is 100% sure that your jdk path is not found correctly. You need to find out whether the java -version used by this machine is the same as the jdk path of your machine , and then move the files under the UnlimitedJCEPolicyJDK7 folder to me. Write the path below, and then restart the service just reported error.
3. If the service does not start successfully halfway, we need to confirm the master file and slaves file of each node to ensure that the master file and slave file of each machine are consistent.
If all services start without problems, after the configuration is complete, go to the master node and create an HDFS super user:
kadmin.local -q "addprinc hdfs"
This will be prompted halfway: just enter the password, it is recommended: netcloud @ 123
Authenticating as principal root/[email protected] with password.
WARNING: no policy specified for [email protected]; defaulting to no policy
Enter password for principal "[email protected]":
Re-enter password for principal "[email protected]":
Principal "[email protected]" created.
After the user creation is complete, we verify that Kerberos is working properly on the cluster:
First, enter on the master node:
hadoop fs -ls /
It will report an error, this is normal, if it does not report an error, it means there is a problem, please check the above steps
Then we enter:
kinit hdfs
or
kinit [email protected]
Prompt us to enter the password, we enter netcloud @ 123 , and then we enter:
hadoop fs -ls /
Found that it can be displayed normally, indicating successful Kerberos configuration
If other clients use root or hdfs users to execute the hadoop fs -ls / command, an error will be reported
Need to execute:
kinit root/[email protected]
or
kinit [email protected]
Note: Create a principal for each user of each node (kinit [email protected])
3. Verify that Kerberos is working properly on the cluster
Confirm HDFS can be used normally
After logging in to a certain node, switch to the hdfs user, and then use kinit to obtain credentials ( kinit [email protected] )
Now use hadoop hdfs -ls / should be able to output the normal result
. After destroying credentials with kdestroy, then use hadoop hdfs -ls / will find an error
Note:
If the ticket has an expiration time, set crontab to refresh it regularly. Or you set the expiration time to be very large
kinit -k -t /etc/krb5.keytab hdfs;
Solution for expired ticket and expired password (recreate an HDFS super user)
kadmin.local -q "addprinc ysj"
This will be prompted halfway: just enter the password, suggest: netcloud @ 123
Then we enter:
kinit ysj
or
kinit [email protected]
Prompt us to enter the password, we enter netcloud @ 123, and then we enter:
hadoop fs -ls /
Found that it can be displayed normally, indicating successful Kerberos configuration
Or delete the previous super user and re-create
kadmin.local -q "delprinc hdfs"
kadmin.local -q "addprinc hdfs"
Then we enter:
kinit ysj
or
kinit [email protected]
Prompt us to enter the password, we enter netcloud @ 123, and then we enter:
hadoop fs -ls /
Found that it can be displayed normally, indicating successful Kerberos configuration
Reference to other questions: https://www.cnblogs.com/xiaodf/p/5968178.html
conclusion of issue:
1) Hue's integrated Kerberos causes the Kerberos Ticket Renewer to not get up
Error, Kerberos Ticket Renewer can't get up, check the log
[19/Jan/2018 07:10:08 +0000] kt_renewer INFO Renewing kerberos ticket to work around kerberos 1.8.1: /usr/bin/kinit -R -c /run/hue/hue_krb5_ccache kinit: KDC can't fulfill requested option while renewing credentials [19/Jan/2018 07:10:08 +0000] kt_renewer ERROR Couldn't renew kerberos ticket in order to work around Kerberos 1.8.1 issue. Please check that the ticket for 'hue/[email protected]' is still renewable: $ klist -f -c /run/hue/hue_krb5_ccache If the 'renew until' date is the same as the 'valid starting' date, the ticket cannot be renewed. Please check your KDC configuration, and the ticket renewal policy (maxrenewlife) for the 'hue/[email protected]' and `krbtgt' principals. [19/Jan/2018 23:10:09 +0000] settings DEBUG DESKTOP_DB_TEST_NAME SET: /usr/local/hue/desktop/desktop-test.db [19/Jan/2018 23:10:09 +0000] settings DEBUG DESKTOP_DB_TEST_USER SET: hue_test [19/Jan/2018 07:10:09 +0000] __init__ INFO Couldn't import snappy. Support for snappy compression disabled. [19/Jan/2018 07:10:09 +0000] kt_renewer INFO Reinitting kerberos from keytab: /usr/bin/kinit -k -t /etc/security/keytabs/hue.service.keytab -c /var/run/hue/hue_krb5_ccache hue/[email protected] [19/Jan/2018 07:10:10 +0000] kt_renewer INFO Renewing kerberos ticket to work around kerberos 1.8.1: /usr/bin/kinit -R -c /run/hue/hue_krb5_ccache kinit: KDC can't fulfill requested option while renewing credentials [19/Jan/2018 07:10:10 +0000] kt_renewer ERROR Couldn't renew kerberos ticket in order to work around Kerberos 1.8.1 issue. Please check that the ticket for 'hue/[email protected]' is still renewable:
