1. Kerberos overview
Kerberos is a computer network authorization protocol used to authenticate personal communications in a secure manner in a non-secure network. The term in turn refers to a suite of computer software developed by the Massachusetts Institute of Technology for this protocol. The software design adopts a client/server structure and can perform mutual authentication, that is, both the client and the server can authenticate each other's identity. It can be used to prevent eavesdropping, prevent replay attacks, protect data integrity, etc. It is a system that uses a symmetric key system for key management.
Hadoop uses Kerberos as the basis for strong authentication and identity propagation of users and services. Kerberos is a computer network authentication protocol that allows an entity to communicate in a non-secure network environment and prove its identity to another entity in a secure manner. Kerberos is a third-party authentication mechanism where users and services rely on a third party (the Kerberos server) to authenticate each other. The Kerberos server itself is called a Key Distribution Center or KDC. At a high level, it has three parts:
An Authentication Server (AS) performs initial authentication and issues a Ticket-Granting Ticket (TGT).
A Ticket-Granting Server (TGS) issues a TGT based on the initial subsequent service.
A user primarily requests authentication from the AS. The AS returns a TGT encrypted with the user principal's Kerberos password known only to the user principal and the AS. The user principal decrypts the TGT locally using its Kerberos password, and from then on, until the ticket expires, the user principal can use the TGT to obtain a service ticket from the TGS. Service tickets allow principals to access various services.
Simply put, Kerberos is a third-party protocol for security authentication. It uses the traditional shared key method to realize the communication between the client and the server in an environment where the network environment does not necessarily guarantee security. It is suitable for client/ server model, developed and implemented by MIT.
The Kerberos service is a single sign-on system, which means that you only need to authenticate yourself to the service once per session, and all subsequent transactions during that session are automatically secured.
Since cluster resources (hosts or services) cannot provide a password every time a TGT is decrypted, they use a special file called keytab which contains the authentication credentials of the resource principal. The set of hosts, users, and services controlled by a Kerberos server is called a realm.
2. Kerberos terminology
Principal (security individual) : A user or a service managed by Kerberos can be understood as an account stored in Kerberos, and its format is usually as follows: primary**/**instance@realm
Realm : A domain or scope managed by Kerberos, called a Realm.
keytab : User authentication in Kerberos can prove identity through password or key file, and keytab refers to the key file.
KDC (key distribution center) : ticket generation management service for the authentication process, which includes two services, AS (Authentication Service) and TGS (Ticket Granting Service).
Ticket : A record that clients use to prove their identity to the server, including client ID, session key, and timestamp.
AS (Authentication Server) : authentication server, a service that generates TGT for the client.
TGS(Ticket Grant Server) : License server, which generates a ticket for a certain service for the client
. TGT(Ticket-grantion Ticket) : Ticket used to obtain the ticket.
AD (Account Database) : Stores the white list of all clients, and only exists in the white list Only clients on the list can successfully apply for tickets.
3. Kerberos authentication process
(1) The client executes the kinit command, enters Principal and Password, proves its identity to AS, and requests to obtain TGT.
(2) AS checks whether the Principal entered by the client exists in the Database, and if so, returns TGT to the client.
(3) After the client obtains the TGT, it requests the ServerTicket from the TGS.
(4) TGS receives the request, checks whether the Principal of the service requested by the client exists in the Database, and if so, returns the ServerTicket to the client.
(5) When the client receives the ServerTicket, it initiates a request to the target service.
(6) The target service receives the request and responds to the client.
4. Kerberos deployment
4.1 Overview
Cloudera provides a very simple Kerberos integration method, which basically achieves automated deployment.
System: CentOS 7.3
Operating user: admin
The roles are distributed as follows:
| Role | Deploy node |
|---|---|
| KDC, AS, TGS,Kerberos Agent | 192.168.0.200 |
| Kerberos Agent | 192.168.0.[201-202] |
Assume that the slaves file is as follows:
192.168.0.201
192.168.0.202
4.3 Kerberos installation
Install the server on 192.168.0.200:
yum -y install krb5-server openldap-clients
Install the client on 192.168.0.[201-202]:
pssh -h slaves -P -l root -A "yum -y install krb5-devel krb5-workstation"
After installing the above software, the configuration files /etc/krb5.conf and /var/kerberos/krb5kdc/kdc.conf will be generated on the KDC host, which reflect the realm name and domain-to-realm mappings respectively.
4.4 Modify configuration
4.4.1 Configure krb5.conf
/etc/krb5.conf: Contains Kerberos configuration information. For example, the location of the KDC, the realms of Kerberos admin, etc. The configuration files on all machines using Kerberos need to be synchronized. Only the required basic configuration is listed here.
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = XICHUAN.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
default_ccache_name = KEYRING:persistent:%{
uid}
[realms]
EXAMPLE.COM = {
kdc = master01-dev
admin_server = master01-dev
}
[domain_realm]
.example.com = XICHUAN.COM
example.com = XICHUAN.COM
Description :
[logging] : Indicates the printing position of the log on the server side
[libdefaults] : The default configuration for each connection, you need to pay attention to the following key small configurations
-
default_realm = EXAMPLE.COM The default realm must be consistent with the name of the realm to be configured.
-
udp_preference_limit = 1 disabling the use of udp can prevent a bug in Hadoop
-
ticket_lifetime indicates the validity period of the certificate, usually 24 hours.
-
renew_lifetime indicates the maximum time limit for which the certificate can be extended, usually one week. When the credentials expire, subsequent access to securely authenticated services will fail.
[realms] : List the realms used.
-
kdc : represents the location of kdc. The format is machine:port
-
admin_server : represents the location of admin. The format is machine:port
-
default_domain : represents the default domain name
[appdefaults] : You can set some application-specific configurations to override the default configurations.
Distribute krb5.conf to all clients:
//This step is not necessary, because after configuring krb5.conf in Cloudera Manager, it will automatically deploy it to the client for you.
pscp -h slaves krb5.conf /tmp
pssh -h slaves "cp /tmp/krb5.conf /etc"
4.4.2 Configure kdc.conf
By default placed in/var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88
[realms]
XICHUAN.COM = {
#master_key_type = aes256-cts
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes128-cts:normal aes256-cts des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
max_life = 24h
max_renewable_life = 7d
}
illustrate:
XICHUAN.COM : It is the set realms. The name is arbitrary. Kerberos can support multiple realms, which increases complexity. This article does not discuss it. Case sensitive, generally use all uppercase letters for identification. This realms has nothing to do with the host of the machine.
max_renewable_life = 8d must be configured to determine whether tickets can be issued.
master_key_type: and supported_enctypes use aes256-cts by default. Since JAVA uses the aes256-cts authentication method, an additional jar package needs to be installed, so it is recommended not to use it.
acl_file : The user authority of admin is marked. The file format is:
SupportsKerberos_principal permissions [target_principal][restrictions] wildcards, etc.
admin_keytab: The keytab for KDC verification. How to create it will be mentioned later.
supported_enctypes : Supported verification methods. Pay attention to remove aes256-cts.
4.4.3 Configure kadm5.acl
Modify the configuration file /var/kerberos/krb5kdc/kadm5.acl on the server 192.168.0.200 to allow admin users with matching conditions to log in remotely:
*/[email protected] *
illustrate:
The user permissions of admin are marked and need to be created by the user. The file format is:
SupportsKerberos_principal permissions [target_principal][restrictions]
wildcards, etc. The simplest way to write it is: If the
name*/[email protected] *
4.5 Create Kerberos database
Initialize the database on 192.168.0.200. The default database path is /var/kerberos/krb5kdc. If you need to rebuild the database, just delete the principal-related files in this directory. Please remember the database password.
kdb5_util create -r XICHUAN.COM -s
illustrate:
[-s] means to generate a stash file and store the master server key (krb5kdc) in it
[-r] to specify a realm name, used when multiple realms are defined in krb5.conf
After the Kerberos database is created, in / The generated principal related files can be seen in var/kerberos/
4.6 Start the Kerberos service
Execute on 192.168.0.200:
#启动服务命令
systemctl start krb5kdc
systemctl start kadmin
#加入开机启动项
systemctl enable krb5kdc
systemctl enable kadmin
View process:
[root@master01-dev ~]# netstat -anpl|grep kadmin
tcp 0 0 0.0.0.0:749 0.0.0.0:* LISTEN 67465/kadmind
tcp 0 0 0.0.0.0:464 0.0.0.0:* LISTEN 67465/kadmind
tcp6 0 0 :::749 :::* LISTEN 67465/kadmind
tcp6 0 0 :::464 :::* LISTEN 67465/kadmind
udp 0 0 0.0.0.0:464 0.0.0.0:* 67465/kadmind
udp6 0 0 :::464 :::* 67465/kadmind
[root@master01-dev ~]# netstat -anpl|grep kdc
tcp 0 0 0.0.0.0:88 0.0.0.0:* LISTEN 67402/krb5kdc
tcp6 0 0 :::88 :::* LISTEN 67402/krb5kdc
udp 0 0 0.0.0.0:88 0.0.0.0:* 67402/krb5kdc
udp6 0 0 :::88 :::* 67402/krb5kdc
4.7 Create Kerberos administrator principal
#需要设置两次密码
kadmin.local -q "addprinc root/admin"
#使用cloudera-scm/admin作为CM创建其它principals的超级用户
kadmin.local -q "addprinc cloudera-scm/admin"
The second part of the principal's name is admin, so according to the previously configured acl file, the principal has administrative privileges, and this account will be used by CDH to generate principals for other users/services.
注意需要先kinit保证已经有principal缓存。
[root@master01-dev ~]# kinit cloudera-scm/admin
Password for cloudera-scm/[email protected]:
[root@master01-dev ~]# klist
Ticket cache: KEYRING:persistent:0:0
Default principal: cloudera-scm/[email protected]
Valid starting Expires Service principal
09/01/2022 11:51:17 09/02/2017 11:51:16 krbtgt/[email protected]
renew until 09/08/2022 11:51:16
Kerberos client supports two types, one is to use principal + Password, the other is to use principal + keytab
Principal + Password is suitable for users to carry out interactive applications, such as hadoop fs -ls, and the latter is suitable for services, such as yarn's rm, nm, etc.
Principal + keytab is similar to ssh password-free login. No password is required when logging in.
Five. Kerberos use
5.1 Introduction to Kerberos Principal
There is a detailed introduction to Principal on the official website, the link is: http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-user/What-is-a-Kerberos- Principal_003f.html
A principal is a unique identifier to which Kerberos can assign tickets. A body can have any number of components. Each component is separated by a component separator (usually /). The last component is the field, separated from the rest of the body by the field separator character (usually @). If there is no realm component in the principal, the principal will be assumed to be in the default realm in the context in which it is used
The format of Principal is generally: primary/instance@REALM
primary is part of the Principal, which is the username if it is a user, or the hostname if it is a host.
instance is an optional string used to qualify the main server. This instance is separated from the main instance by a slash (/). For users, this instance is usually null, but users may also have an additional principal, an instance called admin, which is used to administer the database. The principal [email protected] is completely separate from the principal root/[email protected], with separate passwords and separate permissions. For hosts, the instance is a fully qualified hostname, such as xichuan.com.
REALM is your Kerberos realm. In most cases, your Kerberos realm is your domain name, in capital letters. For example, the machine daffodil.example.com is in the domain EXAMPLE.COM
5.2 Kerberos command usage
kadmin.local 与 kadmin区别：
kadmin.local：需要在 KDC server 上面操作，无需密码即可管理资料库
kadmin：可以在任何一台 KDC 领域的系统上面操作，但是需要输入管理员密码
5.2.1 Log in to kinit
kinit root/[email protected]
[root@node01 ~]# kinit root/[email protected]
Password for root/[email protected]: root
[root@node01 ~]#
Or use keytap to log in
kinit -kt /opt/xichuan.keytab xichuan/admin
[root@node01 ~]# kinit -kt /opt/xichuan.keytab xichuan/admin
[root@node01 ~]#
5.2.2 Query login status klist
klist
[root@node01 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: root/[email protected]
Valid starting Expires Service principal
10/26/2022 15:19:15 10/27/2022 15:19:15 krbtgt/[email protected]
renew until 11/02/2022 15:19:15
5.2.3 Exit and log in to kdestroy
kdestroy
[root@node01 ~]# kdestroy
[root@node01 ~]# klist
klist: No credentials cache found (filename: /tmp/krb5cc_0)
5.2.4 Log in to the KDC backend kadmin.local
kadmin.local
[root@node02 etc]# kadmin.local
Authenticating as principal root/[email protected] with password.
kadmin.local:
If you cannot find the kadmin.local command, you can use find / -name kadmin to find it.
The general location is: /usr/bin/kadmin
5.2.5 View user list listprincs
listprincs
[root@node01 ~]# kadmin
Authenticating as principal root/[email protected] with password.
Password for root/[email protected]:
kadmin: listprincs
HTTP/[email protected]
HTTP/[email protected]
HTTP/[email protected]
K/[email protected]
cloudera-scm/[email protected]
hdfs/[email protected]
hdfs/[email protected]
hdfs/[email protected]
hive/[email protected]
httpfs/[email protected]
hue/[email protected]
impala/[email protected]
impala/[email protected]
impala/[email protected]
kadmin/[email protected]
kadmin/[email protected]
kadmin/[email protected]
kiprop/[email protected]
krbtgt/[email protected]
kudu/[email protected]
kudu/[email protected]
kudu/[email protected]
mapred/[email protected]
root/[email protected]
yarn/[email protected]
yarn/[email protected]
yarn/[email protected]
zookeeper/[email protected]
zookeeper/[email protected]
zookeeper/[email protected]
kadmin:
5.2.6 Modify account password change_password
change_password root/[email protected]
[root@node01 ~]# kadmin
Authenticating as principal root/[email protected] with password.
Password for root/[email protected]:
kadmin: change_password root/[email protected]
Enter password for principal "root/[email protected]": root
Re-enter password for principal "root/[email protected]": root
Password for "root/[email protected]" changed.
kadmin:
5.2.7 Create user addprinc
addprinc -pw root xichuan/admin #创建用户名hadoop用户,密码为123456
[root@node01 ~]# kadmin
Authenticating as principal root/[email protected] with password.
Password for root/[email protected]: root
kadmin: addprinc -pw root xichuan/admin #创建用户名hadoop用户,密码为123456
WARNING: no policy specified for xichuan/[email protected]; defaulting to no policy
Principal "xichuan/[email protected]" created.
When creating a user, if there is no REALM, use the default REALM
5.2.8 Delete user delprinc
delete_principal test/admin
[root@node01 ~]# kadmin
Authenticating as principal root/[email protected] with password.
Password for root/[email protected]: root
kadmin: delete_principal test/admin
Are you sure you want to delete the principal "test/[email protected]"? (yes/no): yes
Principal "test/[email protected]" deleted.
Make sure that you have removed this principal from all ACLs before reusing.
kadmin:
5.2.9 Export keytab file
ktadd -k /opt/xichuan.keytab -norandkey xichuan/[email protected]
[root@node02 etc]# kadmin.local
Authenticating as principal root/[email protected] with password.
kadmin.local: ktadd -k /opt/xichuan.keytab -norandkey xichuan/[email protected]
Entry for principal xichuan/[email protected] with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type des3-cbc-sha1 added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type arcfour-hmac added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type camellia256-cts-cmac added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type camellia128-cts-cmac added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type des-hmac-sha1 added to keytab WRFILE:/opt/xichuan.keytab.
Entry for principal xichuan/[email protected] with kvno 1, encryption type des-cbc-md5 added to keytab WRFILE:/opt/xichuan.keytab.
-norandkeyIndicates that the password will not be changed
The default exported directory is the current directory, which is operated on the kadmin.local command
5.2.10 View the user list in the keytab file
klist -ket /opt/xichuan.keytab
[root@node02 opt]# klist -ket /opt/xichuan.keytab
Keytab name: FILE:/opt/xichuan.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
1 10/26/2022 15:41:35 xichuan/[email protected] (aes256-cts-hmac-sha1-96)
1 10/26/2022 15:41:35 xichuan/[email protected] (aes128-cts-hmac-sha1-96)
1 10/26/2022 15:41:35 xichuan/[email protected] (des3-cbc-sha1)
1 10/26/2022 15:41:35 xichuan/[email protected] (arcfour-hmac)
1 10/26/2022 15:41:35 xichuan/[email protected] (camellia256-cts-cmac)
1 10/26/2022 15:41:35 xichuan/[email protected] (camellia128-cts-cmac)
1 10/26/2022 15:41:35 xichuan/[email protected] (des-hmac-sha1)
1 10/26/2022 15:41:35 xichuan/[email protected] (des-cbc-md5)
5.2.11 Update ticket kinit
kinit -R
[root@node01 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: xichuan/[email protected]
Valid starting Expires Service principal
10/26/2022 15:45:07 10/27/2022 15:45:07 krbtgt/[email protected]
renew until 11/02/2022 15:45:07
[root@node01 ~]# kinit -R
[root@node01 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: xichuan/[email protected]
Valid starting Expires Service principal
10/26/2022 15:45:54 10/27/2022 15:45:54 krbtgt/[email protected]
renew until 11/02/2022 15:45:07
You can see that the Expires time of the two Tickets is different
5.2.12 View Ticket details
getprinc xichuan/admin
[root@node01 ~]# kadmin
Authenticating as principal xichuan/[email protected] with password.
Password for xichuan/[email protected]: root
kadmin: getprinc xichuan/admin
Principal: xichuan/[email protected]
Expiration date: [never]
Last password change: Wed Oct 26 15:24:39 CST 2022
Password expiration date: [never]
Maximum ticket life: 1 day 01:00:00
Maximum renewable life: 8 days 00:00:00
Last modified: Wed Oct 26 15:24:39 CST 2022 (root/[email protected])
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 8
Key: vno 1, aes256-cts-hmac-sha1-96
Key: vno 1, aes128-cts-hmac-sha1-96
Key: vno 1, des3-cbc-sha1
Key: vno 1, arcfour-hmac
Key: vno 1, camellia256-cts-cmac
Key: vno 1, camellia128-cts-cmac
Key: vno 1, des-hmac-sha1
Key: vno 1, des-cbc-md5
MKey: vno 1
Attributes:
Policy: [none]
kadmin: