Outline
To Hortonworks Data Platform (HDP) platform, for example, security mechanisms hadoop big data platform includes the following two aspects:
- Namely to verify the identity of the true identity of a user, a user to use big data engine platform, the user needs to show who he is, namely to provide proof of their identity, big data platform need to test this proof, this proved to be effective to determine the , and not a forgery. Otherwise, reject the user to enter the big data engine.
- After the true identity of authorized users to manage this verification, the need to define user permissions on this, that the user can use what components in big data platform, which can acquire resources, which can operate on resource management.
Authentication
Big Data platform based on the general use of the Kerberos authentication mechanism
Overview
Simply put, big data platform has a dedicated authentication server KDC, you can see it as a local police station, in advance to all platform users (human users and user machines and programs) issued proof of residence, namely keytab ( key). After each user wants to use big data platform, we should take this to go to prove KDC certification, after the authentication is correct, to be able to use big data platform engine.Operation Example
We must first kerberos of the entire platform, hdp ambari provides a fool-operation, attached article describes.
As shown above, after kerberos technology, we ambari management interface secondary development. Here create a local user, is already registered with the KDC server synchronization related information of the user, while generating the user's keytab, click on the user, enter the details page to download key. After the user to use big data platform (either a command line or API call or whatever mode), it must first carry keytab to KDC for authentication. In this example command line, for example, a user needs to use the above hdfs hadoop to read the file. He needs to light out their own identity, that is, execute the command:
kinit -k -t keytab路径 用户名 After this time go hdfs execute commands like to be able to authenticate through without error:
Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]Authorization Management
Big Data platform based authorization management apache ranger's. Authorization Manager is supposed to be built on top of authentication, imagine the following, even if the identity can not be confirmed, then any license you are unreliable (Are you sure you authorized this user is the user do), your authorization Manager is actually a pie in the sky.
Overview
ranger provides resource-based authorization management mechanism, the so-called resource is the big data platform of the components (for example hdfs, hive, hbase, etc.), as well as specific resources (for example, a path of hdfs, hive, hbase inner components a table). A component corresponding to the resource ranger in the service (service), create policies in a single service can be in the user's authority management of specific resources for this component.Guide
us to ambari interface secondary development, the ranger native interface transplant
As shown, as an example, ranger detect and automatically generate the various components of the service platform, click to enter service details
Add or edit a policy to specific resources authorized to manage. As shown above, we grant the user testranger all operating authority for all columns of all tables hbase. If you edit this policy, delete testranger the user, then the user query and other operations in hbase do when an error occurs:
ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘[email protected]',action: scannerOpen, tableName:test, family:f1 In addition you can also create new strategies to manage policies provide fine-grained permission management, refer to the detailed ranger official website.