table of Contents
2. Detailed explanation of Ranger Aduit audit function module
2. Introduction to Ranger Audit function module
1. Ranger introduction
1. Ranger concept
Ranger uses an attribute-based approach to define and enforce security policies. When used with Apache Hadoop's data governance solution and metadata warehousing component Apache Atlas, it can define a tag-based security service that uses tags to classify files and data assets, and controls the pairing of users and user groups. Series tab access.
The overall architecture of Ranger is shown in the figure below, which is mainly composed of the following three components:
- AdminServer: Provides an interface for adding, deleting, modifying and checking policies in the form of RESTFUL, and a built-in web management page.
- AgentPlugin: Embedded in the execution process of each system, regularly pulls the strategy from AdminServer, executes the access decision tree according to the strategy, and regularly records the access audit. The implementation principle of the plug-in will be described in detail later.
- UserSync: Periodically load users from LDAP/File and report to AdminServer.
Ranger targets the permissions within components, such as HDFS read and write execution, Hive and Hbase read and write updates, yarn queue resource usage rights, task submission permissions, currently ranger supports hdfs, hive, hbase, kafka, yarn and other components, for Groups and users have fine-grained control over resource access.
- user: The users managed by Ranger itself are divided into internal and external. The former is Ranger's own user, such as admin; the latter is a Linux or LDAP user. After users are added to the operating system/LDAP, they will be synchronized to Ranger .
- group: The user group managed by Ranger itself can also be divided into internal and external. Similar to user, the LDAP group will be synchronized after the setting is synchronized with LDAP. This group is an external group; if it is a group added by Ranger's own user, it is internal group.
- Service: The authorization management service, each component can be set with multiple services.
- Policy: There can be multiple policies in each service, with different components and different policy authorization models.
Ranger mainly provides the following features:
- Policy-based access permission model
- General strategy synchronization and decision-making logic, convenient to control the extended access of plug-ins
- Built-in control plug-ins for common systems (such as 12 HDFS, YARN, HBase, etc.), and can be extended
- Built-in user synchronization mechanism based on LDAP, File and Unix, and can be extended
- Unified centralized management interface, including policy management, audit review, plug-in management, etc.
2. Permission model
Access permissions are nothing more than defining the relationship between the "user-resources-permissions". Ranger abstracts this relationship based on policies, and then extends its own permission model. The meaning of "user-resource-authority" is explained in detail:
- User: Expressed by User or Group, User represents the user who accesses the resource, and Group represents the user group to which the user belongs.
- Resource: Expressed by Resource. The business resources corresponding to different components are different, such as File Path of HDFS and Table of HBase.
- Permission: Expressed by (AllowACL, DenyACL), similar to whitelist and blacklist mechanisms, AllowACL is used to describe the situation of allowed access, and DenyACL is used to describe the situation of denied access. The permissions for different components are also different.
2. Detailed explanation of Ranger Aduit audit function module
1、Ambari 中 Ranger config
1.Ranger Admin
Description: Ranger Amin uses mysql to store admin metadata information
2.Ranger User Info
Note : The following picture is the Ranger user info of Ranger's config in Ambari. Ranger supports synchronizing user information from UNIX, FILE, and LDAP. After users are created on the LINUX platform, they will be synchronized to Ranger. The default is to synchronize from the operating system, and vice versa. The user cannot synchronize to the operating system.
3.Ranger Plugin
4.Ranger Audit
Ranger Audit: Ranger audit function, Ranger audit log is stored in HDFS path. The audit function module can help us better manage the cluster components. For example, the access to data resources such as HDFS, HBase, Hive can be controlled by Ranger, and the historical operations of users can be well monitored and inquired, so that it can be verified.
Audit log storage path in HDFS
5.Ranger Tagsync
Aparche Atlas Data Governance Project, which is used to manage shared metadata, data classification, auditing, security, and data protection, and strive to integrate with Apache Ranger for data access control strategies.
2. Introduction to Ranger Audit function module
1.Audit Access
Note: For the policy created by the kangll user, the operations of the kangll user on the cluster components are also recorded before the policy is created, but the permissions are not allowed. The Kangll user performed a query operation in Hive, and the execution result of this permission is allowed.
Note : The operation record of Kangll users in the Ranger audit function module is allowed for kangll users to write to the /kangll folder under HDFS.
2.Audit Admin
Note: The operation record of the Ranger Admin user is the user we used when logging in
3.Audit Login Sessions
4.Audit Plugins
5.Audit Plugin Status
6.Audit User Sync
As you can see in the figure, Ranger will synchronize users from the default Unix operating system every minute by default, and get this user, and the user's operations on cluster components can be seen in Access.
Reference link: https://www.jianshu.com/p/d0bf6e77bb8f