Row-level filtering and column masking in Hive
You can use Apache Ranger row-level filters to set access policies for rows
in Hive tables. You can also use Ranger column masking to set a strategy for masking the data in the Hive column, such as displaying only the first four characters or the last four characters of the column data.
Row-level filtering in Hive using Ranger strategy Row-
level filtering helps simplify Hive queries. Hive moves the access restriction logic down to the Hive layer and applies the access restriction every time it tries to access data. This helps simplify the writing of Hive queries and provides seamless row-level segmentation of background execution without adding this logic to the predicate of the query.
About this task
Row-level filtering also improves the reliability and robustness of Hadoop. By providing row-level security for Hive tables and reducing the security surface area, Hive data access can be restricted to specific rows based on user characteristics (such as group membership) and the runtime context of the request.
Typical use cases where row-level filtering can work include:
• A hospital can create a security policy that allows doctors to view only the rows of data for their own patients, and allows insurance claims administrators to view only specific rows of their specific sites.
• The bank can be based on the employee's business department, region, or employee role (for example, only employees in the financial department are allowed to view customer invoices, payments and accountable data; only European HR employees can see European employee data).
• Multi-tenant applications can create a logical separation of each tenant's data so that each tenant can only see its own data rows. You can use Apache Ranger row-level filters to set access policies for rows in Hive tables. The row-level filter policy is similar to other administrator access policies. You can set filters for specific users, groups, and conditions.
When using row-level filters, the following conditions apply:
• The filter expression must be a valid WHERE clause of the table or view.
• Each table or view should have its own row-level filtering strategy.
• The database or table name does not support wildcard matching.
• The filters are evaluated in the order listed in the strategy.
• Each time a row-level filter is applied to a table or view, an audit log entry is generated.
process
-
On the "Service Manager" interface, select an existing Hive service.
-
Select the row-level filter tab and click Add New Policy.
-
On the "Create Policy" page, add the following information for the row-level filter:
Table I
| Field | description |
|---|---|
| Policy Name(required) | Enter the appropriate policy name. This name cannot be repeated in the system. By default, this policy is enabled. |
| normal/override | Allows you to specify the coverage strategy. When "Overwrite" is selected, the access rights in the policy will overwrite the access rights in the existing policy. This feature can be used together with "Add Validity Period" to create temporary access policies that override existing policies. |
| Hive Database(required) | Type the applicable database name. The auto-completion feature displays the available databases based on the entered text. |
| Hive Table(required) | Type the applicable table name. The auto-completion feature displays the available tables based on the entered text. |
| Audit Logging | The audit log is set to Yes by default. Select No to turn off audit logging. |
| Description | Enter an optional description for the strategy. |
| Add Validity Period | Specify the start and end time of the strategy. |
Table II
| label | description |
|---|---|
| Select Group | Specify the group to which this policy applies. The public group contains all users, so granting access to the public group will grant access to all users. |
| Select User | Specify one or more users to apply this policy. |
| Access Types | "Select" is currently the only type of access available. This will be used with the WHERE clause specified in the row-level filter field. |
| Add Row Filter | (1) To create a row filter for the specified users and groups, click "Add Row Filter", and then enter a valid WHERE clause in the "Enter Filter Expression" box. (2) In order to allow Select access for specified users and groups without row-level restrictions, no row filter is added (retain the "Add Row Filter" setting). (3) The filters are evaluated in the order listed in the strategy. The filter at the top of the row filter criteria list is applied first, then the second, then the third, and so on. |
- To move a condition in the list of row filter conditions (thus changing the order in which it is evaluated), click the dotted line icon to the left of the condition row, and then drag the condition to a new position in the list.
- Click Add to add a new row-level filtering policy.
Marker-based column shielding using Ranger strategy in Hive
Ranger's resource-based Hive shielding strategy will anonymize data from the Hive column identified by the database, table and column, while the label-based shielding strategy will be based on the tags and tag attribute values related to the Hive column (usually designated as metadata classification in Atlas) ) Anonymous Hive column data.
About this task
When using the Ranger column shielding strategy to shield the data returned by the Hive query, the following conditions must be met
:
• Multiple shielding types can be used, such as displaying the last 4 characters, displaying the first 4 characters, hash, Nullify, and date Blocked (only the year is displayed).
• You can specify the type of blocking for specific users, groups, and conditions.
• Does not support wildcard matching.
• If multiple label shielding strategies are applied to the same Hive column, select the shielding strategy with the smallest lexicographic order to execute, for example, strategy "a" is executed before strategy "aa".
• Masks are evaluated in the order listed in the strategy.
• When a shielding strategy is applied to a column, an audit log entry will be generated.
process
-
Choose Access Manager> Tag-Based Policies, and then select Tag-Based Services.
-
Select the Masking tab, and click Add New Policy.
-
On the "Create Policy" page, add the following information for the column mask filter:
Table 1: Strategy details
| Field | description |
|---|---|
| Policy Type(required) | The default setting is blocked. |
| Policy Name(required) | Enter the appropriate policy name. This name cannot be repeated in the system. By default, this policy is enabled. |
| normal/override | Allows you to specify the coverage strategy. When "Overwrite" is selected, the access rights in the policy will overwrite the access rights in the existing policy. This feature can be used together with "Add Validity Period" to create temporary access policies that override existing policies. |
| TAG(required) | Enter the applicable label name, such as MASK. |
| Audit Logging | The audit log is set to Yes by default. Select No to turn off audit logging. |
| Description | Enter an optional description for the strategy. |
| Add Validity Period | Specify the start and end time of the strategy |
| Policy Conditions (applied at the policy level) | Click the + icon to add a policy condition. Currently "Access after expiration date?" "(Yes/No)" is the only policy condition available. Visit after expiry_date (yes/no)?: To set this condition, enter yes in the text box, and then click the check mark button to add the condition. Enter a Boolean expression: it can be used to allow or deny conditions on label-based policies. For examples and details, see "Using Tag Attributes and Values in Ranger Tag-Based Policy Conditions". Click "Save" to save the policy conditions. |
Table 2: Mask conditions
| label | description |
|---|---|
| Select Group | Specify the group to which this policy applies. The public group contains all users, so granting access to the public group will grant access to all users. |
| Select User | Specify one or more users to apply this policy. |
| Policy Conditions (applied at the item level) | Click "Add Condition" to add a policy condition. Currently "Access after expiration date?" "(Yes/No)" is the only policy condition available. Visit after expiry_date (yes/no)?: To set this condition, enter yes in the text box, and then click the check mark button to add the condition. Enter a Boolean expression: it can be used to allow or deny conditions on label-based policies. For examples and details, see "Using Tag Attributes and Values in Ranger Tag-Based Policy Conditions". |
| Access Types | "Select" is the only access type currently available for hive components. |
| Select Masking Option | To create a line filter for specified users and groups, click "Select Mask Options", and then select the mask type: • Edit-Use "x" to block all alphabetic characters and "n" to block all numeric characters. • Partial mask: display the last 4 characters-only the last 4 characters are displayed. • Partial mask: display the first 4 characters-only the first 4 characters are displayed. •Hash-Replace all characters with the Hash of the entire cell value. • Nullify-Replace all characters with null values. • Unshielded (keep original value)-No application is shielded. •Date: Only display year-Only display the year part in the date string, and the default month and date are 01/01 • Custom-Specify a custom mask value or expression. Custom shielding can use any valid Hive UDF (Hive that returns the same data type as the data type in the shielded column). The blocking conditions are evaluated in the order listed in the policy. The condition at the top of the blocking condition list is applied first, then second, then third, and so on. |
- You can use the plus (+) symbol to add additional conditions. Conditions are evaluated in the order listed in the strategy. The condition at the top of the list is applied first, then the second condition, then the third condition, and so on.
- Click "Add" to add a new policy.