Dry goods 丨 Authority management and security of DolphinDB

DolphinDB database provides a powerful, flexible and secure access control system. The controller, as the authority management center, uses RSA encryption to encrypt key user information.

The main function:

Provide user and group roles to facilitate access control
Provide 8 types of permission control to adapt to various scenarios
Rich access control functions
Function view takes care of protecting data privacy and providing analysis results
Dynamic authentication of task scheduling and streaming data tasks to ensure system security
Use RSA to encrypt key user information
Support SSO, simplify login and facilitate system expansion

1. Permission overview

1.1 Users and groups

Users and groups are entities that carry permissions. A user can belong to multiple groups, and a group can also include multiple users.

Introducing the concept of groups, you can conveniently configure and manage permissions for users with the same permissions. The final actual permissions of a user are the permissions of the user itself plus the permissions of the group to which it belongs.

1.2 System Administrator

When the DolphinDB cluster is started for the first time, it will automatically create a system administrator with the user name "admin" and the password "123456". This administrator has all permissions, such as creating or deleting databases, reading and writing all tables, creating or deleting data tables in the database, using all function views, executable or test scripts. This administrator can create or delete other administrators. After other administrators have just been created, they can create or delete other administrators, users, or groups, but they don't have any other permissions. Administrators can authorize themselves or each other. Please note that the administrator "admin" cannot be deleted.

The administrator can use the functions or commands createUser, deleteUser, createGroup, ddeleteGroup, addGroupMember, dlelteGroupMember, getUserAccess, getUserList, getGroupList, resetPwd to perform convenient operations on users and groups.

1.3 Permission category

DolphinDB provides the following 8 types of permissions:

TABLE_READ: Read data from the specified data table
TABLE_WRITE: Write data to the specified data table
DBOBJ_CREATE: Create an object (data table) in the specified database
DBOBJ_DELETE: Delete the object (data table) in the specified database
VIEW_EXEC: Run function view
DB_MANAGE: Create and delete databases
SCRIPT_EXEC: Run script file
TEST_EXEC: execute unit test

Among them, the first 5 types need to provide operation objects, and the latter 3 types do not need to provide operation objects.

Please note that the databases and data tables involved in the permission configuration are all established in the Distributed File System (DFS).

1.4 Permission settings

Only the administrator can set permissions, and can only perform permission operations on the control node. The newly created user or group has not been granted or forbidden any permissions. Administrators can use grant/deny/revoke commands to set the permissions of users or groups. The 8 permissions in 1.3 can be used as the accessType parameter values of these three commands.

The following two examples illustrate the operation of permission setting.

Example 1

Administrator login:

login(`admin, `123456)

Create user NickFoles:

createUser("NickFoles","AB123!@")

Give user NickFoles the permission to read any DFS data table:

grant("NickFoles",TABLE_READ,"*")

Prevent user NickFoles from creating or deleting databases:

deny("NickFoles",DB_MANAGE)

Create the SBMVP group and add the user NickFoles to the group:

createGroup("SBMVP", "NickFoles")

Give the SBMVP group permission to create data tables in the databases "dfs://db1" and "dfs://db2":

grant("SBMVP",DBOBJ_CREATE,["dfs://db1","dfs://db2"])

Finally, the permissions of user NickFoles are: can access all data tables, cannot create or delete databases, and can create data tables on dfs://db1 and dfs://db2.

Example 2

User permissions can be easily set through groups:

createUser("EliManning", "AB123!@")  
createUser("JoeFlacco","CD234@#")  
createUser("DeionSanders","EF345#$")  
createGroup("football", ["EliManning","JoeFlacco","DeionSanders"])  
grant("football", TABLE_READ, "dfs://TAQ/quotes")  
grant("DeionSanders", DB_MANAGE)

This example creates 3 users and 1 group, and these three users belong to the group. Give this group of readable data tables dfs://TAQ?quotes permissions, and only give user DeionSanders permissions to create and delete databases.

Example 3

You can use grant or deny to grant or deny permissions to all objects (represented by *). For example, to give user JoeFlacco the permission to read any DFS data table:

grant("JoeFlacco",TABLE_READ,"*")

When using grant or deny all objects, you can only use revoke for all objects. If revoke is invalid for a single object:

revoke("JoeFlacco",TABLE_READ,"dfs://db1/t1")

The above command is invalid.

revoke("JoeFlacco",TABLE_READ,"*")

The above command cancels the permission of user JoeFlacco to read any DFS data table.

Similarly, after using grant or deny to grant or disallow permissions to a group, you can only use revoke on the regroup to cancel the permission setting. If revoke is used for a group member to cancel the permission setting, it is invalid.

1.5 Authority determination rules

The user's final authority is determined by the user's own authority and the authority of all groups to which it belongs. Different groups may conflict with the rules of a certain user and certain authority. The following are rules for determining permissions:

If a user is granted a certain permission in at least one group, and the permission is not prohibited in any group, the user has this permission.
If a user is forbidden with a certain permission in at least one group, even if the user is given this permission in other groups, the permission is also forbidden. In this case, if the user wants to obtain this permission, the administrator must use revoke or grant in all groups that prohibit the permission to cancel the permission prohibition, and the user is granted this permission in at least one group. Please note that in the above rules, for the convenience of description, the users themselves can also be regarded as a special group.

createUser("user1","123456")  
createUser("user2","123456")  
createGroup("group1")  
createGroup("group2")  
addGroupMember(["user1","user2"],"group1")
addGroupMember(["user1","user2"],"group2")
grant("user1",TABLE_READ,"*")  
deny("group1",TABLE_READ,"dfs://db1/t1")  
deny("group2",TABLE_READ,"dfs://db1/t2")

The result of the above three lines is that user user1 can read all data tables except "dfs://db1/t1" and "dfs://db1/t2".

grant("user2",TABLE_WRITE,"*")  
deny("group1",TABLE_WRITE,"*")  
grant("group2",TABLE_WRITE,"dfs://db1/t2")

The result of the above three lines is that users user1 and user2 cannot write data to any data table.

1.6 Permission control based on function view

The function view provides a flexible way to control the user's access to the data table, allowing users to obtain the information generated by the function view without giving the user permission to read all the original data in the data table. Only the system administrator has the authority to create and delete function views.

The administrator defines a function view:

def countTradeAll(){  
	return exec count(*) from loadTable("dfs://TAQ","Trades")  
}
addFunctionView(countTradeAll)  
grant("NickFoles",VIEW_EXEC,"countTradeAll")

countTradeAll()

Although user NickFoles does not have access to the table "dfs://TAQ/Trades", he can run the function view in this example to get the number of rows in the table.

The function view can also take parameters. The user can enter the parameters to obtain the corresponding result when using it. In the following example, we create a function view to get all transaction records of a certain stock on a certain day.

def getTrades(s, d){
	return select * from loadTable("dfs://TAQ","Trades") where sym=s, date=d
}
addFunctionView(getTrades)
grant("NickFoles",VIEW_EXEC,"getTrades")

getTrades("IBM", 2018.07.09)

2. Program scheduling and permission control in stream computing

Program scheduling and stream computing run in the background. In many cases, there is no explicit login, so permission verification is somewhat different from the case of user explicit login. Both types of background tasks are run as the user who created the task.

2.1 Schedule jobs permission setting

Program scheduling means that users specify to execute a series of tasks at a specific time and at a specific frequency, which is mostly used for batch processing business scenarios.

login("NickFoles","AB123!@")  
def readTable(){  
	read_t1=loadTable("dfs://db1","t1")  
	return exec count(*) from read_t1  
}  
scheduleJob("readTableJob","read DFS table",readTable,minute(now()),date(now()),date(now())+1,'D');

Regardless of whether NickFoles has the permission to read "dfs://db1/t1", the readTable task can be set successfully.

When the readTable task is actually running, if the user NickFoles has the permission to read "dfs://db1/t1", it will be executed successfully, otherwise the authentication will fail.

In addition, when using the deleteScheduleJob command, the system administrator can delete tasks created by other users, and non-administrator users can only delete tasks created by themselves.

2.2 streaming permission settings

When a user uses the subscribeTable function to subscribe to real-time data from a streaming data table and save it into the data table, he should confirm that he has the permission to write to this data table.

login("NickFoles","AB123!@")
def saveTradesToDFS(mutable dfsTrades, msg): dfsTrades.append!(select today() as date, * from msg)  
subscribeTable("NODE1", "trades_stream", "trades", 0, saveTradesToDFS{trades}, true, 1000, 1)

In the above example, the streaming data processing task saves the received data to the data table dfsTrades. When this stream data processing task is executed, the system will dynamically identify whether NickFoles has the permission to write to the data table dfsTrades, if not, the authentication will fail.

3. Use HTTPS to achieve secure communication

DolphinDB supports the use of HTTPS security protocol to communicate with the web.

3.1 Enable HTTPS configuration

Two ways to configure HTTPS:

Add enableHTTPS=true to the configuration file of the control node
Add -enableHTTPS true to the command line to start the control node

3.2 HTTPS certificate settings

DolphinDB uses a security policy of server-side certificate verification. By default, a self-made certificate will be generated, and the client needs to install the server certificate, otherwise the browser will prompt an insecure connection. Each physical server needs a certificate, so the control node and the proxy node need to generate a certificate, and the data node uses the certificate generated by the proxy node on the same physical server. Users can also purchase certificates certified by third parties.

3.2.1 Third-party certification

Rename the third-party certificate to server.crt and copy it to the keys folder in the home directory of the control node and agent node. If the keys folder does not exist, you need to create it manually. Since the third-party certificate is issued by a recognized authority, the browser trusts the certificate by default and does not need to be manually installed. This method is suitable for most application scenarios.

3.2.2 Install self-made certificate

When communicating within a small closed cluster, users can also use a self-made certificate for OPENSSL secure communication, but installing a self-made certificate is a bit cumbersome. The specific process is as follows:

Set publicName Because the domain name of the computer needs to be known to generate the certificate, set this option to the domain name of the computer for the physical server that needs to generate the certificate, which can be set in the command line or configuration file. The following is an example of a command line to start the control node in Linux. Here http://www.ABCD.com is the domain name of the computer where the control node is located.

./dolphindb -enableHTTPS true -home master -publicName www.ABCD.com -mode controller -localSite 192.168.1.30:8500:rh8500 -logFile  ./log/master.log

2. Check whether the certificate is generated correctly.
Start the control node and check whether there is a self-made certificate file server.crt and a private key file serverPrivate.key in the keys folder under the home directory .

3. Install the self-made certificate to the browser's credit certificate center. Different browser installation options are slightly different. Take Google Chrome as an example. Select Settings->Advanced->Manage certificates->AUTHORITIES->Import and import the server generated above. crt file.

Enter https://www.XXXX.com:8500/ in the browser to access the cluster manager. If a small green lock is displayed in the address bar of the browser, it means that the certificate is installed successfully and HTTPS access is possible.

4. Support SSO (Single Sign On)

In the cluster management interface, you can click on any data node to link to the notebook of that node. Jump from the control node to the data node, it is possible that a different physical server is accessed (cross-domain access). DolphinDB provides SSO so that users do not need to log in to the system again when accessing different physical servers during cluster operations.

DolphinDB provides two API functions for SSO:

getAuthenticatedUserTicket() Get the encrypted ticket of the currently logged in user
authenticateByTicket(ticket) Use the ticket obtained above to log in to the system

DolphinDB developers can easily and safely use these interfaces to extend the system.