As a distributed file system, JuiceFS deals with massive amounts of data every day, so data security is particularly critical. Today, I will introduce JuiceFS' efforts in data encryption.
Data encryption in transit
JuiceFS encrypts data as it travels over the network to prevent unauthorized users from eavesdropping on network communications.
JuiceFS clients always upload data to the object storage service using HTTPS, except in the following cases:
- Upload to Alibaba Cloud OSS using the internal endpoint
- Upload to UCloud US3 using internal endpoint
Data at rest encryption
JuiceFS supports data-at-rest encryption, which encrypts data before uploading to object storage. In this case, the data stored in the object storage will be encrypted, which can effectively prevent data leakage if the object storage itself is destroyed.
JuiceFS uses industry standard encryption methods (AES-GCM and RSA) in client-side encryption. Encryption and decryption are performed on the JuiceFS client. The only thing the user needs to do is to provide a private key or password when JuiceFS is mounted and use it like a normal filesystem. It is completely transparent to the application.
Note: Data cached on the client side is not encrypted. However, only the root user or owner can access this data. If you want to encrypt the cached data as well, you can put the cache directory on an encrypted filesystem or block storage.
Encryption and Decryption Methods
A global RSA key must be created for each encrypted file system M
. Every object held in the object store will have its own random symmetric key S
. Data is encrypted S
with and withS
a global RSA key encrypted with a user-specified password.M
The detailed process of data encryption is as follows:
- Data blocks are compressed using LZ4 or ZStandard before being written to object storage.
- Generate a random 256-bit symmetric key
S
andN
. - Each block is encrypted using
S
and .N
- Encrypt the symmetric key with an RSA key
M
toS
obtain the ciphertextK
. - The encrypted data, ciphertext
K
and random seed areN
combined into an object and written to object storage.
The steps for data decryption are as follows:
- Read the entire encrypted object (it may be a bit larger than 4MB).
- Parse the object data to get ciphertext
K
, random seedN
and encrypted data. - Decrypt with RSA key
K
to get symmetric keyS
. - Decrypt the data using
S
andN
get the data block plaintext. - Decompress the data block.
key management
The security of the RSA key is extremely important when enabling encryption. If the key is compromised, it could lead to a data breach. If the key is lost, all encrypted data is lost and cannot be recovered.
When juicefs format
creating a new volume with , you can --encrypt-rsa-key
enable encryption at rest by specifying the RSA private key, which will be saved to Redis. When the private key is password protected, an environment variable can be used JFS_RSA_PASSPHRASE
to specify the password.
Instructions:
Generate RSA key
$ openssl genrsa -out my-priv-key.pem -aes256 2048
Provide this key when formatting
$ juicefs format --encrypt-rsa-key my-priv-key.pem META-URL NAME
Note: If the private key is password protected, it
juicefs mount
shouldJFS_RSA_PASSPHRASE
be specified with when executing .
performance
TLS, HTTPS, and AES-256 are implemented very efficiently in modern CPUs. Therefore, enabling encryption has little effect on the performance of the file system. The RSA algorithm is relatively slow, especially the decryption process. A 2048-bit RSA key is recommended for storage encryption. Using 4096-bit keys can have a significant impact on read performance.
Summarize
The file system encryption technology can be applied to almost any file system-based database storage encryption requirements. This article introduces the process of data encryption and decryption in JuiceFS from the principle to the actual operation of how to generate and use the key. In the future, JuiceFS will continue to work hard to escort your data security.
Recommended reading: Zhihu x JuiceFS: Using JuiceFS to Accelerate Flink Container Startup
If it is helpful, please follow our project Juicedata/JuiceFS ! (0ᴗ0✿)