JuiceFS data encryption principle

As a distributed file system, JuiceFS deals with massive amounts of data every day, so data security is particularly critical. Today, I will introduce JuiceFS' efforts in data encryption.

Data encryption in transit

JuiceFS encrypts data as it travels over the network to prevent unauthorized users from eavesdropping on network communications.

JuiceFS clients always upload data to the object storage service using HTTPS, except in the following cases:

• Upload to Alibaba Cloud OSS using the internal endpoint
• Upload to UCloud US3 using internal endpoint

Data at rest encryption

JuiceFS supports data-at-rest encryption, which encrypts data before uploading to object storage. In this case, the data stored in the object storage will be encrypted, which can effectively prevent data leakage if the object storage itself is destroyed.

JuiceFS uses industry standard encryption methods (AES-GCM and RSA) in client-side encryption. Encryption and decryption are performed on the JuiceFS client. The only thing the user needs to do is to provide a private key or password when JuiceFS is mounted and use it like a normal filesystem. It is completely transparent to the application.

Note: Data cached on the client side is not encrypted. However, only the root user or owner can access this data. If you want to encrypt the cached data as well, you can put the cache directory on an encrypted filesystem or block storage.

Encryption and Decryption Methods

A global RSA key must be created for each encrypted file system M. Every object held in the object store will have its own random symmetric key S. Data is encrypted Swith and withS a global RSA key encrypted with a user-specified password.M

The detailed process of data encryption is as follows:

• Data blocks are compressed using LZ4 or ZStandard before being written to object storage.
• Generate a random 256-bit symmetric key Sand N.
• Each block is encrypted using Sand .N
• Encrypt the symmetric key with an RSA key Mto Sobtain the ciphertext K.
• The encrypted data, ciphertext Kand random seed are Ncombined into an object and written to object storage.

The steps for data decryption are as follows:

• Read the entire encrypted object (it may be a bit larger than 4MB).
• Parse the object data to get ciphertext K, random seed Nand encrypted data.
• Decrypt with RSA key Kto get symmetric key S.
• Decrypt the data using Sand Nget the data block plaintext.
• Decompress the data block.

key management

The security of the RSA key is extremely important when enabling encryption. If the key is compromised, it could lead to a data breach. If the key is lost, all encrypted data is lost and cannot be recovered.

When juicefs formatcreating a new volume with , you can --encrypt-rsa-keyenable encryption at rest by specifying the RSA private key, which will be saved to Redis. When the private key is password protected, an environment variable can be used JFS_RSA_PASSPHRASEto specify the password.

Instructions:

Generate RSA key

$ openssl genrsa -out my-priv-key.pem -aes256 2048

Provide this key when formatting

$ juicefs format --encrypt-rsa-key my-priv-key.pem META-URL NAME

Note: If the private key is password protected, it juicefs mountshould JFS_RSA_PASSPHRASEbe specified with when executing .

performance

TLS, HTTPS, and AES-256 are implemented very efficiently in modern CPUs. Therefore, enabling encryption has little effect on the performance of the file system. The RSA algorithm is relatively slow, especially the decryption process. A 2048-bit RSA key is recommended for storage encryption. Using 4096-bit keys can have a significant impact on read performance.

Summarize

The file system encryption technology can be applied to almost any file system-based database storage encryption requirements. This article introduces the process of data encryption and decryption in JuiceFS from the principle to the actual operation of how to generate and use the key. In the future, JuiceFS will continue to work hard to escort your data security.

Recommended reading: Zhihu x JuiceFS: Using JuiceFS to Accelerate Flink Container Startup

If it is helpful, please follow our project Juicedata/JuiceFS ! (0ᴗ0✿)

{{o.name}}

{{m.name}}