WebDAV is an HTTP-based file sharing protocol. It was originally designed for collaborative editing of multi-user documents, and is also widely used in Internet-based file storage, data synchronization and other network disk application scenarios.
There are a large number of applications on mobile phones and PCs with built-in support for WebDAV, such as the well-known document management tool Zotero, the popular note-taking tool Notability on the iPad, the domestic office software WPS, the cross-platform Markdown note-taking tool Joplin, ES file manager, etc. , which means that as long as you have your own WebDAV storage, you can use WebDAV to save the document data we generated on these software.
In order to avoid confusion, it is necessary to clarify the concepts related to WebDAV. As shown in the figure below, WebDAV is divided into two parts: server and client:
- WebDAV server : The blue cloud represents the WebDAV server, which is used to respond to the client's connection and read and write requests, and store data.
- WebDAV client : various programs with built-in WebDAV client, running on mobile phones, tablets or computers.
After understanding the composition of WebDAV architecture, what we need to do is to configure a set of WebDAV servers of our own. In fact, we can also call it WebDAV sharing. In short, what we need to do is the "blue cloud" part in the architecture diagram.
In the past, web server software such as Apache HTTP service or Nginx was usually used to configure WebDAV sharing. Since it was not available out of the box, additional loading or manual compilation of WebDAV modules was required, which increased the complexity of configuring WebDAV to a certain extent.
Although some NAS operating systems also provide the WebDAV sharing configuration function, but limited by the intranet environment, it is difficult for the configured WebDAV sharing to take advantage of its Internet-based service provision.
If you are currently looking for a more general, more effective, and simpler WebDAV sharing service configuration solution, JuiceFS may be a solution worth trying for you.
JuiceFS is a cloud-oriented open source distributed file system, relying on object storage and database, it can quickly build a high-performance storage system with elastic capacity. JuiceFS provides POSIX, S3 API, HDFS API, Kubernetes CSI driver, Docker Volume Plugin, WebDAV and other rich access interfaces, which can easily access various applications.
Next, I will share with you how to use JuiceFS to easily and quickly configure a WebDAV service with space flexibility, security and practicality.
Preparation
The WebDAV share introduced in this article is an access interface provided by JuiceFS, which is built on the JuiceFS file system. So before configuration, you need to create a JuiceFS file system.
The JuiceFS file system consists of two parts: data storage and metadata engine. Data storage can be object storage services provided by various cloud computing platforms, or self-built MinIO or local disks. The metadata engine is used to store the metadata information of the file. It can use a network-based database such as Redis, MySQL, PostgreSQL, or a stand-alone database such as SQLite or BadgerDB.
In order to make the WebDAV share accessible over the Internet, we will create it next on a cloud server with a fixed public IPv4 address. Because WebDAV is created in one place, it can be accessed from anywhere by IP address or domain name. Therefore, the data storage and metadata engine that make up JuiceFS can be more flexibly matched, such as:
- Local disk + stand-alone database: When the available space of the local disk of the cloud server is sufficient.
- Object storage + stand-alone database: When a larger capacity storage space is required.
- Object storage + network database: When larger storage space is required and other hosts need to access JuiceFS at the same time.
Cloud Server
Suppose we have a cloud service configured as follows:
- System: Ubuntu Server 22.04 AMD64
- CPU: 1 core
- Memory: 1GB
- Hard Disk: 25GB
- IP address: 120.118.8.10 (this is a randomly written FAKE IP for demonstration purposes)
For the scenario described in this article, an entry-level cloud server of any cloud platform can meet the needs, and the key point is to have a public network IPv4 address.
object storage
Suppose we have prepared the following buckets:
- Storage type: Alibaba Cloud OSS
- Bucket URL: https://myjfs.oss-cn-shanghai.aliyuncs.com
- Access Key: abcdefg
- Secret Key: gfedcba
For the scenario described in this article, object storage is not necessary. If the hard disk space of the cloud server is sufficient, the hard disk can be used as the object storage of JuiceFS.
database
For the scenario described in this article, the stand-alone database is the easiest choice, here we use a single-file SQLite database. No need to prepare in advance, it will be automatically generated when the JuiceFS file system is created.
If necessary, you can also refer to the document " How to set up the metadata engine " to use other types of databases.
Create JuiceFS file system
- Install the client
curl -sSL https://d.juicefs.com/install | sh -
- create file system
The following are two ways to create a file system, choose one according to actual needs:
Method 1: Use a local hard disk as object storage
juicefs format sqlite3://myjfs.db myjfs
Method 2: Use object storage service (data is stored in object storage)
juicefs format --storage oss \
--bucket https://myjfs.oss-cn-shanghai.aliyuncs.com \
--access-key abcdefg \
--secret-key gfedcba \
sqlite3://myjfs.db myjfs
Deploy the WebDAV service
Once the JuiceFS file system is ready, you can start to configure WebDAV sharing. The easiest way is to directly open the JuiceFS storage in the form of a WebDAV interface without making any settings.
sudo juicefs webdav sqlite3://myjfs.db 120.118.8.10:80
Use any built-in WebDAV client software to access http://120.118.8.10 to connect and access.
Although it is very convenient, it is not difficult to see that this anonymous access method has great security risks. First, anyone who knows the IP address can read and write our files directly through the WebDAV client without identity authentication; second, use Unencrypted HTTP protocol, the communication process is easy to be eavesdropped and attacked. It is very necessary for us to improve the deficiencies in these two aspects. In fact, it is very simple. To solve the first problem, we only need to set up identity authentication for WebDAV, and to solve the second problem, we only need to configure the SSL certificate to enable HTTPS encrypted connection support.
Set up authentication
JuiceFS v1.1 (as of the publication of this article, this version has not been officially released) and above versions start to support WebDAV authentication and SSL certificate functions, please confirm your client version before setting, it is recommended to refer to "Manually Compile JuiceFS Client " Compile the main branch of the GitHub repository.
Setting up a username and password for WebDAV access is as simple as setting the corresponding environment variables:
export WEBDAV_USER=user
export WEBDAV_PASSWORD=mypassword
SSL certificate
Enabling HTTPS encrypted connections for WebDAV requires an SSL certificate, either a trusted certificate issued by a CA or a self-signed certificate issued by OpenSSL.
CA organizations usually only issue certificates for domain names, so you need to own a domain name and bind it to the server IP. Free certificate issuing tools include certbot, acme.sh, etc. You can use these tools to apply for free SSL certificates.
Applying for and issuing a certificate through a third-party CA certificate authority is another topic. For simplicity, a self-signed certificate is used to achieve this function.
- Generate server private key
openssl genrsa -out private.key 4096
- Generating a Certificate Signing Request (CSR) This step requires a series of information to be provided interactively.
openssl req -new -key private.key -out client.csr
- Issue a certificate using a CSR
openssl x509 -req -days 365 -in client.csr -signkey private.key -out client.crt
- perform cleanup
rm client.csr
After the above steps, there are two files, private.key and client.crt, in the current directory, and then use them to run the WebDAV service.
Officially run WeDAV service
You can now combine the above to run a WebDAV service with basic authentication and an HTTPS encrypted connection:
export WEBDAV_USER=user
export WEBDAV_PASSWORD=mypassword
sudo juicefs webdav --cert-file ./client.crt --key-file ./private.key sqlite3://myjfs.db 120.118.8.10:443
At this point, we have enabled the HTTPS encrypted connection, and we need to use the address with https when accessing WebDAV https://120.118.8.10. At the same time, we need to enter the user name and password to successfully connect.
In addition, since the self-signed certificate is used here, some clients may issue a warning that the certificate is not trusted during access, just ignore it.
If you have bound a domain name to the server and applied for an SSL certificate, the command to deploy WebDAV should be adjusted as follows:
export WEBDAV_USER=user
export WEBDAV_PASSWORD=mypassword
sudo juicefs webdav --cert-file ./your-domain.crt --key-file ./your-domain.key sqlite3://myjfs.db www.your-domain.com:443
Correspondingly, the access address should also be changed to your domain name, for example: https://www.your-domain.com . Using the certificate issued by the CA institution, various WebDAV client accesses will not issue warnings, which can effectively solve the problem that some applications cannot connect to the WebDAV service.
Application example
Taking ES file browser and Joplin as examples to introduce how to configure and use the self-built WebDAV service to synchronize document data.
Note: When setting up WebDAV in some applications, if the address and identity authentication information are entered correctly, the program still reports that it cannot connect to WebDAV. This may be because the program does not support self-signed certificates. Trusted certificates to configure the WebDAV service to resolve this issue.
ES file browser
ES file browser is a commonly used third-party file manager on the mobile terminal. In addition to being able to manage files in the mobile phone, it also supports adding many third-party storage services, including WebDAV storage.
Just add a WebDAV type of storage in the service menu, and fill in the address, user name and password as shown in the figure below.
Joplin
Joplin is an open source Markdown note-taking software that supports synchronizing documents using WebDAV. Just enter your WebDAV link, username and password in the sync settings.
It should be noted that here we are using a self-signed certificate, and Joplin will report an error when checking the synchronization configuration. The solution is to expand the advanced options, check "Ignore TLS certificate errors", and the application settings can pass the verification after trying to check.
write at the end
Like FTP, WebDAV is a relatively old file sharing protocol, but they are still widely used today. There is never the best tool in the field of information technology, only the most suitable tool. In terms of functions, WebDAV may not have as many functions as a dedicated network disk client, but the WebDAV protocol is more general and open, and can be used directly on a large number of built-in supported applications, which can not only protect your privacy, but also avoid being hacked. Specific platform bundles.
WebDAV is just one of the many access interfaces supported by JuiceFS. If you are interested, you can further explore other access interfaces, such as POSIX accessed in the form of local disk, S3 gateway accessed in the form of S3 API, and Docker Volume Plugin for containerized access , Kubernetes CSI driver, and more. Using these access methods flexibly can achieve more efficient and flexible cloud storage management capabilities. We will share more about related content in subsequent articles.
If you are helpful, please pay attention to our project Juicedata/JuiceFS ! (0ᴗ0✿)