docker volumes--docker volumes usage learning

1. Manage data in Docker

1.1. How Docker saves container data

• Docker volume ( Docker Volumes): You can save data in a Docker volume, so that you can share data between the container and the host, and ensure that the data in the container will not be lost when the container is deleted. Docker volumes can be used to save application configurations, logs, databases and other data, and can also be used to share data between multiple containers.

• Docker mount point ( Docker Mounts): Directories or files in the host file system can be mounted into the container. The data in the host file system can be shared to the container through the mount point and persisted.

• Docker data volume container ( Docker Data Container): You can create a special container dedicated to storing data volumes. Mount this data volume container in other containers to share data among different containers.

• Docker backup and restore ( Docker Backup and Restore): You can use Docker's backup and restore feature to save container data. Use Docker commands to back up container data into a tar package, and then restore the tar package to another container.

All of the above methods can be used to save the data in the Docker container and make it persistent. Which method to use depends on the application scenario and needs.

1.2. Three different mounting methods of Docker

• bind mount: Mount the host's files or directories directly into the container. This mounting method allows the container to access the file system on the host, and also allows the host to access the file system in the container. bind mount mounts files or directories and supports read and write modes.

• volumes: It is a method of persisting data in a Docker container, which can save data in a Docker volume and share data between the container and the host. Volumes mount is for volumes and supports read-write mode.

• tmpfs mount: Mount the host's tmpfs file system into the container. The tmpfs file system is a memory-based file system . Using tmpfs mount can create a temporary file system in the container. The data of this file system will be stored in the memory and will not be written to the host disk. tmpfs mount is also mounted for directories and supports read and write modes.

In general, bind mount is for the host file system, while volumes and tmpfs mount are for Docker volumes and memory file systems. The bind mount mount is suitable for development environments and continuous integration environments, and the volumes and tmpfs mount mounts are suitable for production environments and distributed environments.

2. Volume

2.1. Volume overview

Docker Volumes are a method of persisting data in Docker containers. It allows sharing data between containers and hosts and ensures that the data in the container will not be lost if the container is deleted. Docker volumes can be used to save application configurations, logs, databases and other data, and can also be used to share data between multiple containers.

Volumes provide the ability to connect a container's specific file system path back to the host . Simply put, it maps the container's directory to the host. If a directory in the container is mounted, changes in that directory will also be seen on the host. If we mount the same directory when the container is restarted, we will see the same files, which is the persistence and synchronization operation of the container.

• The Docker Volume command allows the container to read files from the host, or persist data from the container to the host, allowing the container to be separated from the data generated by the container. A container can mount multiple different directories.
• The life cycle of a Volume is independent of the life cycle of the container . Even if the container is deleted, the Volume will be retained. Docker will not automatically recycle the Volume because it is not used by the container.
• Within the container, adding or modifying files in this folder will not affect the container's union file system.

Data volume mapping

When the container is running, there is a data storage space in the container, but when the container is closed, the content of this storage space will be lost. Therefore, the data cannot be saved long-term! So, what strategy is needed for how to store data for a long time? The answer is saved on the host machine, and does not hinder the reading and writing of the container.

Experimental verification of data loss without using the mounting method:

# 启动容器并新建文件
$ docker run -it nginx:latest

[root@31f3303bdc66 /]
$ cd /home/
$ vi test.txt   # 输入文字 “nihao”

# 退出容器查找容器数据保存目录，并验证文件是否存在
$ docker inspect 31f3303bdc66 | grep UpperDir
"UpperDir": "/var/lib/docker/overlay2/5c91c9d866f4071c18e4268929fd2e50ed37f554694f6e4932c30059c0f46330/diff",
$ ll /var/lib/docker/overlay2/5c91c9d866f4071c18e4268929fd2e50ed37f554694f6e4932c30059c0f46330/diff/home/
total 4
-rw-r--r-- 1 root root 7 Jun  6 10:26 test.txt

# 删除容器再次查看文件是否存在
$ docker rm 31f3303bdc66
31f3303bdc66
$ ll /var/lib/docker/overlay2/5c91c9d866f4071c18e4268929fd2e50ed37f554694f6e4932c30059c0f46330/diff/home/
ls: cannot access /var/lib/docker/overlay2/5c91c9d866f4071c18e4268929fd2e50ed37f554694f6e4932c30059c0f46330/diff/home/: No such file or directory

By establishing a virtual file path in the container, the physical address pointed to by this path is the file system of the host . At this time, when the container is suddenly closed, the file system on the host is still intact. This serves the purpose of permanent data protection.

Container data volume characteristics

A data volume is a special directory that can be used by one or more containers. It maps the host operating system directory directly into the container (that is, the data volume can map data directly into the container)

Features of data volumes:

• Data volumes can be shared and reused between containers
• Modifications to the data volume will take effect immediately
• Updates to the data volume will not affect the mirror
• The data volume will always exist by default, even if the container is deleted

2.2. Advantages of volumes

Compared with ordinary data mounting methods, Docker Volume has the following advantages:

• Supports the backup and recovery of data volumes: Docker Volume supports the backup and recovery of data volumes. The data in the data volumes can be backed up to local or remote storage for disaster recovery or data migration.
• Volumes can be managed directly using Docker CLI commands or the Docker API.
• Volumes can run on Linux and Windows containers.
• Volumes can be shared more securely between multiple containers.
• Support multiple types of data volumes: Docker Volume supports multiple types of data volumes, such as local storage volumes, remote storage volumes, cloud storage volumes, etc. This makes the management of data volumes more flexible, and different types of data volumes can be selected according to specific needs.
• The content of the new volume can be pre-populated by the container.
• Volumes on Docker Desktop have higher performance than bind mounts from Mac and Windows hosts.
• The life cycle of the data volume is independent of the container: Docker Volume can exist independently of the container, even if the container is deleted, the data in the Volume still exists. This makes the management of data volumes more flexible, and data can be shared between containers.
• Support for encryption of data volumes: Docker Volume supports encryption of data volumes, which can protect sensitive data and ensure data security.
• Support the expansion and contraction of data volumes: Docker Volume supports the expansion and contraction of data volumes, and can dynamically adjust the size of data volumes according to actual needs.

2.3. Use of volumes

2.3.1. Creation of volumes

Create local data volume by default

docker volume create test_volume

Other types of data volumes can also be created, as follows:

Remote data volume:

docker volume create --driver=remote --opt=remote-host=my-remote-host --opt=remote-path=/my-volume my-remote-volume

This command will create a remote storage data volume named my-remote-volume. The data volume will be stored in the /my-volume directory on the remote host. You need to specify the address and remote path of the remote host through the --opt parameter. You need to install the corresponding remote storage driver to use this command.

Install docker volume remote driver

Install and enable the Docker Volume driver named "remote" on the Docker host. You can install the "remote" driver through the following steps:
1. Open the Docker host terminal and log in using root or other accounts with administrator rights.
2. Run the following command to install the "remote" driver:

$ docker plugin install ssh://<remote-host>/path/to/remote-volume-plugin.tar.gz

Here you need to replace with the IP address or domain name of the remote host, and /path/to/remote-volume-plugin.tar.gzreplace with the path where the "remote" driver is stored on the remote host. If the "remote" driver is distributed via the Docker Registry, it can be installed using the following command:

$ docker plugin install <registry>/<repo>/<image>:<tag>

Here you need to <registry>/<repo>/<image>:<tag>replace with the correct image name and label.

3. After the installation is complete, run docker plugin lsthe command to check whether the "remote" driver has been successfully installed.

4. Run docker volume create --driver=remotethe command to create a remote storage data volume.

Cloud data volume:

docker volume create --driver=cloud --opt=cloud-provider=aws --opt=cloud-region=us-west-2 my-aws-volume

This command will create a cloud storage data volume named my-aws-volume. The data volume will be stored in the us-west-2 region of AWS. You need to specify the provider and region of the cloud storage service through the --opt parameter. You need to install the corresponding cloud storage driver to use this command.

Create tags

$ docker volume create test_volume_01 --label=app=test --label=use=test_function 

The meta information is as follows:

$ docker volume inspect test_volume_01
[
    {
    
    
        "CreatedAt": "2023-06-02T17:26:13+08:00",
        "Driver": "local",
        "Labels": {
    
    
            "app": "test",
            "use": "test_function"
        },
        "Mountpoint": "/var/lib/docker/volumes/test_volume_01/_data",
        "Name": "test_volume_01",
        "Options": {
    
    },
        "Scope": "local"
    }
]

2.3.2. Viewing volumes

docker volume lsView volume list

$ docker volume ls
DRIVER              VOLUME NAME
local               0ff4f6d947508d887de39e67f5e9ff24be875473756106aed4c7312ce9bffc8e
...
local               test_volume
local               test_volume_01

docker volume ls -qOnly display volume name

$ docker volume ls -q
0ff4f6d947508d887de39e67f5e9ff24be875473756106aed4c7312ce9bffc8e
...
test_volume
test_volume_01

docker volume ls -fMatch query:

docker volume ls -f name=test*
DRIVER              VOLUME NAME
local               test_volume
local               test_volume_01

docker volume inspectView volume metadata

$ docker volume inspect test_volume
[
    {
    
    
        "CreatedAt": "2023-06-02T16:59:00+08:00",
        "Driver": "local",
        "Labels": {
    
    },
        "Mountpoint": "/var/lib/docker/volumes/test_volume/_data",
        "Name": "test_volume",
        "Options": {
    
    },
        "Scope": "local"
    }
]

docker volume inspect -fGo language templates are also supported:

$ docker volume inspect -f {
    
    {
    
    ."Mountpoint"}} test_volume
/var/lib/docker/volumes/test_volume/_data

2.3.3. Use of volumes

-v

Use -v to start the container and view Mountssome of the content in the container information

docker run -d --name=con-test -v test_volume:/home nginx:latest /bin/bash


docker inspect con-test | grep -A 10 Mounts
        "Mounts": [
            {
    
    
                "Type": "volume",
                "Name": "test_volume",
                "Source": "/var/lib/docker/volumes/test_volume/_data",
                "Destination": "/home",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],

--mount

Use –mount to start the container and view Mountssome of the content in the container information

docker run -d --name=con-test-01 --mount source=test_volume_01,target=/home nginx:latest /bin/bash

docker inspect con-test-01 | grep -A 10 Mounts
        "HostConfig": {
    
    
            ...
            "Mounts": [
                {
    
    
                    "Type": "volume",
                    "Source": "test_volume_01",
                    "Target": "/home"
                }
            ],
        ...    
        }
        "Mounts": [
            {
    
    
                "Type": "volume",
                "Name": "test_volume_01",
                "Source": "/var/lib/docker/volumes/test_volume_01/_data",
                "Destination": "/home",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],

-vand --mountdifference

-v can only create bind mount
–mount is used to mount volume by default, but can also be used to create bind mount and tmpfs.
Comparison of creating bind mount and mounting volume

Comparative item	bind mount	volume
Source location	user specified	/var/lib/docker/volumes/
Source is empty	Override dest is empty	Keep dest content
Source is not empty	Overwrite dest content	Overwrite dest content
Source type	file or directory	It can only be a directory
portability	General (self-maintained)	Strong (docker hosting)
Host direct access	Easy (just chown)	Restricted (requires root user login)*

For detailed differences, please refer to: https://blog.csdn.net/inrgihc/article/details/109001886

2.3.4. Deletion of volumes

docker volume prune

Delete unused volumes

$ docker volume prune
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Volumes:
e6555d73c125e424f8fa71f8434933a9bd0c7668553fee5ae93f031db000c0c7

Total reclaimed space: 56.32kB

docker volume rm

Directly deleting a volume in use will result in an error:

$ docker volume rm test_volume_02
Error response from daemon: remove test_volume_02: volume is in use - [1b4f1ef946411b16fa6ea46c29099839ebfd6fd8d5210ea68be90975792bc635]

You need to delete the corresponding container and then delete the volume

$ docker stop 1b4f1ef946411b16fa6ea46c2
1b4f1ef946411b16fa6ea46c2


$ docker rm 1b4f1ef946411b16fa6ea46c2
1b4f1ef946411b16fa6ea46c2


$ docker volume rm test_volume_02
test_volume_02

2.4. Anonymous mounting and named mounting

In Docker, you can use anonymous mounting and named mounting to mount the host file system or Docker Volume into the container.

2.4.1. Anonymous mounting

Anonymous mounting means that when creating a container, the host file system or Docker Volume is directly mounted into the container without specifying the name of the mounting volume. The syntax format of anonymous mount is as follows:

docker run -v <host-path>:<container-path> image-name

or

docker run --mount type=<type>,source=<source>,destination=<destination> image-name

This method creates a temporary mount point within the container, whose name is automatically generated by Docker. Once the container is deleted, the mount point is also deleted.

For example, the following command anonymously mounts the /data directory on the host to the /mydata directory in the container:

docker run -v /data:/mydata nginx

or

docker run --mount type=bind,source=/data,target=/mydata nginx

2.4.2. Named mount

Named mounting means specifying a name for the mounted volume when creating a container. The syntax format of a named mount is as follows:

docker run -v <volume-name>:<container-path> image-name

or

docker run --mount type=<type>,source=<volume-name>,destination=<destination> image-name

This method mounts the host file system or Docker Volume into the container and creates a mount point with the specified name in the container. Once the container is deleted, the mount points and data will not be deleted and can continue to be used in other containers.

For example, the following command mounts a Docker Volume named my-volume to the /mydata directory in the container:

docker run -v my-volume:/mydata nginx

or

docker run --mount type=volume,source=my-volume,target=/mydata nginx

To sum up, anonymous mounting and named mounting are two ways to mount the host file system or Docker Volume into a container. Anonymous mounting is a fast and simple method, but it is inconvenient for data management and sharing; named mounting can facilitate data management and sharing, but requires manual creation and management of Docker Volumes.

2.5. Volume backup and restore

In Docker, you can use the docker volume command to back up and restore the data volume so that the data in the data volume can be restored when needed.

2.5.1. Backup data volume

Use the docker run command to create a container with the --rm parameter, mount the data volume to be backed up into the container, and use the tar command to package the data in the data volume into a tar file (in the container mounting directory). For example, The following command backs up the data volume named my-volume to the /backup directory:

docker run --rm -v my-volume:/data -v /backup:/backup busybox tar czvf /backup/my-volume.tar.gz /data

This command will create a temporary container, mount the my-volume data volume to the /data directory in the container, package the data in the data volume into a tar file, and save it to the /backup directory on the host. After the backup is complete, the temporary container is deleted.

2.5.2. Restore data volume

If you need to restore the data in the data volume, you can use the docker run command to create a container with the --rm parameter, mount the backup file into the container, and use the tar command to decompress the backup file into the data volume. For example, the following command restores the my-volume data volume from the my-volume.tar.gz file in the /backup directory:

docker run --rm -v my-volume:/data -v /backup:/backup busybox tar xzvf /backup/my-volume.tar.gz -C /data

This command will create a temporary container, mount the backup file /backup/my-volume.tar.gz to the /backup directory in the container, and extract the data in the backup file to the my-volume data volume. After data restoration is complete, the temporary container is deleted.

2.6. Sharing volumes between containers

Use docker run --volumes-fromthe command to mount one or more data volumes from an existing container to a new container when creating a container to share and transfer data between containers. The following is an example of using the docker run --volumes-from command:

First, assume that there is already a container named my-container, which uses a Docker Volume named my-volume to store some data. Now you need to create a new container to use the data in the my-volume data volume. You can use the following command:

docker run --volumes-from my-container --name my-new-container image-name

This command will create a new container named my-new-container and mount all data volumes used in the my-container container into my-new-container. In this way, the data in the my-volume data volume can be used in my-new-container without re-creating and initializing the data volume.

In addition to using container names for mounting, the docker run --volumes-from command also supports mounting using container IDs, container name prefixes, etc. For example, the following command mounts all data volumes in the container with ID 123456 into a new container:

docker run --volumes-from 123456 --name my-new-container image-name

It should be noted that when using the docker run --volumes-from command to mount a data volume, you should ensure that the mounted container and the newly created container are both on the same Docker host , otherwise data sharing and transfer will not be possible.

By using the docker run --volumes-from command, you can easily mount one or more data volumes from an existing container to a new container to realize data sharing and transfer between containers. This command can avoid repeated creation of data volumes and improve the reusability and scalability of the container.

reference documents

1、https://blog.csdn.net/inrgihc/article/details/109001886

2、https://www.cnblogs.com/wwchihiro/p/9316504.html

3、https://blog.csdn.net/weixin_46618592/article/details/126591142

4、https://blog.csdn.net/qq_35745940/article/details/119336510

5、https://blog.frognew.com/2021/07/relearning-container-23.html

6、https://blog.csdn.net/gongdiwudu/article/details/128756465

7、https://docs.docker.com/engine/reference/commandline/volume_create/