09 | Resource isolation: Why do I need Namespace to build containers?
What is Namespace
Below is the Wikipedia definition of Namespace
Namespace is a function of the Linux kernel that partitions kernel resources so that a group of processes can see a group of resources, and another group of processes can see another group of resources. Namespace works by setting the same Namespace for a group of resources and processes, but these Namespaces refer to different resources. Resources may exist in multiple Namespaces. These resources can be process ID, host name, user ID, file name, name related to network access, and inter-process communication.
-
Set the same Namespace for a group of resources and processes
-
Namespace refers to different resources
Docker uses the Namespace feature of the Linux kernel to realize the isolation of the resources of each container, thereby ensuring that the container can only access the resources of its own Namespace.
The latest Linux 5.6 kernel provides 8 types of Namespace:
| Namespce name | effect |
|---|---|
| Mount（mnt） | Isolate mount point 2.4.19 |
| Process ID (pid) | Isolation process ID2.6.24 |
| Network (net) | Isolate network devices, port numbers, etc. 2.6.29 |
| Interprocess Communication (ipc) | Isolate System V IPC and POSIX message queues 2.6.19 |
| UTS Namespace(uts) | Isolate host name and domain name 2.6.19 |
| User Namespace (user) | Isolate users and user groups 3.8 |
| Control group (cgroup) Namespace | Isolate the Cgroups root directory 4.6 Time Namespace |
Although the Linux kernel provides 8 namespaces, the latest version of Docker only uses the first 6 of them, namely Mount Namespace, PID Namespace, Net Namespace, IPC Namespace, UTS Namespace, User Namespace.
Below, we will learn more about the functions of the six Namespaces used by Docker.
The role of various Namespaces
（1）Mount Namespace
Mount Namespace is the first Namespace implemented by the Linux kernel, which has been added since version 2.4.19 of the kernel. It can be used to isolate the mount points seen by different processes or process groups. In layman's terms, it is possible to see different mount directories in different processes . Using Mount Namespace can realize that only your own mount information can be seen in the container, and the mount operation in the container will not affect the host's mount directory.
Let's use an example to demonstrate Mount Namespace. Before the demonstration, let's get to know a command line tool unshare. unshare is a tool in the util-linux toolkit. CentOS 7 system has integrated this tool by default. The unshare command can be used to create and access different types of namespaces.
First, we use the following commands to create a bash process and create a new Mount Namespace:
$ sudo unshare --mount --fork /bin/bash
[root@centos7 centos]#
After executing the above command, we have created a new Mount Namespace on the host, and the newly created Mount Namespace has been added to the current command line window. Let me use an example to verify that creating a mount directory in an independent Mount Namespace does not affect the host's mount directory.
First create a directory under the /tmp directory
[root@centos7 centos]# mkdir /tmp/tmpfs
After creating the directory, use the mount command to mount a tmpfs type directory. The command is as follows:
[root@centos7 centos]# mount -t tmpfs -o size=20m tmpfs /tmp/tmpfs
Then use the df command to view the mounted directory information:
[root@centos7 centos]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 500G 1.4G 499G 1% /
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 16G 0 16G 0% /sys/fs/cgroup
tmpfs 16G 57M 16G 1% /run
tmpfs 3.2G 0 3.2G 0% /run/user/1000
tmpfs 20M 0 20M 0% /tmp/tmpfs
You can see that the /tmp/tmpfs directory has been mounted correctly. In order to verify that this directory is not mounted on the host, we open a new command line window and execute the df command to view the mount information of the host:
[centos@centos7 ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 16G 57M 16G 1% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/vda1 500G 1.4G 499G 1% /
tmpfs 3.2G 0 3.2G 0% /run/user/1000
From the above output, we can see that /tmp/tmpfs is not mounted on the host. It can be seen that the mount operation in our independent Mount Namespace will not affect the host.
From the above results, we can conclude that using the unshare command can create a new Mount Namespace, and the mount in the newly created Mount Namespace is completely isolated from the outside .
####（2）PID Namespace
The function of PID Namespace is to isolate the process. In different PID Namespaces, processes can have the same PID number. Using PID Namespace can realize that the main process of each container is process No. 1, while the processes in the container have different PIDs on the host. For example, the PID of a process on the host is 122, and the PID Namespace can be used to realize that the process sees the PID of 1 in the container.
Below we use an example to demonstrate the role of PID Namespace. First, we use the following command to create a bash process and create a new PID Namespace:
$ sudo unshare --pid --fork --mount-proc /bin/bash
After executing the above command, we created a new PID Namespace on the host, and added the newly created PID Namespace to the current command line window. Use the ps aux command in the current command line window to view the process information:
[root@centos7 centos]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 115544 2004 pts/0 S 10:57 0:00 bash
root 10 0.0 0.0 155444 1764 pts/0 R+ 10:59 0:00 ps aux
Through the output of the above command, you can see that bash is the No. 1 process under the current Namespace, and we can't see other process information on the host.
（3）UTS Namespace
UTS Namespace is mainly used to isolate host names. It allows each UTS Namespace to have an independent host name. For example, our host name is docker. Using UTS Namespace, the host name in the container can be gavindocker or any other custom host name.
Similarly, we use an example to verify the role of UTS Namespace. First, we use the unshare command to create a UTS Namespace:
$ sudo unshare --uts --fork /bin/bash
After the UTS Namespace is created, the current command line window is already in an independent UTS Namespace. Next, we use the hostname command (hostname can be used to view the host name) to set the host name:
root@centos7 centos]# hostname -b gavindocker
Then check the host name again:
[root@centos7 centos]# hostname
gavindocker
Through the output of the above command, we can see that the host name in the current UTS Namespace has been modified to gavindocker. Then we open a new command line window and use the same command to check the hostname of the host:
[centos@centos7 ~]$ hostname
centos7
（4）IPC Namespace
IPC Namespace is mainly used to isolate inter-process communication. For example, PID Namespace and IPC Namespace can be used together to realize that processes within the same IPC Namespace can communicate with each other, but processes with different IPC Namespaces cannot communicate.
Similarly, we use an example to verify the role of IPC Namespace. First, we use the unshare command to create an IPC Namespace:
$ sudo unshare --ipc --fork /bin/bash
Next, we need to use two commands to verify the IPC Namespace.
-
ipcs -q command: used to view the list of communication queues between systems.
-
ipcmk -Q command: used to create a communication queue between systems.
We first use the ipcs -q command to view the list of system communication queues under the current IPC Namespace:
[centos@centos7 ~]$ ipcs -q
------ Message Queues --------
key msqid owner perms used-bytes messages
From the above, we can see that there is currently no system communication queue, and then we use the ipcmk -Q command to create a system communication queue:
[root@centos7 centos]# ipcmk -Q
Message queue id: 0
Use the ipcs -q command again to view the list of system communication queues under the current IPC Namespace:
[root@centos7 centos]# ipcs -q
------ Message Queues --------
key msqid owner perms used-bytes messages
0x73682a32 0 root 644 0 0
You can see that we have successfully created a system communication queue. Then we open a new command line window and use the ipcs -q command to view the system communication queue of the host:
[centos@centos7 ~]$ ipcs -q
------ Message Queues --------
key msqid owner perms used-bytes messages
Through the above experiment, it can be found that the system communication queue created in a separate IPC Namespace cannot be seen on the host. That is, IPC Namespace realizes the isolation of system communication queues.
（5）User Namespace
User Namespace is mainly used to isolate users and user groups. A typical application scenario is that a process running as a non-root user on the host can be mapped to the root user in a separate User Namespace. Using User Namespace can realize that the process has root privileges in the container, but it is just an ordinary user on the host.
User Namesapce can be created without root authority. Below we create a User Namespace as a normal user, the command is as follows:
[centos@centos7 ~]$ unshare --user -r /bin/bash
CentOS7 defaults to allow the creation of User Namespace to 0. If the execution of the above command fails (the error returned by the unshare command is unshare: unshare failed: Invalid argument), you need to use the following command to modify the number of User Namespaces allowed to be created by the system. The command is: echo 65535> /proc/sys/user/max_user_namespaces, and then try to create User Namespace again.
Then execute the id command to view the current user information:
[root@centos7 ~]# id
uid=0(root) gid=0(root) groups=0(root),65534(nfsnobody) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
From the above output, we can see that we are already the root user in the new User Namespace. Below we use the reboot command that only the root user of the host can execute to verify, execute the reboot command in the current command line window:
[root@centos7 ~]# reboot
Failed to open /dev/initctl: Permission denied
Failed to talk to init daemon.
As you can see, although we are the root user in the newly created User Namespace, we do not have the authority to execute the reboot command. This shows that in the isolated User Namespace, the root authority of the host cannot be obtained, which means that the User Namespace realizes the isolation of users and user groups.
（6）Net Namespace
Net Namespace is used to isolate information such as network devices, IP addresses, and ports. Net Namespace allows each process to have its own independent IP address, port and network card information. For example, the host IP address is 172.16.4.1, and an independent IP address can be set to 192.168.1.1 in the container.
Also use example verification, we first use the ip a command to check the network information on the host:
As you can see, there are network devices such as lo, eth0, docker0, etc. on the host, and our newly created Net Namespace is different from the network devices on the host.
Why does Docker need Namespace?
The Linux kernel added Mount Namespace from version 2.4.19 in 2002, and it wasn't until the kernel version 3.8 added User Namespace that it provided sufficient support for containers.
When Docker creates a new container, it creates these six Namespaces, and then adds the processes in the container to these Namespaces, so that the processes in the Docker container can only see the system resources in the current Namespace.
It is precisely because Docker uses these Namespace technologies of Linux that the isolation of Docker containers is realized. It can be said that without Namespace, there is no Docker container.
to sum up
- Namespace is a feature of the Linux kernel, which can isolate resources such as process ID, host name, user ID, file name, network, and inter-process communication in the same host system. Docker combines the functions of these six Namespaces to give birth to the Docker container.
- The core of resource isolation is to set the same Namespace for a group of resources and processes. Namespace refers to different resources.
- When the docker run --net=host command starts the container, it does not create the Net Namespace, but shares it with the host. Therefore, Docker does not recommend using this.