[Docker 2] docker network mode, network communication, data management, resource control

Similar to a switch, it can only forward the same network segment and find the MAC address of the target device through flood broadcast. learning mode

2. Docker network implementation principle:

Bridge mode is a network mode, and the way it works in Docker can be divided into the following steps:

1. Virtual network creation:When you start the Docker daemon, Docker creates a virtual network bridge (often called Docker0) is a virtual network device, similar to a physical network device switch

2. Assign a unique IP address:Every time you run a container, Docker assigns a unique IP address to the container< /span>

This IP address is in a subnet of a bridged mode network

3. Connect the container to the bridge network:When the container starts, Docker connects the container's virtual network interface to the virtual network bridge. One of the endpoints is on the container and the other endpoint is on the host

4. Communication between containers:If there are multiple containers running on the same bridge network, they can communicate through their respective IP addresses. Address direct communication. Docker will automatically set up routing on the bridged network so that the container can communicate with the external network, and the external network sees the IP address of the host

5. NAT (Network Address Translation):By default, Docker uses NAT technology to map the container’s private IP address to the host public IP address. In this way, the container can communicate with the outside world, and what the external network sees is the IP address of the host

Problem: Docker's network bridge is virtualized by the host machine and is not a real network device. It cannot be addressed by the external network. The external network cannot directly access the IP address assigned to the container by the docker:0 virtual bridge.

The Docker bridge is virtualized by the host and is not a real network device. The external network cannot be addressed, that is, the container IP cannot be directly accessed to access the container.

If the container wants to be accessible from the outside, it can map the container port to the host (port mapping)

That is, when Docker run creates a container, use -p/-P to enable it. When accessed, the container is accessed through the host IP + container port.

Do port mapping for the container:

-P: When creating a container, do a port mapping between the host and the container. Not followed by a number means that a port is randomly assigned to the host, and the port of the container remains unchanged.

-P32768 starts

-P (capital): randomly formulated

-p 80:80 (lowercase): Specify the port

Enter the container and start the service

In fact, docker performs an iptables NAT address translation between the host and the container.

docker run -itd --name test1 -P nginx:1.22.0 /bin/bash

-p 80:80

The first 80 is the port of the host, and the second is the port 80 of the container.

docker run -itd --name test3 -p 80:80 nginx:1.22.0 /bin/bash

When doing port mapping, the host port must be unoccupied.

The host directly views the logs generated by the container:

docker run -itd --name test11 -p 456:80 nginx

It must be run without adding /bin/bash

docker logs container name/container id view all logs

docker logs -f container name/container id tail view all, dynamic view

docker logs --tail=10 -f container name/container id tail 10 entries, dynamic view

/bin/bash is to allow the container to have a running program in the background to ensure that the container will not exit.

-d background daemon runs, but the container automatically exits after a long time

Adding /bin/bash is the standard output of the container. The docker logs capture the standard output of cmd and entrypoint. /bin/bash and capture logs conflict

3. Docker network mode:

3.1. Bridge mode:

There is no need to specify the network type when creating docker. The default is bridge.

docker run -itd --name test1 -p 4300:80 nginx:1.22.0

4300 is the host port

80 is the port of the nginx container

Accessing the host's IP+port is equivalent to accessing the nginx container

When using docker run -p, docker actually makes DNAT rules in iptables to implement the port forwarding function.
You can use iptables -t nat -vnL to view.

3.2. Host mode:

The container will not virtualize its own network card, nor will it have its own IP address. All use the host’s IP address and port

host:The container and the hostshare a network namespace

Specify the network mode when creating the container: --network host

docker run -itd --name test1 --network host nginx:latest bin/bash

Public mode: If a single container is running, host mode can be used. The port of the container and the host port are the same, but there are multiple containers and cannot be accessed

3.3. Container mode:

container:Containers and containersshare a network namespace

If you want all containers to be up, you must follow /bin/bash

docker run -itd --name test1 --network host nginx:1.22.0 /bin/bash

Create a test2 and test1 using the same IP and port

docker run -itd --name test2 --network=container:test1 nginx:1.22.0 /bin/bash

How to start the service in the container:

cd /usr/bin

nginx start

nginx -s stop stop

Containers share ports but cannot be started at the same time. To start one, the other containers must be shut down.

In other words, only one shared port can be used

Start nginx of test1:

Stop nginx of test1

nginx -s stop

3.4. none mode:

The docker container has its own network-space, but this container does not have any network settings.

This container has no network card, no IP, no routing, only lo loopback network. In none mode, the container cannot be connected to the Internet (it is used for container function testing at work)

Unable to connect to the Internet and cannot access

A closed network can ensure the security of the container.

docker run -itd --name test3 --network none nginx:1.22.0 /bin/bash

3.5. Customized network mode:

We can create a self-defined network segment for docker

docker network ls

NETWORK ID: This is the unique network ID of the docker network

NAME: the name of the docker network

DRIVER: Network driver

SCOPE：

docker run -itd --name test6 --network bridge --ip 172.17.0.10 nginx:latest /bin/bash

When using the default docker:0 bridge, you cannot specify an IP address when creating a container. It can only be automatically assigned by the docker bridge.

Unless the network is user-defined, the IP address can be customized for the container.

Create a custom network:

You can customize the network first and then use the specified IP to run docker

docker network create --subnet=172.18.0.0/16 --opt "com.docker.network.bridge.name"="docker1" mynetwork

--opt "com.docker.network.bridge.name"="docker1": You don’t need to add it here, but the network card name is difficult to identify and the system will give you a complex name.

mynetwork: the name of the custom network

After you have a custom network segment, you can customize the IP address when creating a container.

docker run -itd --name nginx1 --network mynetwork --ip 172.18.0.10 nginx:latest /bin/bash

Customize the IP address for the container to 172.18.0.10

To customize the IP address, you must first customize the network bridge.

4. Docker network summary:

Docker network mode:

bridge mode: It is also the default mode of docker and does not need to be specified when creating
Host mode: The container will not have its own network device, but can share IP and ports with the host
Container mode: IP and port are shared between containers
None mode: There is no network device of its own, no network card, and no IP. There is only one local loopback address. 127.0.0.1. Can't connect to the Internet. Equivalent to a flight mode, you can only access yourself. Generally used to test the functionality of containers
Custom network mode: When creating a container, the default docker0 bridge cannot be used to define an IP address for the container. It can be automatically assigned. When creating a custom network, you can specify an IP address for the container.

How docker specifies port mapping when creating a container:
Docker does not have a secondary configuration mechanism and can only be specified when creating a container

-P (capital): Randomly specify the port

-p (lowercase): Customize the specified port

-p 32768:80 32768 is the port of the host, and the following 80 is the port of the container

View logs outside docker container

docker run -itd --name test11 -p 456:80 nginx

It must be run without adding /bin/bash

docker logs container name/container id view all logs

docker logs -f container name/container id tail view all, dynamic view

docker logs --tail=10 -f container name/container id tail 10 entries, dynamic view

/bin/bash is to allow the container to have a running program in the background to ensure that the container will not exit.

-d background daemon runs, but the container automatically exits after a long time

Adding /bin/bash is the standard output of the container. The docker logs capture the standard output of cmd and entrypoint. /bin/bash and capture logs conflict

2. Docker data management:

1. Data volume

1.1. Data sharing between container and host:

Data volume: It is a special directory used by the container. In the container, it is mapped to the directory of the host. The files in the directory can be modified between the host and the host, and both parties take effect simultaneously. It has no effect on mirroring. Data migration between host and container

MySQL 33066:3306

The host directory and the directory in the container are mounted (mapping relationship)

docker run -itd --name test1 -v /opt/test1:/opt/test centos:7 /bin/bash

test1 is the host directory

test is the container directory

docker run -itd --name test2 -v /opt/test2:/opt/test:ro centos:7 /bin/bash

Create a read-only mode, which can only read in the container

The files in the directory inside the container can only be viewed

1.2. Data sharing between containers:

test1 test2 can have one or more mapping directories to achieve mutual data transmission and data synchronization.

Data volume container: only provides a mount point and lets another party collect data

docker run -itd --name test11 -v /opt/data1 -v /opt/data2 centos:7 /bin/bash

docker run -itd --volumes-from test11 --name test12 centos:7 /bin/bash

test11 and test12 realize data sharing between containers

3. Docker network communication

1. Container interconnection:

Implement network communication between two containers

Old version method:

The first container is created:

docker run -itd -P --name test111 centos:7 /bin/bash

second container

docker run -itd -P --name test222 --link test111:test222 centos:7 /bin/bash

To download the net-tools tool in the centos container, you can use the Linux command

yum -y install net-tools

Need to ping each other and do mapping

New versions are specified with network:

Customize the network to realize mutual ping between network segments

docker run -itd -P --name test112 --network=mynetwork centos:7 /bin/bash

docker run -itd -P --name test113 --network=mynetwork centos:7 /bin/bash

You can ping each other in the same network segment without mapping.

It is recommended to use the new version

4. Docker network exercises:

1. Deploy a MySQL image, create a MySQL container, and use navicat to directly access MySQL in the container. MySQL host: 33066

MySQL container: 3306

2. Create a data volume. You can see the log files of MySQL in the container on the host machine

Create and run a mysql container and create a data volume:

docker run -itd --name mysql1 -v /opt/demo1:/opt/test -p 33066:3306 -e MYSQL_ROOT_PASSWORD=123456 mysql:5.7.20

Native MySQL remote login test:

navicat：

Drag out the my.cnf configuration file and modify it

docker cp mysql1:/etc/mysql/my.cnf /opt

Restart docker container

docker restart mysql1

5. Docker resource control:

Limit container use of host resources

Docker uses Linux self-guided cgroup for control

Control groups are a mechanism provided by the Linux kernel system that can limit, record, and isolate the physical resources used by process groups.

Docker uses this mechanism to control resources

cgroup itself is the infrastructure that provides functions and interfaces for grouping management of processes. Allocation control mechanism to achieve resource control.

Other resources are still isolated

1. CPU resource control:

Linux uses CFS (completely fair scheduler) to schedule the use of the CPU by each process. The scheduling period of CFS is 100ms.

We can also customize the scheduling cycle of the container. And within this cycle time, each container can use the CPU scheduling time.

--CPU-period sets the period for container scheduling CPU

--CPU-quota sets the time the container can use the CPU in each cycle

Both can be used together

cd /sys/fs/cgroup/cpu/docker/09103baf76369ad9f6a77576665c49b384b34d01ed1f09f1c6b091abfae27115

cpu.cfs_period_us: Indicates the length of the CFS scheduling period, in microseconds.

During each cycle, a container can use a specified proportion of CPU time. By default, the value of cpu.cfs_period_us is 100000 (100 milliseconds).

cpu.cfs_quota_us: Indicates the amount of CPU time that the container can use during the cpu.cfs_period_us period, also in microseconds.

It defines a quota relative to the period. If set to -1, there is no limit. If set to a positive value, represents the quota within the period.

For example, if cpu.cfs_quota_us is set to 50000, the container can use up to 50 milliseconds of CPU time in a cycle.

In the Linux CFS scheduler, the cpu.cfs_period_us parameter defines a period,

This cycle is actually the basic time unit used to schedule tasks (including containers).

However, the specific scheduling time is determined by the scheduler, and this time changes dynamically under normal circumstances.

The valid range of the CFS period is between 1ms and 1s. Then --cpu-period 1000-1000000

The quota time for the container to use the CPU must be greater than 1ms, and the value of --cpu-quota must be >=1000

cpu.cfs_quota_us: After scheduling the request, the kernel allocates the CPU time to the container according to the quota.

-1

If the configuration is -1, then there is no limit on the time the container uses the host CPU.

cpu.rt_period_us

100000

The length of the CFS scheduling cycle, in microseconds. In each cycle, the container can use a specified proportion of CPU time. By default, it is 100 milliseconds.

CFS scheduler, 100 milliseconds defines a cycle, within this cycle, the basic time unit for scheduling tasks (containers).

The container is scheduled to request CPU resources once every 100 milliseconds. Then the kernel allocates CPU resources to the container

Check the usage of host resources by running containers

docker stats container name/id

View the mapping relationship between the PID in the container and the host

docker top container name/id

1.1. Limit the time the container occupies the CPU:

Perform a CPU stress test

docker run -itd --name test1 centos:7 /bin/bash

docker exec -it test1 bash

because /opt/cpu.sh

#!/bin/bash

i=0

while true

let i++

done

chmod +x /cpu.sh

./cpu.sh

top #You can see that this script takes up a lot of cpu resources.

Set a 50% ratio to allocate the upper limit of CPU usage time

1. You can re-create a container and set limits

docker run -itd --name test6 --cpu-quota 50000 centos:7 /bin/bash

2. Change the cpu.cfs_quota_us configuration file to modify the CPU usage time

cd /sys/fs/cgroup/cpu/docker/3ed82355f81151c4568aaa6e7bc60ba6984201c119125360924bf7dfd6eaa42b/

echo 50000 > cpu.cfs_quota_us

The default is 100000 microseconds, 100 milliseconds corresponds to 100% usage time

Change it to 50000 microseconds, which is 50%

docker exec -it 3ed82355f811 /bin/bash

./cpu.sh

top #You can see that the cpu occupancy rate is close to 50%, and cgroups has an effect on cpu control.

In a multi-core scenario, if the container process is allowed to completely occupy two CPUs,

Then you can set cpu-period to 100000 (that is, 0.1 seconds) and cpu-quota to 200000 (0.2 seconds).

docker top test checks the mapping relationship between the PID in the container and the host.

UID is the username.

PID is the PID of the process on the host system.

PPID is the PID of the process inside the container.

STIME is the start time of the process.

TTY is the terminal type and number.

TIME is the running time of the process.

CMD is the command of the process

1.2. Set the CPU resource usage ratio ( is only valid when multiple containers are set up)

Docker specifies CPU shares through --cpu-shares. The default value is 1024 and the value is a multiple of 1024.

Create two containers, c1 and c2. If there are only these two containers, set the weight of the container so that the CPU resource ratios of c1 and c2 are 1/3 and 2/3.

--cpu-shares specifies the CPU shares occupied by the container. The default weight is 1024, and the set value can only be a multiple of 1024.

docker run -itd --name c1 --cpu-shares 512 centos:7

docker run -itd --name c2 --cpu-shares 1024 centos:7

--cpu-shares sets a relative weight for each container to use the CPU. A higher weight can use more CPU resources. However, if only one container is running, even if the weight is set, there is no other higher weight. Use weight to occupy resources. Containers with low weight are still not restricted.

Enter the containers separately and conduct pressure tests.

yum install -y epel-release

yum install -y stress

#stress is a tool used to simulate system load, which can test the stability of the system under high load conditions.

stress -c 4 #Generate four processes, each process repeatedly calculates the square root of random numbers

#View container running status (dynamic update)

docker stats

It can be seen that when the CPU performs time slice allocation, container c2 has twice the chance of obtaining the CPU time slice than container c1.

However, the result of the allocation depends on the running status of the host and other containers at that time. In fact, there is no guarantee that container c1 will definitely get the CPU time slice.

For example, if the process of container c1 is always idle, container c2 can obtain more CPU time slices than container c1.

In extreme cases, for example, if only one container is running on the host, even if its CPU share is only 50, it can monopolize the entire host's CPU resources.

Cgroups only take effect when the resources allocated by the container are in short supply, that is, when it is necessary to limit the resources used by the container.

Therefore, it is not possible to determine how much CPU resources are allocated to a container based solely on its CPU share.

The resource allocation results depend on the CPU allocation of other containers running at the same time and the running conditions of the processes in the container.

1.3. Set the container to bind the specified CPU

#First allocate 4 CPU cores to the virtual machine

docker run -itd --name test7 --cpuset-cpus 1,3 centos:7 /bin/bash

#Enter the container and perform a stress test

yum install -y epel-release

yum install stress -y

stress -c 4

#Exit the container, execute the top command and press 1 to check the CPU usage.

Summary: Limitations on CPU:

Containers occupy CPU time

The weight ratio of the CPU occupied by the container (multiple containers are required to be effective)

The number of CPU cores occupied by the container (bind the specified CPU cores to the container)

2. Limit the memory usage of containers:

The -m(--memory=) option is used to limit the maximum memory that the container can use

docker run -itd --name test8 -m 512m centos:7 /bin/bash

docker stats

If --memory-swap is set to 0 or not set, the swap size that the container can use is twice the -m value.

If the value of --memory-swap is the same as the value of -m, the container cannot use swap.

If the --memory-swap value is -1, it indicates that the memory used by the container program is limited.

There is no limit on the swap space that can be used (you can use as many swap containers as the host has).

3. Restrict the use of swap:

--memory-swap

docker run -itd --name test -m 512m --memory-swap=1g centos:7 /bin/bash

Limiting the use of swap must be used together with limiting memory

--memory-swap must be used with --memory.

Setting up swap has the following centralized situations:

1. -m 512m --memory-swap=1g Then the swap space that the container can actually use is 1g-512m=512m

2. If you do not set --memory-swap=1g and only have -m 512m, then the space used for swap is twice the value after -m, which is 1024m.

3. If the value of --memory-swap set is the same as the -m memory limit, the container cannot use swap space 512-512=0

4. If -m 512m --memory-swap=-1 is set to -1, the memory limit is still 512m, but the container’s use of swap space is not restricted.

4. Disk IO quota (understand):

Limit the container's read speed on disk:

--device-read-bps：

Limit the read speed bps (data volume) on a certain device, the unit can be kb, mb (M) or gb.

Remove:docker run -itd --name test11 --device-read-bps /dev/sda:1M centos:7/bin/bash

Limit the write speed of the container on disk:

--device-write-bps ：

Limit the writing speed bps (data volume) on a certain device, the unit can be kb, mb (M) or gb.

例:docker run -itd --name test12 --device-write-bps /dev/sda:1mb centos:7/bin/bash

Verify write speed through dd

dd if=/dev/zero of=test.out bs=1M count=10 oflag=direct

When using dd to obtain the empty character set, it is input from the file system cache, which is relatively fast. Disabling the file system cache and writing the data directly to the disk can test the device lines more realistically and simulate direct writing to the physical device. Condition

Limit the number of container reads:

docker run -itd --name test --device-read-iops /dev/sda:100 centos:7 /bin/bash

Limit read operations to 100 times per second

Limit the number of writes to a container:

docker run -itd --name test --device-write-iops /dev/sda:50 centos:7 /bin/bash

Limit write operations to 50 times per second

5. Clean up the disk space occupied by docker:

docker system prune -a

Delete a stopped container

Delete all unused bridge devices (docker1)

Delete all unused images

Delete the cache when creating the container and useless data volumes