Docker common knowledge records

Docker—images and containers

• A good memory is not as good as a bad pen

About Docker Images

• The docker image is similar to a layered structure, and its bottom layer is based on the Linux Kernel structure.

• Each image includes the Linux kernel and the basic Centos/Ubuntu image, and a custom image can be built on top of it

• We can think of each mirror as a Linux server with our custom content

Image acquisition and image accelerator settings

• We can pull the image from the docker mirror warehouse source to our local machine for use
  • The mirror warehouse source is similar to the concept of maven warehouse
  • Use the docker search command to find images in the docker warehouse
  • We can set up our own mirror warehouse source by ourselves

vi /etc/docker/daemon.json
{
    
    
   "registry-mirrors": ["http://hub-mirror.c.163.com"]
}

or

{
    
    
   "registry-mirrors": ["https//orptaaqe.mirror.aliyuncs.com"]
}
# 重启docker服务  or  重启docker
systemctl restart docker.service

#  国内仓库镜像加速地址
	# Docker中国区官方镜像
https://registry.docker-cn.com
	# 网易镜像源
http://hub-mirror.c.163.com
  # 中国科技大学
https://docker.mirrors.ustc.edu.cn
	# 阿里云容器服务
https://cr.console.aliyun.com/

You can also build the image yourself through the Dockerfile

Dockerfile image creation

• Docker provides Dockerfile files to build images

Dockerfile syntax

# FROM  指定基础镜像
	# 例如指定ubuntu:14.04作为基础镜像
FROM ubuntu:14.04

# RUN  在镜像内部执行一些命令，例如：安装软件、配置环境、安装相关命令等等
RUN groupadd -r mysql && useradd -r -g mysql mysql

# ENV  设置变量的值，如下。可以通过 docker run --e key=value进行修改，可以使用  ${MYSQL_MAJOR} 来取值
ENV MYSQL_MAJOR 5.7

# LABEL  设置镜像标签
LABEL email="[email protected]" version="1.0"
LABEL name="zh2020yy"

# VOLUME   指定数据的挂载目录，改变外部文件夹的内容，会影响到容器内文件夹
VOLUME /var/lib/mysql

# COPY 将主机的文件复制到镜像内，如果目录不存在，会自动创建所需要的目录，注意只是复制，不会提取和解压
COPY demo.sh /usr/local/bin/

# ADD 将主机的文件复制到镜像内，和COPY类似，ADD命令会对压缩文件进行提取和解压
ADD demo.zip /etc/zh/

# WORKDIR  指定镜像的工作目录，之后的命令都是基于此目录工作，若不存在则创建
	# 这几个命令的意思是创建 /usr/local/tomcat  文件夹，接下来的命令在该目录下执行
WORKDIR /usr/local
WORKDIR tomcat
RUN touch test.txt

# CMD 容器启动的时候会执行的命令，若有多个CMD命令，则最后一个生效
CMD mysqld

# ENTRYPOINT  和CMD的使用类似
  #  和CMD命令不同的是，docker run执行时，会覆盖CMD的命令，而ENTRYPOINT不会被覆盖
ENTRYPOINT ["demo.sh"]

# EXPOSE  指定镜像要暴露的端口，启动镜像时，可以使用  -p  将该端口映射给宿主机
EXPOSE 3306

make a mirror image yourself

• First prepare a Java jar package by yourself and upload it to the specified location on the server

• Create Dockerfile

• Write Dockerfile

  • MAINTAINER: set mirror owner

FROM openjdk:8
MAINTAINER zh2020
LABEL name="dockerfile-demo" version="1.0" author="zh2020"
COPY spring_boot_docker-0.0.1-SNAPSHOT.jar dockerfileEx.jar
#  启动容器后执行的命令
CMD ["java", "-jar", "dockerfileEx.jar"]

• Build images based on Dockerfile

docker build -t my-docker-image .

• The build is successful and a new image is obtained

• Create container based on image

docker run -d --name zh -p 6666:8080 imageID

• Access through localhost:6666, verify

Mirror related operations

#  删除所有镜像
docker rmi -f $(docker image ls)
#  通过 容器创建镜像，假设我们有一个容器名字叫做   containerA
docker commit containerA newImageName

Mirror push

• There are many images in the mirror warehouse source, and we can also push the mirror images we created to the mirror warehouse
• Here take the official docker warehouse as an example https://hub.docker.com/ (it can also be a cloud server built by Alibaba Cloud or yourself)
• First register and log in. own account
• Log in to the official docker warehouse and create your own warehouse. After the warehouse is created, it will have its own warehouse name. If the warehouse name is myrepo and the account name is account
• Then the steps for image push are

# 首先进行登录
docker login --username=https://hub.docker.com/
# 输入用户名和密码 ，假设推送的镜像id为   a1b2c3
# 通过docker tag  给镜像打上标签  这里 v1  表示镜像的标签
docker tag a1b2c3 account/myrepo:v1
# 然后进行push
docker push account/myrepo:v1

Docker harbor

• In the same way, it is similar to maven private servers. Docker provides a way to build docker harbor, which can be used as a private server.

#  首先从  github  上下载  harbor项目，选择合适的版本
https://github.com/goharbor/harbor/releases
#  在一台安装了  docker-compose 的机器，上传并解压
tar -zxvf xxx.tar.gz
#  解压完成之后，进入到 harbor 目录，修改 harbor.cfg 文件，主要修改 IP 地址为当前机器的 IP 地址，同时可以修改 harbor 密码

#  安装  harbor
sh install.sh
#  最后浏览器访问 IP，输入用户名和密码即可

About Docker containers

• Run the image by docker run to create a container

container resource limit

• A running container occupies physical resources
• You can check the resource status through docker stats
• You can also use the free memory resource limit command of docker, as follows

memory limit

#  --memory     限制容器使用内存大小为 100M
docker run -d --memory 100M --name containerName imageID

CPU limit

#  --cpu--shares  权重,如下，这里配置权重为10
docker run -d --cpu--shares 10 --name containerName imageID 

Graphical resource monitoring

• The docker container resources can be monitored more intuitively through weaveworks scope
  • https://github.com/weaveworks/scope

sudo curl -L git.io/scope -o /usr/local/bin/scope
sudo chmod a+x /usr/local/bin/scope
scope launch ip

#  停止 scope
scope stop

Container common operations

#  删除所有的 container
docker rm -f $(docker ps -a)
#  进入一个容器当中
docker exec -it containerID  bash
#  查看  container 日志
docker logs containerID 
#  查看容器详情信息
docker inspect containerID
#  停止启动容器
docker stop/start containerID 

NIC and Docker

Relevant pre-knowledge

• Network related knowledge - Configure networking
• Seven-layer model of computer network
  • Advanced Layer, Transport Layer, Base Layer
• Data communication between computers, data packets are packaged and unpacked
  • Each computer has a network card, and communication is established through the network card
  • Network in docker, network card information
  • Network card information can be viewed through ipconfig
• Definition of network card
  • In the computer network, the hardware support in the computer to be able to communicate
  • Each network card will have a specific and unique Mac address

• How to view the machine network card through the command line

#  以文件的形式去查看机器的网卡
ls /sys/class/net
#  查看机器网卡
ip a
ip link show

Interpretation of ip a

• link/ether: MAC address
• inet: Binding IP address

The network card is the configuration file

• In Linux, the network card corresponds to the file, and the corresponding network card file can be found through the following methods

cat /etc/sysconfig/network-scripts/ifcfg-eth0

• You can add/delete IP addresses to the network card by modifying the ifcfg file

ip addr add 192.168.0.100/24 dev eth0
ip addr delete 192.168.0.100/24 dev eth0

Network card startup and shutdown

• restart network card

service network restart
systemctl restart network

• Enable or disable a network card

ifup/ifdown eth0 or ip link set eth0 up/down

Network Namespace

• In Linux, network isolation is managed through network namespaces. Different network namespaces are isolated from each other

View the network namespace on the current machine

# 查看
ip netns list
# 添加
ip netns add nsl
# 删除
ip netns delete nsl

Create a namespace

# 1.创建一个network namespace   nsl
ip netns add nsl
# 2.查看该 namespace 下的网卡情况
ip netns exec nsl ip a
# 3.启动nsl上的网卡
ip netns exec nsl ifup lo
or
ip netns exec nsl ip link set lo up
# 4.再次查看进行验证   发现state 变成了 UNKOWN
ip netns exec nsl ip a
# 5.再次创建一个network namespace
ip netns add ns2
# 6.现在想让两个namespace网络联通起来，两个namespace在本地拥有各自的网卡。网卡是网络通信的基石
	#为了两个  ns  彼此能够通信，为每个 ns 分别配置网卡和IP
  #  通过  veth pair:  Virtual Ethernet Pair 技术创建网卡
  #  创建出来的网卡能够彼此联通，网卡创建完成之后，分别派给  ns
# 7.创建一对  link  ,也就是接下来通过  veth pair 技术连接的link
ip link add veth-ns1 type veth peer name veth-ns2
# 8.查看  link  的情况
ip link
# 9. 将  veth-ns1   加入  ns1中，  veth-ns2加入ns2中
ip link set veth-ns1 netns ns1
ip link set veth-ns2 netns ns2
# 10.查看 宿主机 和 ns1、ns2的link情况
ip link
ip netns exec ns1 ip link
ip netns exec ns2 ip link
# 11.此时 veth-ns1和veth-ns2还没有IP地址，通信仍然缺少必要条件
ip netns exec ns1 ip addr add 192.168.0.11/24 dev veth-ns1
ip netns exec ns2 ip addr add 192.168.0.12/24 dev veth-ns2
# 12.再次查看，发现 state 是 DOWN ，仍然没有IP地址
ip netns exec ns1 ip link
ip netns exec ns2 ip link
# 13.启动  veth-ns1  和  veth-ns2
ip netns exec ns1 ip link set veth-ns1 up
ip netns exec ns2 ip link set veth-ns2 up
# 14.再次查看，发现  state 是 UP  ,同时有  IP
ip netns exec ns1 ip a
ip netns exec ns2 ip a
# 15.此时两个  network namespace 互相  ping 一下，发现可以进行  ping 通
ip netns exec ns1 ping 192.169.0.12
ip netns exec ns2 ping 192.168.0.11

Docker and network cards

• Each container corresponds to a network namespace and is independent
• After we create multiple containers, the containers can ping each other. How to ping here?

container network-Bridge

# 查看宿主机 网络  
ip a 
# 查看容器网络   
docker exec -it containerName ip a

• Ping the network of the container in the host, and you can find that it can be pinged
• From this, it can be speculated that there is a pair of eth0 in the container and a veth3 in the docker0 of centos. Similar to veth-ns1 and veth-ns2 installed before

# 命令确认  brctl
yum install bridge-utils
brctl show

• Through the test, it is found that the ping between the container and the container can also be passed
• The way to connect is as follows
  • There are multiple veths in docker0, each corresponding to a container's veth
  • This network connection method is called Bridge
  • You can view the network mode in docker by command

#  Bridge也是docker中默认的网络模式
docker network ls
#  检查  Bridge
docker network inspect bridge

Internet access is available in the container

• The container performs network address translation in the form of NAT through docker0 paired veth
• NAT is implemented through iptables

Create a custom network

# 1.创建一个  network   类型为 bridge
docker network create demo-bri
or 
docker network create --subnet=172.18.0.0/24 demo-bri
# 2.查看已有的 network
docker network ls
# 3.查看  demo-bri 详情信息
docker network inspect demo-bri
# 4. 创建新的容器，并指定使用的  network
docker run -d --name container-name --network demo-bri imageID
# 5. 查看 container-name  的网络信息
docker exec -it container-name ip a
# 6. 查看网卡信息
ip a
# 7. 查看网卡接口
brctl show
# 8. 此时使用不同 network 的容器，无法  ping  通 
docker exec -it network1 ping network2-ip
# 9. 使用 connect  进行连接
docker network connect demo-bri network1-container
# 10. 此时查看  demo-bri  网络，能够发现  network1-container 也在其中
# 11. 此时，进入到  network1-container 中，不仅可以通过  ip ping通，而且可以通过名字  ping  通，因为都连接到了自定义的  bridge  上

Container network-HOST & NONE

Host mode

#  1.  创建一个 容器，名字为  demo-host   并且指定网络为  host
docker run -d --name demo-host --network host imageID
#  2.  此时查看IP地址，可以发现和宿主机是一样的
docker exec -it demo-host ip a

None mode

#  1.  创建一个容器，名字为   demo-none  ，并且指定网络为  none
docker run -d --name demo-none --network none imageID
#  2.  查看IP地址  发现没有IP地址
docker exec -it demo-none ip a

Docker data persistence

The key to data persistence lies in the Volume setting

# 查看 volume  可查看目前  docker 容器的  卷  列表
docker volume ls

# 查看 volume 卷详情信息
docker volume inspect volumeID

# volume 使用,   my_volume代表宿主机目录，  /var/lib/mysql代表容器中的目录
docker run *** -v my_volume:/var/lib/mysql ***

• It is worth noting that when the container is deleted, the created volume will not be deleted and can be used again

MySQL cluster construction

Use docker to build a mysql database cluster

#  first of all
#  拉取  pxc  镜像，pxc 镜像是  docker  用来搭建mysql集群的一个成熟的解决方案
docker pull percona/percona-xtradb-cluster:5.7.21

#  创建一个单独的网段给  mysql 数据库集群来使用
docker network create --subnet=ip/port pxc-net
     #    查看详情
     docker network inspect pxc-net
     #    删除
     docker network rm pxc-net
     
     #    删除  volume
     docker volume rm v1
     #    创建  volume
     docker volume create --name volumeName
     
#  运行三个 PXC 容器，搭建 PXC[mysql] 集群
docker run -d -p 3301:3306 -v v1:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=jack123 \
-e CLUSTER_NAME=PXC \
-e XTRABACKUP_PASSWORD=jack123 \
--privileged --name=node1 \
--net=pxc-net --ip 172.18.0.2 pxc

docker run -d -p 3302:3306 -v v2:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=jack123 \
-e CLUSTER_NAME=PXC -e XTRABACKUP_PASSWORD=jack123 \
-e CLUSTER_JOIN=node1 \
--privileged --name=node2 --net=pxc-net --ip 172.18.0.3 pxc

docker run -d -p 3303:3306 -v v3:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=jack123 \
-e CLUSTER_NAME=PXC \
-e XTRABACKUP_PASSWORD=jack123 \
-e CLUSTER_JOIN=node1 \
--privileged --name=node3 --net=pxc-net --ip 172.18.0.4 pxc

#  工具连接测试

Database cluster load balancing

Reasonable use of cluster database services by exposing an address to the outside world

#  首先拉取  haproxy  镜像
docker pull haproxy
#  创建 haproxy ，使用 bind mounting 的方式
touch /tmp/haproxy/haproxy.cfg

haproxy.cfg

global 
  #工作目录，这边要和创建容器指定的目录对应 chroot /usr/local/etc/haproxy #日志文件
  log 127.0.0.1 local5 info 
  #守护进程运行
  daemon

defaults
	log global
	mode   http
	#日志格式
	option httplog 
	#日志中不记录负载均衡的心跳检测记录 
	option dontlognull 
	#连接超时(毫秒)
	timeout connect 5000 
	#客户端超时(毫秒) 
	timeout client 50000 
	#服务器超时(毫秒) 
	timeout server 50000
	#监控界面
	listen admin_stats
	#监控界面的访问的IP和端口
	bind 0.0.0.0:8888
	#访问协议
	mode http
	#URI相对地址
	stats uri  /dbs_monitor
	#统计报告格式
	stats realm Global\ statistics
	#登陆帐户信息
	stats auth admin:admin
	#数据库负载均衡
  listen proxy-mysql 
  #访问的IP和端口，haproxy开发的端口为3306 
  #假如有人访问haproxy的3306端口，则将请求转发给下面的数据库实例 
  bind 0.0.0.0:3306
  #网络协议
  mode tcp
  #负载均衡算法(轮询算法)
  #轮询算法:roundrobin
  #权重算法:static-rr
  #最少连接算法:leastconn
	#请求源IP算法:source
	balance roundrobin
	#日志格式
	option tcplog 
	#在MySQL中创建一个没有权限的haproxy用户，密码为空。
	#Haproxy使用这个账户对MySQL数据库心跳检测
	option mysql-check user haproxy
	server MySQL_1 172.18.0.2:3306 check weight 1 maxconn 2000 
	server MySQL_2 172.18.0.3:3306 check weight 1 maxconn 2000 
	server MySQL_3 172.18.0.4:3306 check weight 1 maxconn 2000 
	#使用keepalive检测死链
  option tcpka

Create haproxy container

docker run -it -d -p 8888:8888 -p 3306:3306 -v /tmp/haproxy:/usr/local/etc/haproxy \
--name haproxy01 --privileged --net=pxc-net haproxy

start haproxy

docker exec -it haproxy01 bash haproxy -f /usr/local/etc/haproxy/haproxy.cfg

Create a user on the MySQL database for heartbeat detection

CREATE USER 'haproxy'@'%' IDENTIFIED BY ''; [小技巧[如果创建失败，可以先输入一下命令]:
    drop user 'haproxy'@'%';
    flush privileges;
    CREATE USER 'haproxy'@'%' IDENTIFIED BY '';
]

Browser access verification

http://centos_ip:8888/dbs_monitor 
# 用户名密码都是:admin

connect to haproxy01

ip:centos_ip
port:3306
user:root
password:123456