【Docker】Intermediate

(1) Docker complex installation details

[1] Install mysql master-slave replication

(1) Create a new main server container instance 3307

docker run -p 3307:3306 --name mysql-master -v /mydata/mysql-master/log:/var/log/mysql -v /mydata/mysql-master/data:/var/lib/mysql -v /mydata/mysql-master/conf:/etc/mysql -e MYSQL_ROOT_PASSWORD=root -d mysql:5.7

(2) Enter the /mydata/mysql-master/conf directory to create a new my.cnf

[mysqld]
##设置server-id，同一局域网中需要唯一
server_id=101
##指定不需要同步的数据库名称
binlog-ignore-db=mysql
##开启二进制日志功能
log-bin=mall-mysql-bin
##设置二进制日志使用内存大小（事务）
binlog_cache_size=1M
##设置使用的二进制日志格式（mixed，statement，row）
binlog_format=mixed
##二进制日志过期清理时间。默认值为0，表示不自动清理
expire_logs_days=7
##跳过主从复制中遇到的所有错误或指定类型的错误，避免slave端复制中断
##如：1062错误是指一些主键重复，1032错误是因为主从数据库数据不一致
slave_skip_errors=1062

(3) Restart the master instance after modifying the configuration

docker restart mysql-master

(4) Enter the mysql-master container

docker exec -it mysql-master /bin/bash

mysql -uroot -proot

(5) Create a data synchronization user in the master container instance

CREATE USER 'slave'@'%' IDENTIFIED BY '123456';

GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'slave'@'%';

(6) Create a new slave server container instance 3308

docker run -p 3308:3306 --name mysql-slave -v /mydata/mysql-slave/log:/var/log/mysql -v /mydata/mysql-slave/data:/var/lib/mysql -v /mydata/mysql-slave/conf:/etc/mysql -e MYSQL_ROOT_PASSWORD=root -d mysql:5.7

(7) Enter the /mydata/mysql-master/conf directory to create a new my.cnf

[mysqld]
##设置server-id，同一局域网中需要唯一
server_id=102
##指定不需要同步的数据库名称
binlog-ignore-db=mysql
##开启二进制日志功能
log-bin=mall-mysql-bin
##设置二进制日志使用内存大小（事务）
binlog_cache_size=1M
##设置使用的二进制日志格式（mixed，statement，row）
binlog_format=mixed
##二进制日志过期清理时间。默认值为0，表示不自动清理
expire_logs_days=7
##跳过主从复制中遇到的所有错误或指定类型的错误，避免slave端复制中断
##如：1062错误是指一些主键重复，1032错误是因为主从数据库数据不一致
slave_skip_errors=1062
##relay_log配置中继日志
relay_log=mall-mysql-relay-bin
##log_slave_updates表示slave将复制事件写进自己的二进制日志
log_slave_updates=1
##slave设置为只读（具有super权限的用户除外）
read_only=1

(8) Restart the slave instance after modifying the configuration

docker restart mysql-slave

(9) View the master-slave synchronization status in the master database

show master status;

(10) Enter the mysql-slave container

docker exec -it mysql-slave /bin/bash

mysql -uroot -proot

(11) Configure master-slave replication in the slave database

change master to master_host='192.168.19.11',master_user='slave',master_password='123456',master_port=3307,master_log_file='mall-mysql-bin.000001',master_log_pos=617,master_connect_retry=30;

(12) Check the master-slave synchronization status in the slave database

show slave status \G;

(13) Enable master-slave synchronization in the slave database

start slave;

(14) View the status of the slave database and find that it has been synchronized

(15) Master-slave replication test

Execute the following sql in the main library

create database testslave;

use testslave;

CREATE TABLE hero ( id int(11) AUTO_INCREMENT, name varchar(30) , hp float , damage int(11) , PRIMARY KEY (id) )  DEFAULT CHARSET=utf8;

insert into hero values (null, '盖伦', 616, 100);

Query from the library, the query is successful.

[2] Install redis cluster

(1) Interview questions

100-200 million pieces of data need to be cached, how to design this storage case?
It is 100% impossible for a single machine and a single machine. It must be distributed storage. How to use redis to implement it? Since it is distributed, which server does a piece of data fall on? Can you guarantee that the next time you read it, it will also be read from that server?

200 million records are 200 million kv. We can’t do it on a single machine, we must distribute multiple machines. Suppose there are 3 machines to form a cluster. Every read and write operation of the user is based on the formula: hash(key)%N number of machines , to calculate the hash value, which is used to determine which node the data is mapped to.

(1) Hash remainder partition
(2) Consistent hash algorithm partition
(3) Hash slot partition

(2) Build a three-master and three-slave redis cluster

(1) Create 6 new docker container instances

docker run -d --name redis-node-1 --net host --privileged=true -v /data/redis/share/redis-node-1:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6381

docker run -d --name redis-node-2 --net host --privileged=true -v /data/redis/share/redis-node-2:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6382

docker run -d --name redis-node-3 --net host --privileged=true -v /data/redis/share/redis-node-3:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6383

docker run -d --name redis-node-4 --net host --privileged=true -v /data/redis/share/redis-node-4:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6384

docker run -d --name redis-node-5 --net host --privileged=true -v /data/redis/share/redis-node-5:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6385

docker run -d --name redis-node-6 --net host --privileged=true -v /data/redis/share/redis-node-6:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6386

(2) Enter the container redis-node-1 and build a cluster relationship for 6 machines

docker exec -it redis-node-1 /bin/bash

redis-cli --cluster create 192.168.19.11:6381 192.168.19.11:6382 192.168.19.11:6383 192.168.19.11:6384 192.168.19.11:6385 192.168.19.11:6386 --cluster-replicas 1

–cluster-replicas 1 means to create a slave node for each master

(3) Link to 6381 as the entry point to view the node status

redis-cli -p 6381

cluster info

(3) Master-slave fault-tolerant switching migration case

(1) Data reading and writing storage
First enter 6381, try to store data, and find that some data slots exceed the slots allocated by 6381, then an error will be reported

docker exec -it redis-node-1 /bin/bash

stand-alone mode connection

redis-cli -p 6381

Change to cluster connection to optimize routing

redis-cli -p 6381 -c

(2) Fault-tolerant switching migration (test master-slave switching)
first check the cluster information in the 6381 host,
stop the 6381 host
and re-enter 6382

docker exec -it redis-node-2 /bin/bash
redis-cli -p 6382 -c

Check the cluster information, you can see that the original 6386 slave has become the master.
After starting 6381, 6381 becomes a new slave. If you want to turn 6381 into the master again, you can stop the current 6386 and restart it.

(4) Master-slave expansion case

(1) Create two new nodes 6387 and 6388, and start them after they are created to check whether there are 8 nodes

docker run -d --name redis-node-7 --net host --privileged=true -v /data/redis/share/redis-node-7:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6387

docker run -d --name redis-node-8 --net host --privileged=true -v /data/redis/share/redis-node-8:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6388

docker ps

(2) Enter the interior of the 6387 container instance

docker exec -it redis-node-7 /bin/bash

(3) Add the newly added 6387 node (empty slot) as the master node to join the original cluster
Add the newly added 6387 as the master node to join the cluster, 6381 is the leader in the original cluster node, which is equivalent to 6387 saying goodbye to the dock of 6381 to find Organization joins the cluster

redis-cli --cluster add-node 192.168.19.11:6387 192.168.19.11:6381

(4) Check the cluster status for the first time

redis-cli --cluster check 192.168.19.11:6381

(5) Reassign the slot number to 6387

redis-cli --cluster reshard 192.168.19.11:6387

Allocate 4096 slots for new
and type all to get some from each slot

(6) Check the cluster status for the second time

redis-cli --cluster check 192.168.19.11:6381

(7) Assign master node 6387 the last number of slave node 6388 is the number of 6387

redis-cli --cluster add-node 192.168.19.11:6388 192.168.19.11:6387 --cluster-slave --cluster-master-id 9322efd4b8f57d0c01fccc2d2ba310021781f74f

(8) Check the cluster status for the third time

redis-cli --cluster check 192.168.19.11:6381

(5) Master-slave shrink case

(1) Check the cluster status 1 to get the node ID of 6388

redis-cli --cluster check 192.168.19.11:6382

(2) Delete 6388, delete No. 4 from node 6388 in the cluster,
and the last node ID is the node ID of 6388

redis-cli --cluster del-node 192.168.19.11:6388 2e4ec79d0731b36c01a8d3739e30c67551b76eeb

(3) Clear the slot number of 6387 and redistribute it. In this example, all the cleared slot numbers are assigned to 6381

redis-cli --cluster reshard 192.168.19.11:6381

Then output done to indicate the end

(4) Check the cluster status for the second time

redis-cli --cluster check 192.168.19.11:6382

(5) Delete 6387
and the last node ID is the node ID of 6387

redis-cli --cluster del-node 192.168.19.11:6387 9322efd4b8f57d0c01fccc2d2ba310021781f74f

(6) Check the cluster situation for the third time

redis-cli --cluster check 192.168.19.11:6382

You can see that there is no node 6387

(2) DockerFile analysis

(3) Docker micro-service combat

(4) Docker network

(1) What is it
(2) Commonly used basic commands
(3) What can it do
(4) Network mode
(5) Docker platform architecture diagram

(5) Docker-compose container arrangement

(6) Portainer, a lightweight visualization tool for Docker

(7) CAdvisor+InfluxDB+Granfana for Docker container monitoring

(8) Final Chapter & Summary