Development and deployment system environment - the use of common docker images

Docker provides many useful images. If you don't want to use them, you can create your own images. Creating an image through a Dockerfile may be a little complicated. You can customize the image you need by updating the basic ubuntu container to secure new applications.

ubuntu mirror

When using the ubuntu image (the simplest official native) in docker, you can deploy the simplest ubuntu container

Ubuntu source replacement

Until Ubuntu is replaced with domestic sources, it will be much faster, but the container run by Docker's native Ubuntu image does not even have vi. If you need to change the source, you can only do it manually. It should be noted that the source version needs to be consistent with the ubuntu version. The version here does not refer to the version number, but the codename code version. You can use to lsb_release -aview the code version

displayed in codename . You can specify the code version in the changed source, for example, mine is focal. Docker's native Ubuntu doesn't even have it. You can use it to install it, or directly change the source of focal.lsb_releaseapt install lsb-release

rm /etc/apt/sources.list	# 删除默认的源文件
# 也可以重命名源文件，以做备份
# mv /etc/apt/sources.list /etc/apt/sources.list.bak

# 添加 163 的镜像源
echo "deb http://mirrors.163.com/ubuntu/ focal main restricted universe multiverse" >/etc/apt/sources.list
echo "deb http://mirrors.163.com/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb http://mirrors.163.com/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb http://mirrors.163.com/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb http://mirrors.163.com/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/ubuntu/ focal main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list

# 添加阿里云的镜像
echo "deb http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse" >/etc/apt/sources.list
echo "deb http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list

# 添加清华源
echo "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse" >/etc/apt/sources.list
echo "deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list

# 添加中科大源
echo "deb https://mirrors.ustc.edu.cn/ubuntu/ focal main restricted universe multiverse" >/etc/apt/sources.list
echo "deb-src https://mirrors.ustc.edu.cn/ubuntu/ focal main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.ustc.edu.cn/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.ustc.edu.cn/ubuntu/ focal-updates main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.ustc.edu.cn/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.ustc.edu.cn/ubuntu/ focal-backports main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.ustc.edu.cn/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.ustc.edu.cn/ubuntu/ focal-security main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb https://mirrors.ustc.edu.cn/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list
echo "deb-src https://mirrors.ustc.edu.cn/ubuntu/ focal-proposed main restricted universe multiverse" >>/etc/apt/sources.list

What does it mean to change source code?

• deb: Install through deb mode
• deb-src: install from source files
• focal: code version, corresponding to the corresponding part under the directory dists/ in the source warehouse
• main restricted…: corresponds to the next directory in the warehouse

Some source certificates errors

Some sources, such as those from the University of Science and Technology of China and Tsinghua University, require the use of certificates. Docker is not installed natively on Ubuntu, so you need to install it.

apt install ca-certificates

ubuntu root user and password

After the ubuntu container is established, you can directly enter the root user. If a password is required, you can use passwd rootthe command to reset it.

Automatic start of services in containers

If the service in the container needs to be started automatically, you can use systemctl enable 服务名to start it. If you don't have systemctl, you need to install it first. (It should be noted that the container must be running in privileged mode, that is, --privileged=true, which can be viewed using docker inspect 容器名. If it is not running in privileged mode, you can configure the config.v2.json (the path can be found later) in the same directory as the file hostconfig.json file, modify "Privileged":false=> "Privileged":true)

apt install systemd

When using systemctl, check system statusthe status. If an error is reported: System has not been booted with systemd as init system (PID 1). Can't operate., you need to configure it. Exit the container, close the docker service, and configure it on the host

docker inspect 容器名
# 查看并记住 id 项,使用 vim 或其他方法编辑 config.v2.json 文件
vim /var/lib/docker/containers/[刚才查看的 id]/config.v2.json

Change "Path":"bash"(or "Path": "/bin/bash") to: "Path":"/sbin/init", change "Cmd":["bash"](or "Cmd":["/bin/bash"]) to: "Cmd":["/sbin/init"].

Then re-run docker and run the container. Sometimes this one will also report an error:
Error response from daemon: failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: exec: "/sbin/init": stat /sbin/init: no such file or directory: unknown
it means there is no /sbin/initfile, then reinstall it.

apt install init

Then run the container, execute it inside the container systemctl enable 服务名, and systemctl statuscheck the status. You will find that the service inside the container is set to start automatically.

If you use an image to create a container that can start automatically, you need to add initialization parameters when generating the container. For example, the initialization program of ubuntu is in /sbin/init, then

docker run -dit --name 容器名称 -p 端口号:端口号 --privileged=true 镜像名称 /sbin/init

Public key error

If apt updatean error is reported:

The following signatures couldn't be verified because the public key is not available: NO_PUBKEY xxxxxxxxxxxxxx
means that the public key is not available, just obtain and register the certificate again

# 获取公钥证书
gpg --keyserver keyserver.ubuntu.com --recv-keys xxxxxxxxxxxxxxxx
# 注册公钥证书
gpg --export --armor xxxxxxxxxxxxxxxx | sudo apt-key add -

Delete redundant cache files

Ubuntu will update the cache list after updating. This list will also occupy a lot of space resources. It can be deleted before packaging the image to save space.

rm -rf /var/lib/apt/lists/*

It should be noted that if the container is submitted as an image before deleting the cache, the actual data information will enter the image's history. If you generate a container with this image, delete the cache, and then submit the image, you will find that the size has not changed.

mysql

To install mysql in the basic container, you can refer to the ordinary Linux system to install mysql, and just use the package manager to install it. You can also download the image directly.

Using images with configuration

The host creates the data directory and configuration file:

mkdir -p /mydata/mysql/conf
mkdir -p /mydata/mysql/data

Create configuration file and input:

echo "[mysqld]" >> /mydata/mysql/conf/my.cnf
echo "port=3306" >> /mydata/mysql/conf/my.cnf
echo "bind-address=0.0.0.0" >> /mydata/mysql/conf/my.cnf
echo "character-set-server=utf8" >> /mydata/mysql/conf/my.cnf
echo "default_authentication_plugin=caching_sha2_password" >> /mydata/mysql/conf/my.cnf
echo "max_allowed_packet=20971520" >> /mydata/mysql/conf/my.cnf
echo "expire_logs_days=7" >> /mydata/mysql/conf/my.cnf
echo "server_id=1" >> /mydata/mysql/conf/my.cnf
echo "[client]" >> /mydata/mysql/conf/my.cnf
echo "default-character-set=utf8" >> /mydata/mysql/conf/my.cnf
echo "[mysql]" >> /mydata/mysql/conf/my.cnf
echo "default-character-set=utf8" >> /mydata/mysql/conf/my.cnf

Create a container using the default image:

docker run -dit --name mysql -p 3306:3306 -v /mydata/mysql/conf/my.cnf:/etc/mysql/conf.d/mysql.cnf -v /mydata/mysql/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123456 --restart=always --network common-network mysql

docker container permissions

It should be noted that the container created when installing mysql needs to add the –privileged=true parameter, otherwise the error Cannot stat file /proc/31/fd/29: Permission denied will be reported.

docker run -dit --privileged=true --name=mysql -p 3306:3306 ubuntu 

remote access

mysql needs to modify the configuration file and user access permissions to support remote access.

The mysql configuration file is: /etc/mysql/mysql.conf.d/mysqld.cnf. Modifications can be copied from inside the container to the local host, and then copied into the container.

# 在宿主机
docker cp 容器:/etc/mysql/mysql.conf.d/mysqld.cnf 本地路径
docker cp 本地路径/mysqld.cnf 容器:/etc/mysql/mysql.conf.d/

Modify user access rights:

# 查看各用户访问权限
select host,user from user;
# 设置 root 用户所有主机均可以使用其进行访问
update user set host = '%' where user = 'root';
# 刷新授权
FLUSH PRIVILEGES;

Delete cache

After mysql is installed, you also need to delete the cache before submitting the image:

rm -rf /var/lib/apt/lists/*
rm -rf /var/lib/mysql/*

redis

The docker image version of redis is okay, and it is quite simple to use and can be used. It should be noted that starting with the default image will report a warning:
WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
it needs to be executed on the server (not within the container) sysctl vm.overcommit_memory=1or execute the following statement:

echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf \
&& sysctl -p

To install redis in the basic container, you can refer to the ordinary Linux system to install redis and use the package manager to install it.

Create redis container with configuration

First create the configuration file

mkdir -p /mydata/redis/conf
touch /mydata/redis/redis.conf

Then write some required configuration information

echo "appendonly yes" >> /mydata/redis/redis.conf			# 持久化
echo "port 6379" >> /mydata/redis/redis.conf				# 端口
echo "bind 0.0.0.0" >> /mydata/redis/redis.conf			# 访问主机
echo "dbfilename dump.rdb" >> /mydata/redis/redis.conf		# 数据库文件
echo "dir /data" >> /mydata/redis/redis.conf				# 工作目录
echo "# requirepass 123456" >> /mydata/redis/redis.conf	# 数据库密码
echo "save 900 1" >> /mydata/redis/redis.conf				# 保存策略
echo "save 300 10" >> /mydata/redis/redis.conf				# 保存策略
echo "save 60 10000" >> /mydata/redis/redis.conf			# 保存策略
echo "maxmemory 1GB" >> /mydata/redis/redis.conf			# 占用最大内存

Then use this configuration file to create a container

docker run -p 6379:6379 --name redis -v /mydata/redis:/data --restart=always --network common-network -dit redis redis-server /data/redis.conf

Let’s reorganize the functions of each parameter:

• -p 6379:6379: Enable container-to-host port mapping
• –name redis: the name of the created container
• -v /mydata/redis/data:/data: Create a mapping of the working directory data (host to container)
• -v /mydata/redis/conf/redis.conf:/etc/redis/redis.conf: Create a mapping of configuration files (host to container)
• –restart=always: automatically run the container after docker starts
• –network common-network: Create an interconnection network (needs to be created in advance)
• -dit: Run in the background and use the work interface
• redis: image file used
• redis-server /etc/redis/redis.conf: execution parameters (that is, use the mapped configuration file as a parameter to execute redis-server)

python

Change source

Docker's official python should be based on debian's bullseye version, so change the source and use it.

rm /etc/apt/sources.list
echo "deb http://mirrors.163.com/debian/ bullseye main non-free contrib" >/etc/apt/sources.list
echo "deb http://mirrors.163.com/debian/ bullseye-updates main non-free contrib" >>/etc/apt/sources.list
echo "deb http://mirrors.163.com/debian/ bullseye-backports main non-free contrib" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/debian/ bullseye main non-free contrib" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/debian/ bullseye-updates main non-free contrib" >>/etc/apt/sources.list
echo "deb-src http://mirrors.163.com/debian/ bullseye-backports main non-free contrib" >>/etc/apt/sources.list

Example

Containers can be created using the official python image. When creating a container, it is best to specify the mapping directory (for storing source code), the working directory and the script file (or instruction) to run. For example (using django as an example):

docker run -dit --name python -p 8000:8000 -p 222:22 -v /mydata/python/src:/src -w /src/project --restart=always --network common-network python python manage.py runserver

In addition to enabling the mapping of service ports, port 22 is also mapped to facilitate ssh login for code modification, debugging and other operations.

Notice

It should be noted that docker will automatically close and release resources when it finds that there is no running program in the foreground. Therefore, if you create a simple python container, for example, just run hello world, it will output characters and close directly. This feature allows you to directly create a python container and run the django program. It will report that there are no dependent libraries (django and other libraries are not installed), and then close the container directly. To deal with this problem, you need to first create a python container that runs the shell (can be mapped to the program folder to obtain requirements.txt), then install the dependent libraries, and then execute it manually python manage.py. If you need to use the automatic restart function, you need to package it as an image and then create a container by adding execution parameters.

Installation library

Install dependent libraries according to requirements.txt

pip install -r requirements.txt -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com

Install ssh

In order to facilitate remote debugging of code, you can enable the ssh service. If not, you can reinstall it.

apt install openssh-server

Then you need to configure the ssh settings file /etc/ssh/sshd_configto allow remote login:

• Change # port 22 to port 22 (remove #)
• Change #PermitRootLogin prohibit-password to PermitRootLogin yes

Once completed, check to see if it is enabled

ps -ef | grep ssh
dpkg -l | grep ssh

If it is not enabled, you need to run it manually

/etc/init.d/ssh start

Also remember to create a root password

passwd root