Learn to use the container orchestration tool docker-compose

1. Introduction to docker-compose

The Docker-Compose project is Docker's official open source project, responsible for realizing the rapid orchestration of Docker container clusters. The Docker-Compose project is written in Python (mainly written in golang after version v2) github:docker-compose , calling the API provided by the Docker service to manage the container. Therefore, as long as the platform you are operating on supports the Docker API, you can use Compose for orchestration management.

Docker-Compose divides the managed containers into three layers, namely project, service and container.

All files in the Docker-Compose running directory (docker-compose.yml, extends files or environment variable files, etc.) form a project. If there is no special specification, the project name is the current directory name.
A project can contain multiple services, and each service defines the image, parameters, and dependencies for the container to run.
A service can include multiple container instances. Docker-Compose does not solve the problem of load balancing, so other tools are needed to achieve service discovery and load balancing.

docker-compose functionality:

Provide tools for defining and running multiple docker container applications;
Use yaml file to configure application service (docker-compse.yml);
You can use a simple command docker-compse up to start all services according to their dependencies;
All services can be stopped with a simple command docker-compose down;
When a service is needed, it can be easily expanded through --scale

Docker Compose has the following characteristics:

For greater portability, Docker Compose only requires one docker-compse upto start all services according to their dependencies, and then docker-compose downeasily tear them down using Docker Compose. Help us deploy complex applications more easily;
Multiple isolated environments on a single host. Compose can use project names to isolate environments from each other. This allows multiple copies of the same environment to be run on one computer. It prevents different projects and services from interfering with each other;

Docker-Compose defect:
it can only be used to orchestrate containers on a single host, and cannot orchestrate containers across node hosts.

2. Install docker-compose

2.1. Docker-compose and docker version dependencies

compose file format version	docker-compose version	docker engie version
3.8		19.03.0+
3.7		18.06.0+
3.6		18.02.0+
3.5		17.12.0+
3.4		17.09.0+
3.3		17.06.0+
3.2		17.04.0+
3.1		1.13.1+
3.0		1.13.0+
2.4	v1.21.0+	17.12.0+
2.3	v1.16.0+	17.06.0+
2.2	v1.13.0+	1.13.0+
2.1	v1.9.0+	1.12.0+
2.0		1.10.0+
1.0		1.9.1.+

For other version information, please check the official website: https://docs.docker.com/compose/compose-file/compose-versioning/#version-3

The corresponding relationship between docker-compose version and compose file format version is as follows:

Docker Compose version	Compose file format version
1.x	1
2.x	2
3.x	3.0, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6

It should be noted that the Compose file format version refers to the syntax version used in the Compose file. For example, use version: "3.4" to specify the Compose file format version as 3.4. The Docker Compose version refers to the version of the Docker Compose tool used.

2.2. Installation

2.2.1, pip installation

pip install docker-compose

2.2.2, yum installation

yum install docker-compose

2.2.3. Installation of official website installation package

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

2.3. Verify whether the installation is successful

docker-compose version
docker-compose version 1.21.0, build 5920eb0
docker-py version: 3.7.3
CPython version: 2.7.5
OpenSSL version: OpenSSL 1.0.2k-fips  26 Jan 2017

3. docker-compose application

3.1. Writing docker-compose.yml configuration file

Insert image description here

For details, see: https://blog.csdn.net/qq_36148847/article/details/79427878

3.2. Usage examples

Create a simple flask web project, using web services and redis services.
Create a new project directory usecompose, and create a new flask application file in this directory app.pywith the following content:

import time

import redis
from flask import Flask

app = Flask(__name__)
cache = redis.Redis(host='redis', port=6379)

def get_hit_count():
    retries = 5
    while True:
        try:
            return cache.incr('hits')
        except redis.exceptions.ConnectionError as exc:
            if retries == 0:
                raise exc
            retries -= 1
            time.sleep(0.5)

@app.route('/')
def hello():
    #count = get_hit_count()
    return 'Hello World! \n'

Create a new dependent file in the same directory requirements.txtwith the following content:

flask
redis

Create a new directory with Dockerfilethe following contents

#基础镜像
FROM python3:1.3

#设置镜像的工作目录
WORKDIR /home/gyy01167504/daily-test/usecompose

#构建的时候设置环境变量
ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0
ENV LC_ALL=en_US.utf-8
ENV LANG=en_US.utf-8

#安装gcc编译器
RUN yum install -y python-pip

#把刚刚我们创建的requirements.txt文件拷贝到镜像里
COPY requirements.txt requirements.txt

#下载requirements.txt里写好的flask和redis
RUN pip3 install -r requirements.txt

#指定5000为对外暴露的端口
EXPOSE 5000

#把当前目录拷贝进镜像里
COPY . .

#指定这个容器启动的时候要运行的命令，执行的命令为：flask run
CMD ["flask", "run"]

Create a new directory docker-compose.ymlwith the following content:

version: "3.3"
#build后可以跟自己写的Dockerfile文件，写在build后面的Dockerfile文件用于生成镜像
services:
  web:
    build: .
    ports:
      - "8000:5000"
#如果build后没值，则镜像用从DockerHub上拉下来的redis镜像
  redis:
    image: "redis:latest"

Pull the required image to the local machine

docker pull python3:1.3
docker pull redis:latest

Start docker-compse:

docker-compose up --build
Building web
Step 1/12 : FROM python3:1.3
 ---> f4b95d7919ec
Step 2/12 : WORKDIR /home/usecompose
 ---> 3c00d1379232
Step 3/12 : ENV FLASK_APP=app.py
 ---> 35a30912fb1e
Step 4/12 : ENV FLASK_RUN_HOST=0.0.0.0
 ---> 0d1e57292603
Step 5/12 : ENV LC_ALL=en_US.utf-8
 ---> 1f9f28e7f8fb
Step 6/12 : ENV LANG=en_US.utf-8
 ---> 649a76ac4f7e
Step 7/12 : RUN yum install -y python-pip
 ---> Running in bd6977d4615d
Loaded plugins: bestyumcache, branch, fastestmirror
Trying branch: stable
Determining fastest mirrors
Resolving Dependencies
--> Running transaction check
---> Package python-pip.noarch 0:7.1.0-1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package         Arch        Version           Repository                  Size
================================================================================
Installing:
 python-pip      noarch      7.1.0-1.el7       alios.7u2.base.x86_64      1.5 M

Transaction Summary
================================================================================
Install  1 Package

Total download size: 1.5 M
Installed size: 6.6 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : python-pip-7.1.0-1.el7.noarch                                1/1 
  Verifying  : python-pip-7.1.0-1.el7.noarch                                1/1 

Installed:
  python-pip.noarch 0:7.1.0-1.el7                                               

Complete!
Removing intermediate container bd6977d4615d
 ---> 3ada8adcd97b
Step 8/12 : COPY requirements.txt requirements.txt
 ---> 3934b409f73b
Step 9/12 : RUN pip3 install -r requirements.txt
 ---> Running in 67f939a97ebc
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting flask (from -r requirements.txt (line 1))
  Downloading http://yum.tbsite.net/pypi/packages/cd/77/59df23681f4fd19b7cbbb5e92484d46ad587554f5d490f33ef907e456132/Flask-2.0.3-py3-none-any.whl (95kB)
Collecting redis (from -r requirements.txt (line 2))
  Downloading http://yum.tbsite.net/pypi/packages/d6/f6/19237b28c632935c7359bddf703395ba13bbd134fc5e2eb297c4c120398c/redis-4.3.6-py3-none-any.whl (248kB)
Collecting Werkzeug>=2.0 (from flask->-r requirements.txt (line 1))
  Downloading http://yum.tbsite.net/pypi/packages/f4/f3/22afbdb20cc4654b10c98043414a14057cd27fdba9d4ae61cea596000ba2/Werkzeug-2.0.3-py3-none-any.whl (289kB)
Collecting click>=7.1.2 (from flask->-r requirements.txt (line 1))
  Downloading http://yum.tbsite.net/pypi/packages/4a/a8/0b2ced25639fb20cc1c9784de90a8c25f9504a7f18cd8b5397bd61696d7d/click-8.0.4-py3-none-any.whl (97kB)
Collecting itsdangerous>=2.0 (from flask->-r requirements.txt (line 1))
  Downloading http://yum.tbsite.net/pypi/packages/9c/96/26f935afba9cd6140216da5add223a0c465b99d0f112b68a4ca426441019/itsdangerous-2.0.1-py3-none-any.whl
Collecting Jinja2>=3.0 (from flask->-r requirements.txt (line 1))
  Downloading http://yum.tbsite.net/pypi/packages/20/9a/e5d9ec41927401e41aea8af6d16e78b5e612bca4699d417f646a9610a076/Jinja2-3.0.3-py3-none-any.whl (133kB)
Collecting packaging>=20.4 (from redis->-r requirements.txt (line 2))
  Downloading http://yum.tbsite.net/pypi/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl (40kB)
Requirement already satisfied: typing-extensions; python_version < "3.8" in /usr/local/lib/python3.6/site-packages (from redis->-r requirements.txt (line 2))
Collecting importlib-metadata>=1.0; python_version < "3.8" (from redis->-r requirements.txt (line 2))
  Downloading http://yum.tbsite.net/pypi/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl
Collecting async-timeout>=4.0.2 (from redis->-r requirements.txt (line 2))
  Downloading http://yum.tbsite.net/pypi/packages/d6/c1/8991e7c5385b897b8c020cdaad718c5b087a6626d1d11a23e1ea87e325a7/async_timeout-4.0.2-py3-none-any.whl
Collecting dataclasses; python_version < "3.7" (from Werkzeug>=2.0->flask->-r requirements.txt (line 1))
  Downloading http://yum.tbsite.net/pypi/packages/fe/ca/75fac5856ab5cfa51bbbcefa250182e50441074fdc3f803f6e76451fab43/dataclasses-0.8-py3-none-any.whl
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib64/python3.6/site-packages (from Jinja2>=3.0->flask->-r requirements.txt (line 1))
Collecting pyparsing!=3.0.5,>=2.0.2 (from packaging>=20.4->redis->-r requirements.txt (line 2))
  Downloading http://yum.tbsite.net/pypi/packages/6c/10/a7d0fa5baea8fe7b50f448ab742f26f52b80bfca85ac2be9d35cdd9a3246/pyparsing-3.0.9-py3-none-any.whl (98kB)
Collecting zipp>=0.5 (from importlib-metadata>=1.0; python_version < "3.8"->redis->-r requirements.txt (line 2))
  Downloading http://yum.tbsite.net/pypi/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl
Installing collected packages: dataclasses, Werkzeug, zipp, importlib-metadata, click, itsdangerous, Jinja2, flask, pyparsing, packaging, async-timeout, redis
Successfully installed Jinja2-3.0.3 Werkzeug-2.0.3 async-timeout-4.0.2 click-8.0.4 dataclasses-0.8 flask-2.0.3 importlib-metadata-4.8.3 itsdangerous-2.0.1 packaging-21.3 pyparsing-3.0.9 redis-4.3.6 zipp-3.6.0
Removing intermediate container 67f939a97ebc
 ---> 9e3d2ef13dde
Step 10/12 : EXPOSE 5000
 ---> 7e3ef9cfeb77
Step 11/12 : COPY . .
 ---> c29447e3ce81
Step 12/12 : CMD ["flask", "run"]
 ---> Running in 298d8ea26547
Removing intermediate container 298d8ea26547
 ---> 1ffb139f73c9

Successfully built 1ffb139f73c9
Successfully tagged usecompose_web:latest
Recreating usecompose_web_1 ... done
Starting usecompose_redis_1 ... done
Attaching to usecompose_redis_1, usecompose_web_1
redis_1  | 1:C 19 Apr 2023 10:26:39.049 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1  | 1:C 19 Apr 2023 10:26:39.049 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1  | 1:C 19 Apr 2023 10:26:39.049 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * monotonic clock: POSIX clock_gettime
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * Running mode=standalone, port=6379.
redis_1  | 1:M 19 Apr 2023 10:26:39.050 # Server initialized
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * Loading RDB produced by version 7.0.11
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * RDB age 52 seconds
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * RDB memory usage when created 0.82 Mb
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * Done loading RDB, keys loaded: 0, keys expired: 0.
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * DB loaded from disk: 0.000 seconds
redis_1  | 1:M 19 Apr 2023 10:26:39.050 * Ready to accept connections
web_1    |  * Serving Flask app 'app.py' (lazy loading)
web_1    |  * Environment: production
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |    Use a production WSGI server instead.
web_1    |  * Debug mode: off
web_1    |  * Running on all addresses.
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |  * Running on http://172.18.0.3:5000/ (Press CTRL+C to quit)
web_1    | 30.39.220.140 - - [19/Apr/2023 18:27:39] "GET / HTTP/1.1" 200 -

View container startup status:

docker ps
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                    NAMES
49f93d2c3ab0        usecompose_web           "flask run"              43 seconds ago      Up 42 seconds       0.0.0.0:8000->5000/tcp   usecompose_web_1
db7c00fd9a2d        redis:latest             "docker-entrypoint.s…"   About an hour ago   Up 42 seconds       6379/tcp                 usecompose_redis_1

Test access:

curl http://11.166.91.186:8000/
Hello World!

4. Common commands of docker-compose

4.1 docker-compose up

Build the image and start the container

docker-compose up
Creating network "usecompose_default" with the default driver
Creating usecompose_web_1   ... done
Creating usecompose_redis_1 ... done
Attaching to usecompose_redis_1, usecompose_web_1
redis_1  | 1:C 20 Apr 2023 02:31:14.377 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1  | 1:C 20 Apr 2023 02:31:14.377 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1  | 1:C 20 Apr 2023 02:31:14.377 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1  | 1:M 20 Apr 2023 02:31:14.377 * monotonic clock: POSIX clock_gettime
redis_1  | 1:M 20 Apr 2023 02:31:14.378 * Running mode=standalone, port=6379.
redis_1  | 1:M 20 Apr 2023 02:31:14.378 # Server initialized
redis_1  | 1:M 20 Apr 2023 02:31:14.378 * Ready to accept connections
web_1    |  * Serving Flask app 'app.py' (lazy loading)
web_1    |  * Environment: production
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |    Use a production WSGI server instead.
web_1    |  * Debug mode: off
web_1    |  * Running on all addresses.
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |  * Running on http://172.19.0.2:5000/ (Press CTRL+C to quit)

4.2 docker-compose ps

View the containers started by docker-compose

docker-compose ps
       Name                     Command               State           Ports         
------------------------------------------------------------------------------------
usecompose_redis_1   docker-entrypoint.sh redis ...   Up      6379/tcp              
usecompose_web_1     flask run                        Up      0.0.0.0:8000->5000/tcp

4.3 docker-compose down

Stop containers, delete containers, remove custom networks

docker-compose down
Stopping usecompose_web_1   ... done
Stopping usecompose_redis_1 ... done
Removing usecompose_web_1   ... done
Removing usecompose_redis_1 ... done
Removing network usecompose_default

docker-compose ps
Name   Command   State   Ports
------------------------------

4.4 docker-compose restart

Restart container

docker-compose restart
Restarting usecompose_web_1   ... done
Restarting usecompose_redis_1 ... done

4.5 docker-compose logs

View container logs

docker-compose logs
Attaching to usecompose_web_1, usecompose_redis_1
redis_1  | 1:C 20 Apr 2023 02:31:14.377 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1  | 1:C 20 Apr 2023 02:31:14.377 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
web_1    |  * Serving Flask app 'app.py' (lazy loading)
redis_1  | 1:C 20 Apr 2023 02:31:14.377 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
web_1    |  * Environment: production
redis_1  | 1:M 20 Apr 2023 02:31:14.377 * monotonic clock: POSIX clock_gettime
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
redis_1  | 1:M 20 Apr 2023 02:31:14.378 * Running mode=standalone, port=6379.
web_1    |    Use a production WSGI server instead.
redis_1  | 1:M 20 Apr 2023 02:31:14.378 # Server initialized
web_1    |  * Debug mode: off
redis_1  | 1:M 20 Apr 2023 02:31:14.378 * Ready to accept connections
web_1    |  * Running on all addresses.
redis_1  | 1:signal-handler (1681958294) Received SIGTERM scheduling shutdown...
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
redis_1  | 1:M 20 Apr 2023 02:38:14.877 # User requested shutdown...
web_1    |  * Running on http://172.19.0.2:5000/ (Press CTRL+C to quit)
redis_1  | 1:M 20 Apr 2023 02:38:14.877 * Saving the final RDB snapshot before exiting.
web_1    | 30.236.61.90 - - [20/Apr/2023 10:31:54] "GET / HTTP/1.1" 200 -
redis_1  | 1:M 20 Apr 2023 02:38:14.879 * DB saved on disk
web_1    |  * Serving Flask app 'app.py' (lazy loading)
redis_1  | 1:M 20 Apr 2023 02:38:14.880 # Redis is now ready to exit, bye bye...
web_1    |  * Environment: production
redis_1  | 1:C 20 Apr 2023 02:38:15.182 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |    Use a production WSGI server instead.
redis_1  | 1:C 20 Apr 2023 02:38:15.182 # Redis version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
web_1    |  * Debug mode: off
redis_1  | 1:C 20 Apr 2023 02:38:15.182 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
web_1    |  * Running on all addresses.
redis_1  | 1:M 20 Apr 2023 02:38:15.182 * monotonic clock: POSIX clock_gettime
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * Running mode=standalone, port=6379.
web_1    |  * Running on http://172.19.0.2:5000/ (Press CTRL+C to quit)
redis_1  | 1:M 20 Apr 2023 02:38:15.183 # Server initialized
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * Loading RDB produced by version 7.0.11
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * RDB age 1 seconds
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * RDB memory usage when created 0.82 Mb
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * Done loading RDB, keys loaded: 0, keys expired: 0.
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * DB loaded from disk: 0.000 seconds
redis_1  | 1:M 20 Apr 2023 02:38:15.183 * Ready to accept connections

Print a specific services log

docker-compose logs -f web
Attaching to usecompose_web_1
web_1    |  * Serving Flask app 'app.py' (lazy loading)
web_1    |  * Environment: production
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |    Use a production WSGI server instead.
web_1    |  * Debug mode: off
web_1    |  * Running on all addresses.
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |  * Running on http://172.19.0.2:5000/ (Press CTRL+C to quit)
web_1    | 30.236.61.90 - - [20/Apr/2023 10:31:54] "GET / HTTP/1.1" 200 -
web_1    |  * Serving Flask app 'app.py' (lazy loading)
web_1    |  * Environment: production
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |    Use a production WSGI server instead.
web_1    |  * Debug mode: off
web_1    |  * Running on all addresses.
web_1    |    WARNING: This is a development server. Do not use it in a production deployment.
web_1    |  * Running on http://172.19.0.2:5000/ (Press CTRL+C to quit)

4.6 docker-compose config

View docker-compose configuration

docker-compose config --services
web
redis

docker-compose config  # 实际就是docker-compose.yml 
services:
  redis:
    image: redis:latest
  web:
    build:
      context: /home/usecompose
    ports:
    - published: 8000
      target: 5000
version: '3.3'

4.7、docker-compose images

List images

docker-compose images
    Container          Repository      Tag       Image Id      Size 
--------------------------------------------------------------------
usecompose_redis_1   redis            latest   eca1379fe8b5   112 MB
usecompose_web_1     usecompose_web   latest   1ffb139f73c9   890 MB

4.8、docker-compose top

Display container process information

docker-compose top
usecompose_redis_1
  UID      PID    PPID    C   STIME   TTY     TIME             CMD        
--------------------------------------------------------------------------
polkitd   56399   56337   0   10:38   ?     00:00:00   redis-server *:6379

usecompose_web_1
UID     PID    PPID    C   STIME   TTY     TIME                        CMD                   
---------------------------------------------------------------------------------------------
root   56868   56814   0   10:38   ?     00:00:00   /usr/bin/python3 /usr/local/bin/flask run

4.9 docker-compose scale

Set the number of containers running the specified service. Set the quantity through the parameter service=num

docker-compose scale web=3 redis=1
WARNING: The scale command is deprecated. Use the up command with the --scale flag instead.
WARNING: The "web" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash.
Starting usecompose_web_1 ... done
Starting usecompose_web_3 ... done
Creating usecompose_web_4 ... done
Desired container number already achieved

Reference documentation:

1、https://blog.csdn.net/qq_36148847/article/details/79427878

2、https://blog.csdn.net/m0_64284147/article/details/126917782