[Docker] Compose container arrangement & LNMP on the cloud

What is Docker-Compose

Compose is a tool software launched by Docker, which can manage multiple Docker containers to form an application. You need to define a configuration file docker-compose.yml in YAML format, and write the calling relationship between multiple containers. Then, you can start/stop these containers at the same time with just one command

Docker-Compose is an official open source project of Docker, which is responsible for the rapid orchestration of Docker container clusters.

passA separate docker-compose.yml template file(YAML format) to define a set of associated application containers as a project.

Download and install

official website

https://docs.docker.com/compose/compose-file/compose-file-v3/
Official website download

https://docs.docker.com/compose/install/
Install
1. curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
2. chmod +x /usr/local/bin/docker-compose
3. docker-compose --version
uninstall

Compose core concept

a file
docker-compose.yml
two elements
- service
  
  Each application container instance, such as order microservice, inventory microservice, mysql container, nginx container or redis container
- project
  
  A complete business unit consisting of a set of associated application containers is defined in the docker-compose.yml file.
three steps
- Write Dockerfile to define each microservice application and build the corresponding image file
- Use docker-compose.yml to define a complete business unit and arrange each container service in the overall application.
- Finally, execute the docker-compose up command to start and run the entire application to complete one-click deployment and go online

Compose common commands

Order	illustrate
`docker-compose -h`	view help
`docker-compose up`	Start all docker-compose services
`docker-compose up -d`	Start all docker-compose services and run in the background
`docker-compose down`	Stop and delete containers, networks, volumes, images
`docker-compose exec yml里面的服务id`	Enter the service id written in the docker-compose exec docker-compose.yml file inside the container instance /bin/bash
`docker-compose ps`	Display all containers currently orchestrated by docker-compose
`docker-compose top`	Show the current container process orchestrated by docker-compose
`docker-compose logs yml里面的服务id`	View container output logs
`docker-compose config`	check configuration
`docker-compose config -q`	Check the configuration, if there is a problem, there will be output
`docker-compose restart`	restart service
`docker-compose start`	start service
`docker-compose stop`	Out of service

Django+Mysql+Redis+Nginx deployment

deployment architecture

According to the principle of one process and one container, there are a total of the following containers in this deployment (celery and uwsgi are highly coupled with web and have not been split)
- Redis Container: Caching Service
- Mongo container: user behavior log storage
- Mysql container: data storage
- Django (uwsgi, celery) container: handling dynamic requests
- Nginx container: reverse proxy, handling static resources
- featech_v2_0 is the django project directory
- The deployment folder holds configuration information for four containers other than the Django container and other files for other mounts (shared volumes of the container and the host).
- Dockerfile: docker environment file
- docker-compose.yml: Arranging container files

Build django container - - - dockerfile writing

# 使用python:3.7.3作为基础镜像
FROM python:3.7.3

# 设置 python 环境变量
ENV PYTHONUNBUFFERED 1

# 创建项目目录并将本地项目目录文件复制到容器
RUN mkdir /featech_v2_0
COPY ./featech_v2_0  /featech_v2_0

# 设置工作目录，后续容器的操作都是基于此目录
WORKDIR /featech_v2_0

# 设置pip源（科学上网者可忽略）
RUN pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
RUN pip config set install.trusted-host mirrors.aliyun.com

# 升级pip并安装python依赖
RUN pip install -U pip
RUN pip install -r /featech_v2_0/requirement.txt

# 暴露端口8888
EXPOSE 8888

Build Nginx container

Nginx container dockerfile writing (this step is a bit redundant, in fact, here are simply created a few folders, which can be executed by the command in docker-compose.yml)

dockerfile writing

# nginx镜像
FROM daocloud.io/nginx

# 创建静态资源文件夹和ssl证书保存文件夹
RUN RUN mkdir -p /usr/share/nginx/html/static;mkdir -p /usr/share/nginx/html/media;mkdir -p /usr/share/nginx/ssl

Modify the Nginx configuration file

When configuring the reverse proxy, note that the host must be changed to web, and web is the name of the django container (configured in docker-compose.yml)

server {
    listen 80; # 监听80端口
    server_name  127.0.0.1;  # 生产环境请换成域名
    location / {
        proxy_pass http://web:8000; # 反向代理 django容器8000端口，web为django容器名称，切记不要写域名或者ip
        proxy_set_header Host $host;
        proxy_redirect off;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /static/ {
        alias /usr/share/nginx/html/static/; #静态资源路径
    }
    location /media/ {
        alias /usr/share/nginx/html/media/; #上传文件路径
    }
}

docker-compose arranges containers

docker-compose.yml

version: '3'
services:
  redis:
    image: daocloud.io/redis:3
    container_name: featech_redis
    command: redis-server
    volumes:
      - ./deployment/redis/data:/data  # 挂载数据库数据文件夹
    ports:
      - "6373:6379"
    restart: always # always表容器运行发生错误时一直重启
  
  db:
    image: mysql:5.7
    container_name: featech_db
    ports:
      - "3303:3306"
    volumes:
      - ./deployment/mysql/data:/var/lib/mysql # 挂载数据库数据
      #    - ./deployment/mysql/conf/my.cnf:/etc/mysql/my.cnf # 挂载配置文件
      - ./deployment/mysql/init:/docker-entrypoint-initdb.d/ # 挂载数据初始化sql脚本
    
    environment:
      MYSQL_ROOT_PASSWORD: 12345678  # 数据库密码
      MYSQL_DATABASE: fetach_v2_0  # 数据库名称
      LNAG: C.UTF-8
    command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
  web:
    build: .
    container_name: featech_v2_0
    
    # tty: true  如果web容器出错想进入容器做修改时，此配置可分配一个伪终端让容器先运行起来
    
    # 容器启动后启动通过uwsgi启动django应用并启动celery,这里我简单实用了nohup启动celery，可选择使用supervisor管理
    command: uwsgi --ini uwsgi && nohup celery -A celery_tasks.sms.tasks worker -l info >> celery_log/celery.log 2>&1 &
    
    depends_on:
      - db
      - redis
      - mongo
    links:
      - db
      - redis
      - mongo
    volumes:
      - ./featech_v2_0:/featech_v2_0  # 挂载项目目录
    restart: always
    ports:
      - "8888:8888"
  nginx:
    build: deployment/nginx
    container_name: featech_nginx
    ports:
      - "80:80"
      - "443:443"
    expose:
      - "8888"
    volumes:
      - ./featech_v2_0/static:/usr/share/nginx/html/static # 挂载静态文件
      - ./featech_v2_0/frontEnd:/usr/share/nginx/html/frontEnd # 挂载静态文件
      - ./featech_v2_0/media:/usr/share/nginx/html/upload # 挂载上传文件
      - ./deployment/nginx/ssl:/usr/share/nginx/ssl # 挂载ssl证书目录
      - ./deployment/nginx/conf/conf.d:/etc/nginx/conf.d # 挂载配置文件
    links:
      - web
    depends_on:
      - web
    restart: always
  mongo:
    image: mongo:4.0
    container_name: featech_mongo
    hostname: mongo
    restart: always
    ports:
      - "27013:27017"
    environment:
      TZ: Asia/Shanghai
    #MONGO_INITDB_DATABASE: test
    #MONGO_INITDB_ROOT_USERNAME: root
    #MONGO_INITDB_ROOT_PASSWORD: 123456
    volumes:
      - /etc/localtime:/etc/localtime
      - ./deployment/mongo/data:/data/db
      - ./deployment/mongo/init:/docker-entrypoint-initdb.d/
    command: mongod

redis, db, web, nginx, mongo are container names.

image means to pull the image name, and build will search for the Dockerfile in the given directory and build the container environment.
expose means to expose the port to other containers, but not to the host (different containers are isolated from each other by default).
ports means mapping the container port to the host port (read from right to left, for example ports: -

"3303:3306" refers to mapping port 3306 of the container to port 3303 of the host), and the container port will also be open to other containers.
volumes

It means to mount, which is to map the files of the local machine and the files in the container. The container and the local environment are originally isolated. Mounting is equivalent to digging a small hole so that the data between the two can communicate.
links means to interconnect containers.
depends_on: Indicates the dependency relationship, because the container starts in sequence, and the django container depends on the mysql container and redis

Container (django needs to read and write data from database and cache), and nginx depends on django container (nginx container needs to reverse proxy port 8888 of django container)

Django project configuration

settings.py file configuration

Change the database connection HOST to the mysql container name db

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',  # mysql驱动
        'NAME': 'fetach_v2_0',  # 数据库名称
        'USER': 'root',  # 登录帐号
        'PASSWORD': '12345678',  # 登录密码
        'HOST': 'db',  # 主机地址（容器部署）
        # 'HOST': '127.0.0.1',  # 主机地址
        'PORT': '3306',  # 端口
        'OPTIONS': {'charset': 'utf8mb4'},
    }
}

Change the host in the cache configuration to the redis container name redis (if you have configured redis as a cache, please ignore it if you have not configured it)

CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': 'redis://redis:6379',  # redis（容器）
        # 'LOCATION': '127.0.0.1:6379',
        'OPTIONS': {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "CONNECTION_POOL_KWARGS": {"max_connections": 100},
            'SOCKET_TIMEOUT': 10,
        },
    },
}

For production environment deployment, please change DEBUG = True in settings.py to DEBUG = False to close the debug mode.

broker_url = "redis://redis/2"
result_backend = "redis://redis/3"

custom_web

Due to the Docker-Compose orchestration, project debugging cannot be practiced locally, so I decided to manually start the containers one by one

mysql container

Externally exposed port 3307

[client]
default_character_set=utf8
[mysqld]
collation_server = utf8_general_ci
character_set_server = utf8

insert image description here

redis container

External exposure6373

Django container

Requirements file generation
Remember to add uwsgi module

Dockerfile file writing

FROM python:3.10.10
MAINTAINER [email protected]
# 以下是重点，.的意思是把当前目录的所有文件（包含文件夹）copy到镜像/SeeStar中
RUN mkdir /custom_web
COPY ./custom_web  /custom_web
WORKDIR /custom_web
RUN pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
RUN pip config set install.trusted-host mirrors.aliyun.com
RUN pip install -U pip
RUN pip install -r /custom_web/requirements.txt

CMD ["uwsgi", "--ini", "uwsgi.ini"]

Writing uwsgi.ini file

[uwsgi]
# Django-related settings
#socket = :8001
http = :8888
# the base directory (project full path)
chdir = /custom_web
# Django s wsgi file
wsgi-file = /custom_web/custom_web/wsgi.py
master = true
processes = 4
chmod-socket = 664
vacuum = true
pidfile = pid.uwsgi
# run process background and save log to daemonize
# daemonize = UWSGI.log

insert image description here

Construct
start up
exhibit

nginx-container

docker run -it -p 80:80 
-v ./custom_web/static:/usr/share/nginx/html/static 
-v ./custom_web/frontEnd:/usr/share/nginx/html/frontEnd 
-v ./custom_web/media:/usr/share/nginx/html/upload 
-v ./nginx/ssl:/usr/share/nginx/ssl 
-v ./nginx/conf/conf.d:/etc/nginx/conf.d 
605c77e624dd

insert image description here

[Docker] Compose container arrangement & LNMP on the cloud

Article directory

What is Docker-Compose

Download and install

official website

Official website download

Install

uninstall

Compose core concept

a file

two elements

three steps

Compose common commands

Django+Mysql+Redis+Nginx deployment

deployment architecture

Build django container - - - dockerfile writing

Build Nginx container

docker-compose arranges containers

Django project configuration

custom_web

mysql container

redis container

Django container

Guess you like