Use docker to install mysql, redis, kafka and other services

foreword

Roughly speaking, the role of docker is as follows

For most applications, developers can create images through docker build, upload images through docker push, users can download images through docker pull, and run applications with docker run.

Users no longer need to care about how to build the environment, how to install, and how to resolve library conflicts in different distributions —and usually it will not need to consume more hardware resources and will not significantly reduce performance. That is, the realization of standardization, container

The article is relatively long, if you want to search for the docker installation of a specific service, make good use of the left directory

If you want to use it easily, you can read this article from the answerer:

https://blog.csdn.net/2301_76154806/article/details/128781197

Continuously updating...

Pen and ink are not easy, give someone a rose, leave a fragrance in your hand

---

1. mysql

Determine which mysql version to install, the driver names of version 5 and version 8 are different

1.1 Default configuration installation

1. Download mirror

docker pull mysql:version number

Version number: generally 5.7 or 8.5
version is relatively simple to install, and can be accessed remotely without manual

2. Create an application container docker run --name some-mysql -d -t -p 3306:3306 -e MYSQL_ROOT_PASSWORD=my-secret-pw mysql: version number

'my-secret-pw' is the password of the mysql root user you set

1.2 Custom configuration installation

If you need to configure some operating parameters of mysql, you can refer to the following two methods.

1. [Recommended] Create an application container with configuration parameters

docker run --name some-mysql -d -t -p 3306:3306 -e parameter name 1 = value of parameter name 1 -e parameter name 2 = value of parameter name 2 mysql: version number --character-set-server =utf8mb4

例: docker run --name some-secret-mysql -d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=root mysql:5.7 --default_storage_engine=InnoDB --lower_case_table_names=1 --max_allowed_packet=50M --character-set-server=utf8mb4 --sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

Common parameters - mandatory parameters 1. MYSQL_ROOT_PASSWORD=root

Set the mysql root user password to root

mysql8 version docker installation ==========

docker pull mysql:8.0.18

docker run -di --name=mysql8 -p 3306:3306 -v /usr/local/tmp:/etc/mysql/tmp --privileged=true -e MYSQL_ROOT_PASSWORD=xxx mysql:8.0.18

docker exec -it mysql8 /bin/bash

mysql -uroot -pxxx

use mysql;

select host,user from user; 让root能远程连接，host修改为"%"，如果是的就不用修改了

navicate for mysql 远程连接
use mysql;
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%';
FLUSH PRIVILEGES;

修改密码为用不过期

    ALTER USER 'root'@'%' IDENTIFIED BY '123456' PASSWORD EXPIRE NEVER;

修改密码并指定加密规则为 mysql_native_password

    ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '123456';

刷新权限

    FLUSH PRIVILEGES;

这样就可以远程连接数据库了；

---

• optional parameters

• MYSQL_DATABASE=mydb After the mysql container is created, create a new database named mydb in mysql

• MYSQL_USER=selton MYSQL_PASSWORD=seltonpassword Create a mysql user named selton and set his password as seltonpassword. This user has administrator authority for all databases created by the MYSQL_DATABASE parameter

• --character-set-server=utf8mb4 is equivalent to the configuration of some key-value pairs in our cnf configuration file of mysql. Here is to select character-set-server, which is the default character set created by mysql, as utf8mb4. If you use navicat The connection tool creates the database. If the database character type is not selected, the utf8mb4 character type set above will be used.

• --sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION In version 5.7, there is a configuration with only_full_group_by in sql_mode by default, which is mandatory for query statements with groupby In the select column must be in the result column of groupby Within, otherwise it will fail. This condition is too strict, and it is not added under normal circumstances, so the above configuration needs to be added under normal circumstances.

• If you want to view the complete configurable parameters, run the following command, remember to replace the tag with the mysql version you downloaded

docker run -it --rm mysql:tag --verbose --help

2. Map the local configuration file to the application container

docker run --name some-mysql-p 3306:3306-v /my/custom:/etc/mysql/conf.d -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag

Create a new configuration file my.cnf in the host of the container (use this name), and then replace /my/custom with the absolute path of the folder where the my.cnf configuration file in the host is located, so that the mysql container All the configurations in my.cnf will be used to create mysql when it is created. Similarly, these configurations in my.cnf can be configured by using the value of -e parameter 1=parameter 1

Attachment : mysql common configuration

port = 3306 #默认
pid-file =  /data/mysql/mysql.pid
basedir =  /usr/local/mysql/ #程序安装目录
symbolic-link = 0  #多客户访问同一数据库，该选项默认开启
tmpdir =  /usr/local/mysql/tmp/ #此目录被 MySQL用来保存临时文件
open_files_limit = 65535 #打开时，和max_connections对比，取大数
datadir = /var/lib/mysql  #数据库目录
log-error = /var/lib/mysql/error.log
slow_query_log=on   #开启慢查询日志相关
long_query_time=2   #默认10秒
slow_query_log_file = /var/log/mysql/slow_query.log  #慢查询日志路径
log-queries-not-using-indexes = 1   #记录没有使用索引的sql

socket=/var/lib/mysql/mysql.sock  #该条配置需在[client]段同时配置
default_storage_engine=InnoDB
innodb_file_per_table = on     #InnoDB为独立表空间模式，每个数据库的每个表都会生成一个数据空间
innodb_buffer_pool_size=4G     #生产中要改，建议为操作系统内存的70%-80%，需重启服务生效
skip_name_resolve = on         #忽略主机名解析，提高访问速度（注意配置文件中使用主机名将不能解析）
lower_case_table_names = 1     #忽略表单大小写
character-set-server=utf8mb4   #设定默认字符为utf8mb4

Note: If the data volume is used to map the data file, most of the mysql configuration including the password follows the data file, and the commands created by docker no longer work

2. tomcat

application (java) server

2.1 installation

1. Download the image docker pull tomcat: version number

Versions 7, 8, and 9 are now more commonly used

2. Create an application container

If you want to test it, run the following command, visit ip:8080, you will visit the home page of tomcat

docker run --name some-tomcat -p 8080:8080 -d -t tomcat:version number

However, generally tomcat is used to publish a webapp, so if you want to publish a webapp, as follows

If you get a war, unzip it to a folder, then rename the folder to ROOT

docker run --name some-tomcat -p 8080:8080 -d -t -v /root/selton/mywebapp:/usr/local/tomcat/webapps tomcat:version number

/root/selton/mywebapp, this folder contains your ROOT folder

custom configuration

If you need to modify some configuration files in the conf directory of tomcat, you need to copy the original tomcat

3. gog

git private server

Create a new file docker-compose.yml

input content

version: '2' services: Mysql: image: gogs/gogs:0.11.79 container_name: gogs ports: - "9876:22" - "9092:3000"

Then docker-compose up -d starts

Access local ip:8080

this setting

ports:
  - "8070:22"
  - "8080:3000"

Map ports 22 and 3000 in the container to ports 8070 and 8080 of the host machine respectively

That is, we need to ensure that the host machine, that is, the local machine, does not occupy ports 8070 and 8080 of the machine that generates docker-compose.yml

gogs is configured on the web page -- worth learning from

Mapping port 3000 to port 80 with docker is risky. Specifically, when restarting, an error is reported and bind 80 permission denied cannot be started.

At this time, the port in the ini configuration file should actually be 3000, and then docker maps the host machine 80 to this 3000, but the internal should not be 80

gogs docker version configuration file path /data/gogs/conf/app.ini

https://github.com/gogs/gogs/issues/3503

installation interface

first of all

database settings

directly see the database name

We need to have a database connection that can be accessed externally, and there is a database, just use the name of the example -> gogs, we need to create a new database called gogs in the database connection

如果您使用 MySQL，请使用 INNODB 引擎以及 utf8_general_ci 字符集

Need to modify the database host ip, 127.0.0.1 is changed to the ip exposed by the database

Fill in the database user password

The next step is to apply the basic settings

Modify the domain name to the ip of the machine where we just installed gogs

Next modify the url

Modify the localhost of the application url to the ip of the machine where gogs is installed

Note that if docker is used here for port mapping, change 3000 to the mapped port

Next is the optional settings section

Mail service settings regardless

Click on Server and other service settings

Uncheck Enable Verification Code Service

Tick ​​Prohibit users from self-registration to start login access restrictions

Canceling the self-registration of users must mean that we need a super user, and the super user creates an account and distributes

Click on administrator account settings

Enter basic information, email address can be entered casually

You're done, click to install now

version: '2'
services:
  Mysql:
    image: mysql:5.6
    container_name: gogsMysql5.6
    ports:
      - "3306:3306"
    environment:
      - MYSQL_ROOT_PASSWORD=root
    volumes:
      - /root/gogs/mysqlConf:/etc/mysql/conf.d
      - /root/gogs/createSh:/docker-entrypoint-initdb.d
  Gogs:
    depends_on:
      - Mysql
    image: gogs/gogs:0.11.79
    container_name: gogs
    ports:
      - "8070:22"
      - "8080:3000"
    volumes:
      - /var/gogs:/data

4. existing

从日志 2019-12-09 10:25:49,214 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/jira/dbconfig.xml

可以看到jira启动会从这个配置中读取数据库配置

在宿主机中新建配置文件

<?xml version="1.0" encoding="UTF-8"?>
<jira-database-config>
<name>defaultDS</name>
<delegator-name>default</delegator-name>
<database-type>mysql</database-type>
<jdbc-datasource>
<url>jdbc:mysql://10.2.132.13:3306/jira?useUnicode=true&amp;characterEncoding=UTF8&amp;sessionVariables=storage_engine=InnoDB
</url>
<driver-class>com.mysql.jdbc.Driver</driver-class>
<username>root</username>
<password>root</password> 
<!--以下参数可以不用配置 -->
<pool-min-size>20</pool-min-size>
<pool-max-size>20</pool-max-size>
<pool-max-wait>30000</pool-max-wait>
<validation-query>select 1</validation-query>
<min-evictable-idle-time-millis>60000</min-evictable-idle-time-millis>
<time-between-eviction-runs-millis>300000
</time-between-eviction-runs-millis>
<pool-max-idle>20</pool-max-idle>
<pool-remove-abandoned>true</pool-remove-abandoned>
<pool-remove-abandoned-timeout>300</pool-remove-abandoned-timeout>
<pool-test-while-idle>true</pool-test-while-idle>
<validation-query-timeout>3</validation-query-timeout>
</jdbc-datasource>
</jira-database-config>

修改相关数据库配置

docker run -p 10909:8080 --name some-jira -d -e TZ='Asia/Shanghai' -v /my/conf: haxqer/jira

5. clickhouse

clickhouse在dockerhub的官网

docker run -dit --name some-clickhouse -e CLICKHOUSE_DB=test1 -e CLICKHOUSE_USER=root -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 -e CLICKHOUSE_PASSWORD=hexin -p 9000:9000 -p 8123:8123 clickhouse/clickhouse-server

用户名root,密码hexin,连接端口8123

6. redis

在Reids 4.x之后，Redis新增了模块功能特性，通过外部拓展，可以实现新的Redis命令，通过写c语言并编译出.so文件，可实现代码执行漏洞。

针对未授权或弱口令的Redis服务，攻击者通过构造特定请求，成功利用漏洞可在目标服务器上执行任意命令，风险极大。攻击者常常使用这个漏洞将用户的服务器变为"肉鸡","矿机"

修复的方式有很多种, 最有效的方式是redis仅在内网环境中使用,最简单的方式是redis连接的时候指定密码

下面是docker创建redis容器并指定密码

尽管现在已经有了6.x版本,但是它是redis有史以来最大的一个版本，所以即使它是稳定的，也要小心，在投入生产之前测试工作负载

docker pull redis
docker run --name some-redis -d -it -p 16379:6379 redis --requirepass "u}J#D=>MryWf" --appendonly yes

如果想要持久化redis中的数据,添加数据卷映射

docker run --name some-redis -d -it -v /your/host/want/store/redisdata/path:/data -p 16379:6379 redis --requirepass "u}J#D=>MryWf" --appendonly yes

7. zookeeper

docker pull zookeeper
docker network create app-bridge --driver bridge
docker run --name some-zookeeper --network app-bridge -e ALLOW_ANONYMOUS_LOGIN=yes -d -it -p 2181:2181 bitnami/zookeeper

8. kafka

注意: 此kafka的安装依赖与zookeeper的安装

#节点1 
docker run -d --name=kafka2 \
 -p 9092:9092 \
 --network app-bridge \
 -e ALLOW_PLAINTEXT_LISTENER=yes \
 -e KAFKA_CFG_ZOOKEEPER_CONNECT=81.70.199.213:2181 \
 -e KAFKA_BROKER_ID=2 \
 -e KAFKA_HEAP_OPTS="-Xmx180m -Xms180m" \
 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://81.70.199.213:9092 \
 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092  \
 bitnami/kafka

 # 节点2
 docker run -d --name=kafka3 \
 -p 9093:9092 \
 --network app-bridge \
 -e ALLOW_PLAINTEXT_LISTENER=yes \
 -e KAFKA_CFG_ZOOKEEPER_CONNECT=81.70.199.213:2181 \
 -e KAFKA_BROKER_ID=3 \
  -e KAFKA_HEAP_OPTS="-Xmx180m -Xms180m" \
 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://81.70.199.213:9093 \
 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092  \
 bitnami/kafka

ALLOW_PLAINTEXT_LISTENER=yes：允许使用PLAINTEXT侦听器

KAFKA_CFG_ZOOKEEPER_CONNECT：zookeeper集群地址，多节点,分割

KAFKA_BROKER_ID：节点id，用来指定 Kafka 集群中 broker 的唯一标识，默认值为 -1。如果没有设置，那么 Kafka会自动生成一个

KAFKA_ADVERTISED_LISTENERS：绑定公网 IP 供外部客户端使用

KAFKA_LISTENERS：绑定私网 IP 地址供 broker 间通信使用

9. dubbo-admin

docker run --name some-dubbo-admin -p 8080:8080 -d -it -e admin.registry.address=zookeeper://selton.cn:2181 -e admin.config-center=zookeeper://selton.cn:2181 -e admin.metadata-report.address=zookeeper://selton.cn:2181  apache/dubbo-admin 

10. sonar

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube

注意: 保持服务器剩余内存足够,默认情况下,sonarqube大约占用1.5G内存

打开http://yourhost:900/,点击"Log in"

登录账号：admin 密码：admin

11. flink

docker-compose.yml

version: "2.2"
services:
  jobmanager:
    image: flink:latest
    ports:
      - "18001:8081"
    command: jobmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager

  taskmanager:
    image: flink:latest
    depends_on:
      - jobmanager
    command: taskmanager
    scale: 1
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 8

12. postgres

docker run --name some-postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres -d postgres

13. fastDFS

docker pull delron/fastdfs
docker run -d --network=host --name some-tracker -v /var/fdfs/tracker:/var/fdfs delron/fastdfs tracker

#注意: 需要暴露22122端口 23000 否则下面的创建会有问题 8888端口也需要暴露 将来通过这个端口访问图片
#selton.cn替换成你的公网host或者局域网host

docker run -d --network=host --name some-storage -e TRACKER_SERVER=selton.cn:22122 -v /var/fdfs/storage:/var/fdfs -e GROUP_NAME=group1 delron/fastdfs storage

14. solr

在宿主机中执行如下命令

mkdir solrdata

sudo chmod 777 solrdata

docker run --name mall-solr -d -p 8983:8983 -v "$PWD/solrdata:/var/solr" solr:8

// 参数详解

run 运行容器

-d 后台运行

-p 容器端口和宿机端口映射

– name 容器名称

创建一个名为mall_core的核心

docker exec -it mall-solr bin/solr create_core -c mall_core

这个命令是在宿主机上执行的, 会让宿主机向容器发送一条指令
bin/solr create_core -c mall_core
这个指令,是使用solr的脚本, 创建一个名称为mall_core的核心

出现 Created new core ‘' 即创建

docker exec --user=root -it mall-solr bash

cp /opt/solr-8.10.1/contrib/analysis-extras/lucene-libs/lucene-analyzers-smartcn-8.10.1.jar /opt/solr-8.10.1/server/solr-webapp/webapp/WEB-INF/lib

在宿主机的solrdata中

vi /data/mall_core/conf/managed-schema

<fieldType name="smartcn" class="solr.TextField" positionIncrementGap="100">
       <analyzer type="index">
         <tokenizer class="org.apache.lucene.analysis.cn.smart.HMMChineseTokenizerFactory"/>
        </analyzer>
        <analyzer type="query">
          <tokenizer class="org.apache.lucene.analysis.cn.smart.HMMChineseTokenizerFactory"/>
        </analyzer>
  </fieldType>

重启docker