Based docker build a zookeeper cluster, kafka --- two clusters (cluster between multiple real machine)

Three real machine, using the host vessel

192.168.0.128  192.168.0.141 192.168.0.142

zookeeper Cluster Setup

Execute scripts on each machine,

#!/bin/bash
#Get zookeeper image
zkimage=`docker images | grep zookeeper | awk {'print $1'}`
if [ -n "$zkimage" ]
then
    echo 'The zookeeper image is already existed.'
else
    echo 'Pull the latest zookeeper image.'
    docker pull zookeeper
fi

#Create network for zookeeper containers
zknet=`docker network ls | grep yapi_net | awk {'print $2'}`
if [ -n "$zknet" ]
then
    echo 'The zknetwork is already existed.'
else
    echo 'Create zknetwork.'
    docker network create --subnet 172.30.0.0/16 yapi_net
fi

#Start zookeeper cluster
echo 'Start 3 zookeeper servers.'
rm -rf   /opt/zookeeper_1/data  /opt/zookeeper_1/datalog  /var/log/zookeeper_1/log
rm -rf   /opt/zookeeper_2/data  /opt/zookeeper_2/datalog  /var/log/zookeeper_2/log
rm -rf   /opt/zookeeper_3/data  /opt/zookeeper_3/datalog  /var/log/zookeeper_3/log

mkdir -p  /opt/zookeeper_1/data  /opt/zookeeper_1/datalog  /var/log/zookeeper_1/log
mkdir -p  /opt/zookeeper_2/data  /opt/zookeeper_2/datalog  /var/log/zookeeper_2/log
mkdir -p  /opt/zookeeper_3/data  /opt/zookeeper_3/datalog  /var/log/zookeeper_3/log


ZOO_SERVERS="server.1=192.168.0.128:2888:3888 server.2=192.168.0.142:2888:3888 server.3=192.168.0.141:2888:3888"
docker run --network host -d --restart always -v /opt/zookeeper_1/data:/data -v /opt/zookeeper_1/datalog:/datalog -v /var/log/zookeeper_1/log:/logs  -e ZOO_SERVERS="$ZOO_SERVERS" -e ZOO_MY_ID=1 --name zookeeper_1 -p 2182:2181 -p 2888:2888 -p 3888:3888 docker.io/zookeeper

Note that each machine id number of different

Different official mirror is possible to configure the environment variables, you need to modify the script, reference Entrypoint files, and configuration files, configuration files are as follows:

clientPort=2181
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
server.1=192.168.0.128:2888:3888
server.2=192.168.0.142:2888:3888
server.3=192.168.0.141:2888:3888

Stand-alone (non-clustered) configuration file, the Telnet ip = 2181 "STAT ==" Mode: Standalone

clientPort=2181
dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60

kafka cluster

Execute scripts on each machine,

#!/bin/bash
#Get zookeeper image
kfkimage=`docker images | grep 'docker.io/wurstmeister/kafka' | awk {'print $1'}`
if [ -n "$kfkimage" ]
then
    echo 'The docker.io/wurstmeister/kafka is already existed.'
else
    echo 'Pull the image.'
    docker pull docker.io/wurstmeister/kafka
fi

#Create network for zookeeper containers
kfknet=`docker network ls | grep yapi_net | awk {'print $2'}`
if [ -n "$kfknet" ]
then
    echo 'The kfknetwork is already existed.'
else
    echo 'Create kfknetwork.'
    docker network create --subnet 172.30.0.0/16 yapi_net
fi

#Start 3  zookeeper cluster
echo 'Start 3 kafka servers.'

rm-rf / opt / kafka_1 / LogData
 rm -rf / opt / kafka_2 / LogData
 rm -rf / opt / kafka_3 / LogData 

mkdir -p / opt / kafka_1 / LogData
 mkdir -p / opt / kafka_2 / LogData
 mkdir -p / opt / kafka_3 / LogData 

#kafka ip 
kfk_1_ip = ' 172.30.0.41 ' 
kfk_2_ip = ' 172.30.0.42 ' 
kfk_3_ip = ' 172.30.0.43 ' 
#zk_jiqun_ip = ' 172.30.0.31:2181 ' 
zk_jiqun_ip = '192.168.0.128:2181,192.168.0.142:2181,192.168.0.141:2181'
docker run --restart always -d --name kafka_1 --network host -e KAFKA_ZOOKEEPER_CONNECT=${zk_jiqun_ip} -e KAFKA_LISTENERS='PLAINTEXT://0.0.0.0:9092' -e KAFKA_BROKER_ID='141' -e KAFKA_ADVERTISED_LISTENERS='PLAINTEXT://192.168.0.141:9092' -v /opt/kafka_1/logdata:/kafka -p 9092:9092  docker.io/wurstmeister/kafka

Note the id and PLAINTEXT: //192.168.0.141: 9092, (after the container is created, modified and then into the container, will reset after the restart, when it is necessary to create a container, it is necessary to specify the environment variable)