In the recent study Kafka, preparing to test a cluster when the state feels both open three virtual machines or to open up three different port number in a virtual machine have too much trouble (ah .. mostly lazy).

Preparing the Environment

A computer with Internet access and there CentOS7 virtual machine computer

Why use a virtual machine? Because laptop use, so each time you connect IP network will change, but also always modify the configuration file, too cumbersome, inconvenient to test. (You can avoid this problem by way of a virtual network Docker, when the experiment did not know when)

Docker installation

If you have installed please ignore this step Docker

Docker CentOS version supports the following:
CentOS 7 (64-bit): The system as in claim 64, kernel version 3.10 or more.
CentOS 6.5 (64-bit) or later: the requirements for the system 64, the system kernel version 2.6.32-431 or higher.
CentOS kernel supports only the release of Docker.

yum install

Docker system requirements CentOS kernel version higher than 3.10, see prerequisites above to verify your CentOS version supports Docker.

# 查看内核版本
$ uname -a

#安装 Docker
$ yum -y install docker

#启动 Docker 后台服务
$ service docker start

# 由于本地没有hello-world这个镜像，所以会下载一个hello-world的镜像，并在容器内运行。
$ docker run hello-world

Script Installation

Login Centos use sudo or root privileges.
Ensure that the yum package up to date.

$ sudo yum update

Docker obtain and execute the installation script.

$ curl -fsSL https://get.docker.com -o get-docker.sh
# 执行这个脚本会添加 docker.repo 源并安装 Docker。
$ sudo sh get-docker.sh

Start Docker

$ sudo systemctl start docker

# 验证 docker 是否安装成功并在容器中执行一个测试的镜像。
$ sudo docker run hello-world
$ docker ps

Mirror accelerate

Let me start domestic mirroring configuration when I was rejected, but after using found that download speeds duang~are about to go up. It is strongly recommended that you configure mirroring domestic sources.
Open / create /etc/docker/daemon.jsonfile, add the following:

{
  "registry-mirrors": ["http://hub-mirror.c.163.com"]
}

Zookeeper Cluster Setup

Zookeeper Mirror: zookeeper: 3.4

Mirroring ready

$ docker pull zookeeper:3.4

Find mirroring can go https://hub.docker.com/
Docker pull ImagesRF Royalty Free: // TAG representatives pulling TAGversion of imageMirror

Establish an independent container Zookeeper

We begin by creating an independent with the most simple way Zookeepernode, and then we create another node according to this example.

$ docker run --name zookeeper -p 2181:2181 -d zookeeper:3.4

By default, the container configuration file, /conf/zoo.cfgdata and log default directories /dataand /datalog, if required, can be mapped to a host in the directory.
Parameter Description

--name: the name of the specified container

-p: port number of the port assigned to the container exposed

-d: print run in the background container and container ID

Cluster Setup

Other nodes in Zookeeperthe container to create a similar way to create separate containers, it is noted that the node to specify idand modify the configuration file, multi-node, create a corresponding command is as follows:

New docker network

$ docker network create zoo_kafka
$ docker network ls

Zookeeper container 1

$ docker run -d \
     --restart=always \
     -v /opt/docker/zookeeper/zoo1/data:/data \
     -v /opt/docker/zookeeper/zoo1/datalog:/datalog \
     -e ZOO_MY_ID=1 \
     -p 2181:2181 \
     -e ZOO_SERVERS="server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888" \
     --name=zoo1 \
     --net=viemall-zookeeper \
     --privileged \
     zookeeper:3.4

Zookeeper container 2

$ docker run -d \
     --restart=always \
     -v /opt/docker/zookeeper/zoo2/data:/data \
     -v /opt/docker/zookeeper/zoo2/datalog:/datalog \
     -e ZOO_MY_ID=2 \
     -p 2182:2181 \
     -e ZOO_SERVERS="server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888" \
     --name=zoo2 \
     --net=viemall-zookeeper \
     --privileged \
     zookeeper:3.4

Zookeeper container 3

$ docker run -d \
     --restart=always \
     -v /opt/docker/zookeeper/zoo3/data:/data \
     -v /opt/docker/zookeeper/zoo3/datalog:/datalog \
     -e ZOO_MY_ID=3 \
     -p 2183:2181 \
     -e ZOO_SERVERS="server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888" \
     --name=zoo3 \
     --net=viemall-zookeeper \
     --privileged \
     zookeeper:3.4

Although this method is also achieved what we wanted, but the steps are too cumbersome and troublesome to maintain them (lazy cancer late), so we use docker-composethe way to achieve.

docker-compose build zookeeper cluster

New docker network

$ docker network create viemall-zookeeper
$ docker network ls

Script writing docker-compose.yml

Use:

installation docker-compose

# 获取脚本
$ curl -L https://github.com/docker/compose/releases/download/1.25.0-rc2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
# 赋予执行权限
$chmod +x /usr/local/bin/docker-compose

Any new directory docker-compose.ymlfile, copy the following
Excuting an order docker-compose up -d

docker-compose.ymlDownload: https://github.com/JacianLiu/docker-compose/tree/master/zookeeper
docker-compose.ymldetails

version: '2'
services:
  zoo1:
    image: zookeeper:3.4 # 镜像名称
    restart: always # 当发生错误时自动重启
    hostname: zoo1
    container_name: zoo1
    privileged: true
    ports: # 端口
      - 2181:2181
    volumes: # 挂载数据卷
      - ./zoo1/data:/data
      - ./zoo1/datalog:/datalog 
    environment:
      TZ: Asia/Shanghai
      ZOO_MY_ID: 1 # 节点ID
      ZOO_PORT: 2181 # zookeeper端口号
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 # zookeeper节点列表
    networks:
      default:
        ipv4_address: 172.23.0.11

  zoo2:
    image: zookeeper:3.4
    restart: always
    hostname: zoo2
    container_name: zoo2
    privileged: true
    ports:
      - 2182:2181
    volumes:
      - ./zoo2/data:/data
      - ./zoo2/datalog:/datalog
    environment:
      TZ: Asia/Shanghai
      ZOO_MY_ID: 2
      ZOO_PORT: 2181
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      default:
        ipv4_address: 172.23.0.12

  zoo3:
    image: zookeeper:3.4
    restart: always
    hostname: zoo3
    container_name: zoo3
    privileged: true
    ports:
      - 2183:2181
    volumes:
      - ./zoo3/data:/data
      - ./zoo3/datalog:/datalog
    environment:
      TZ: Asia/Shanghai
      ZOO_MY_ID: 3
      ZOO_PORT: 2181
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      default:
        ipv4_address: 172.23.0.13

networks:
  default:
    external:
      name: zoo_kafka

verification

We can see from the figure, there is a Leader, two Flower, bringing our Zookeepercluster has built a good

Kafka Cluster Setup

With the above basis, go and get Kafkaa cluster or a problem? In fact, several different variable values only.

With the top of the examples, we do not engage in strenuous single node Kafka, the direct use of docker-composemethods to deploy three nodes, in fact, much the same way, also when it comes to the top, in fact, a number of different attributes it; this time we do not need to go New Docker network, and set up directly in front of Zookeeperthe network can be created when a cluster!