试用Kafka

zookeeper环境参照《Zookeeper集群环境搭建》

1、下载解压kafka

在宿主机器内操作

$ wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.1.1/kafka_2.11-1.1.1.tgz
$ tar xvf kafka_2.11-1.1.1.tgz

通过docker volumes将kafka映射到docker容器中。

2、启动docker集群

在宿主主机内操作

编写docker-compose.yml如下

version: '2'
 
networks:
  thinking:
  
services:
  test_1.thinking.com:
    image: ubuntu
    ports:
      - "3333:3333"
    container_name: test_1.thinking.com
    command: /bin/bash
    tty: true
    networks:
      - thinking
    volumes:
      - /etc/java-8-openjdk:/etc/java-8-openjdk
      - /usr/lib/jvm/java-8-openjdk-amd64:/usr/lib/jvm/java-8-openjdk-amd64
      - /home/yong/Desktop/ZK-test/zookeeper-3.4.12:/home/zookeeper-3.4.12
      - /home/yong/Desktop/kafka-test/kafka_2.11-1.1.1:/usr/local/kafka
    environment:
      - JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
      - PATH=${PATH}:/usr/lib/jvm/java-8-openjdk-amd64/bin:/usr/lib/jvm/java-8-openjdk-amd64/jre/bin

 
  test_2.thinking.com:
    image: ubuntu
    ports:
      - "4444:4444"
    container_name: test_2.thinking.com
    command: /bin/bash
    tty: true
    networks:
      - thinking
    volumes:
      - /etc/java-8-openjdk:/etc/java-8-openjdk
      - /usr/lib/jvm/java-8-openjdk-amd64:/usr/lib/jvm/java-8-openjdk-amd64
      - /home/yong/Desktop/ZK-test/zookeeper-3.4.12:/home/zookeeper-3.4.12
      - /home/yong/Desktop/kafka-test/kafka_2.11-1.1.1:/usr/local/kafka
    environment:
      - JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
      - PATH=${PATH}:/usr/lib/jvm/java-8-openjdk-amd64/bin:/usr/lib/jvm/java-8-openjdk-amd64/jre/bin

  test_3.thinking.com:
    image: ubuntu
    ports:
      - "5555:5555"
    container_name: test_3.thinking.com
    command: /bin/bash
    tty: true
    networks:
      - thinking
    volumes:
      - /etc/java-8-openjdk:/etc/java-8-openjdk
      - /usr/lib/jvm/java-8-openjdk-amd64:/usr/lib/jvm/java-8-openjdk-amd64
      - /home/yong/Desktop/ZK-test/zookeeper-3.4.12:/home/zookeeper-3.4.12
      - /home/yong/Desktop/kafka-test/kafka_2.11-1.1.1:/usr/local/kafka
    environment:
      - JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
      - PATH=${PATH}:/usr/lib/jvm/java-8-openjdk-amd64/bin:/usr/lib/jvm/java-8-openjdk-amd64/jre/bin

执行

$ docker-compose -f docker-compose.yml up -d

3、借助kafka自带daemon运行单机服务

进入test_1.thinking.com容器

$ docker exec -it test_1.thinking.com /bin/bash

启动kafka

# cd /usr/local/kafka/bin
# ./kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties

创建主题并查询

# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
# ./kafka-topics.sh --zookeeper localhost:2181 --describe --topic test
Topic:test	PartitionCount:1	ReplicationFactor:1	Configs:
	Topic: test	Partition: 0	Leader: 0	Replicas: 0	Isr: 0

向主题发消息，以ctrl+D结束

# ./kafka-console-producer.sh --broker-list localhost:9092 --topic test
>Test Msg 0001
>Test Msg 0002

从主题读取消息

# ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
Test Msg 0001
Test Msg 0002
^CProcessed a total of 2 messages

猜你喜欢