Big data cluster construction --- "kafka (cluster)" - rookie Xiaohui

Big data cluster construction-"kafka (cluster) chapter"-rookie Xiaohui

---

1. create:mkdir -p /opt/kafka
2. enter:cd /opt/kafka
3. Upload
4. Unzip:tar -zxvf kafka_2.12-1.1.0.tgz
5. Modify the configuration file server.properties

cd /opt/kafka/kafka_2.12-1.1.0/config/
vi server.properties 
//修改broker.id
broker.id=1
//修改zookeeper
zookeeper.connect=zhiyou001:2181,zhiyou002:2181,zhiyou003:2181
//文件最后添加host.name
host.name=zhiyou001

6. Copy to other nodes

scp  -r  /opt/kafka/ root@zhiyou002:/opt/kafka/
scp  -r  /opt/kafka/ root@zhiyou003:/opt/kafka/

7. Change the broker.id and host.name of other nodes.
   zhiyou002 broker.id=02
   host.name=zhiyou02
   zhiyou003 broker.id=03
   Host.name=zhiyou03
8. Start zookeeper first (3 units)

//启动zookeeper（三台）
cd /opt/zookeeper/zookeeper-3.4.12/bin/
./zkServer.sh start

9. Restart Kafka (3 units)

cd /opt/kafka/kafka_2.12-1.1.0
 ./bin/kafka-server-start.sh -daemon config/server.properties &

10. Create theme (host)

./bin/kafka-topics.sh --create --zookeeper zhiyou01:2181,zhiyou02:2181,zhiyou03:2181 --replication-factor 3 --partitions 3 --topic test

11. Start producer

./bin/kafka-console-producer.sh --broker-list zhiyou001:9092, zhiyou002:9092, zhiyou003:9092 --topic test
//可以在此之后写入内容等待消费者
./bin/kafka-console-producer.sh --broker-list zhiyou01:9092, zhiyou02:9092, zhiyou03:9092 --topic test

12. Start consumer (slave node)

./bin/kafka-console-consumer.sh --bootstrap-server zhiyou001:9092, zhiyou002:9092, zhiyou003:9092 --from-beginning --topic test

13. Test: Input the message from the producer and view the message on the consumer