Kafka environment installation
Apache Kafka is an open source distributed event streaming platform, a high-throughput distributed publish-subscribe message system, which can process all action stream data of consumers in the website.
1. Download address
2. Install
Note: kafka needs jdk support, install jdk first
# ubuntu环境下安装jdk
$ sudo apt install openjdk-11-jre-headless
kafka decompression
$ tar -xzf kafka_2.13-3.1.0.tgz
$ cd kafka_2.13-3.1.0
3. Start the kafka environment
# 启动ZooKeeper服务
# 将来，kafka将不在需要ZooKeeper
$ bin/zookeeper-server-start.sh config/zookeeper.properties
# 在另一个终端启动kafka服务
$ bin/kafka-server-start.sh config/server.properties
4. Create a topic to store events
$ bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
5. The producer writes some events to the topic
$ bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
This is my first event
This is my second event
6. Consumers receive events
$ bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9092
This is my first event
This is my second event
7. Kafka structure
Figure: This example topic has four partitions P1–P4. Two different Producer clients write events to the topic's partitions over the network, publishing new events to the topic independently of each other. Events with the same key (indicated by their color in the graph) are written to the same partition. Note that both producers can write to the same partition if appropriate.