Kafka was developed by the Apache Software Foundation, an open source stream processing platform, is a high throughput of distributed publish-subscribe messaging system, it can handle all the actions of consumers streaming data in the site.
Characteristics:
(1) providing a message by O (1) persistent disk data structure, the structure for the message even when the number of TB in storage stability can be maintained long.
(2) high throughput: even a very ordinary hardware Kafka can support millions of messages per second.
(3) support to partition messaging server by Kafka and consumption of machine clusters.
(4) supports Hadoop parallel data loading.
Related terms:
(. 1) Broker
Kafka cluster comprising one or more servers, which is called the server broker.
(2) Topic
each cluster issued to Kafka's message has a category, the category is called Topic. (Topic different physically separate storage of messages, the message is logically a Topic although stored in the message, but the user to specify one or more of Topic broker to production or consumption data without concern for where the data are stored in)
(3 ) Partition
Partition is a concept of physics, each containing one or more Topic Partition.
(4) Producer
responsible for issuing messages to Kafka broker
(5) consumer
news consumers, read a message to Kafka broker clients.
(6) Consumer Group
each belonging to a specific Consumer Consumer Group (Consumer may be specified for each group name, if the group name specified in the Default group is).
Official website: http://kafka.apache.org/
Chinese document: http://kafka.apachecn.org/
First, download
Download: http://kafka.apache.org/quickstart
Click Download Open https://www.apache.org/dyn/closer.cgi?path=/kafka/2.3.0/kafka_2.12-2.3.0.tgz
click http://mirror.bit.edu.cn /apache/kafka/2.3.0/kafka_2.12-2.3.0.tgz Download
After the download, unzip, I unpacked in D: \ ProgramFiles \ Java \ kafka_2.12-2.3.0
Note: The latest version of Kafka has built zookeeper, no additional download zookeeper.
Second, start the server
Kafka use ZooKeeper, you need to start a ZooKeeper server. You can quickly and easily create a single node ZooKeeper instance by kafka convenient packaged together with the script.
Open a command line window, cd into the unpacked directory, execute the command
zookeeper-server-start.bat config\zookeeper.properties
The successful implementation will occupy 2182 port, the command line window is not closed.
Start Kafka server
open a command line window, cd into the unpacked directory, execute the command
bin\windows\kafka-server-start.bat config\server.properties
The successful implementation will occupy 9092 port, the command line window is not closed.
Third, create a topic
打开命令行窗口，cd进入解压目录，执行下面命令，创建一个名为“test”的topic，它有一个分区和一个副本：
bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
可以运行list（列表）命令来查看这个topic：
bin\windows\kafka-topics.bat --list --zookeeper localhost:2181
四、启动生产者
Kafka自带一个命令行客户端，它从文件或标准输入中获取输入，并将其作为message（消息）发送到Kafka集群。默认情况下，每行将作为单独的message发送。
打开命令行窗口，cd进入解压目录，执行下面命令，运行 producer，然后在控制台输入一些消息以发送到服务器。
bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test
五、启动消费者
打开命令行窗口，cd进入解压目录，执行下面命令，
bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test --from-beginning
会收到生产者发送的消息，在生产者命令行窗口继续输入消息，这里会收到消息。
