windows install and run kafka

One, install JAVA JDK

1. Download the installation package
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Note: Download the corresponding installation package according to the 32/64-bit operating system

2. Add system variables: JAVA_HOME=C:\Program Files\Java\jdk1.8.0_77

Two, configure zookeeper

Zookeeper:
Official website: https://zookeeper.apache.org/releases.html#download

Baidu network disk link: https://pan.baidu.com/s/1R8N0Y8c5kjsG_NZNMzJb4A
Extraction code: ncln

1. After decompressing the file, we first enter the conf directory

2. You can see that there is a zoo.cfg file, if not, you can copy "zoo_sample.cfg" and rename it to "zoo.cfg"

3. After opening, modify the path of dataDir and change it to the path of the unzipped file, such as: E:\SOFT\zookeeper-3.4.13 (note that it must be \\)

4. Edit system variables and add zookeeper's bin directory: E:\SOFT\zookeeper-3.4.13\bin

5. Modify the default Zookeeper port in the zoo.cfg file (the default port is 2181, but the kafka configuration file should be modified after the change here)

6. Open a new cmd, enter "zkServer", and run Zookeeper

7. The command line prompt is as follows: It means that the local Zookeeper started successfully.
   
   
   
   Note: Do not close this window

Three, configure kafka

Kafka:
Government network: http://kafka.apache.org/downloads

Baidu network disk link: https://pan.baidu.com/s/1G-4EH8ZQtq0Zt7Z339qDDA
extraction code: ncln

Note to download the binary version

1. Unzip and enter the Kafka directory

2. Enter the config directory to find the file server.properties and open it

3. Find and edit log.dirs=E:\SOFT\kafka_2.11-2.1.1\logs

4. Find and edit zookeeper.connect=localhost:2181 (if modified before, change to the corresponding port)

5. Kafka will run on port 9092 by default and connect to the default port of zookeeper: 2181

6. Enter the Kafka installation directory E:\SOFT\kafka_2.11-2.1.1, press Shift+right key, select the "open powershell window here" option, open the command line, and enter:

.\bin\windows\kafka-server-start.bat .\config\server.properties

或bin\kafka-server-start.sh config\server.properties

Note: Do not close this window, and make sure that the ZooKeeper instance is ready and running before enabling Kafka

Four, test

(Linux directly under the bin directory. Sh, windows need to enter the .bat under bin\winndows)

1. Create a theme:

Enter the Kafka installation directory E:\SOFT\kafka_2.11-2.1.1, press Shift+right key, select the "open powershell window here" option, open the command line, and enter:

.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Note: do not close this window

2. View topic:

.\bin\windows\kafka-topics.bat --list --zookeeper localhost:2181

3. Create a producer:

Enter the Kafka installation directory E:\SOFT\kafka_2.11-2.1.1, press Shift+right key, select the "open powershell window here" option, open the command line, and enter:

.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test

Note: do not close this window

4. Create consumers:

Enter the Kafka installation directory E:\SOFT\kafka_2.11-2.1.1, press Shift+right key, select the "open powershell window here" option, open the command line, and enter:

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test --from-beginning

At this point, write a message to the producer window, and the consumer window can also receive the message synchronously

5. Processing of creating consumer operation logs:

• After kafka is started, if you go to check the root directory where kafka is located, or the directory of kafka itself, you will find that a bunch of operation logs have been generated by default (this looks really messy):

• And it will continuously generate operation logs with different timestamps. At first I was at a loss. After some research, I looked at the content of the startup script and found that the configuration in the log4j.properties file is used by default when it is started, and the startup meeting itself will not be seen in zoo.cfg Calling this, I thought there was only one log path:

• Just configure it here, find log4j.properties under config:

• Just change the path so that it can be archived under a folder. The path is defined according to your preferences:

• In addition, how to eliminate the problem of continuously generating logs is that they will be generated continuously at different times of the same day.

• Modify this, or in log4j.properties:

• They are all traces, literally understood as generating a bunch of trace logs, just change them to INFO.

Reference URL of this article:

http://kafka.apache.org/

https://blog.csdn.net/weixin_38004638/article/details/91893910

http://blog.csdn.net/evankaka/article/details/52421314