Kafka installation process for Linux

 

Kafka is a high throughput of distributed publish-subscribe messaging system

Installation process for Linux

 

• Front zookeeper installation (high version kafka comes zk, you can direct the next step)

  • Download extract
  • Modify the configuration file
  • Modify environment variables
  • Start zookeeper
  • other

 

• Kafka AnSo

  • Download extract
  • Start Service
  • Creating topic
  • Start producer
  • Start consumer

 

Pre-installed zookeeper

Download extract

　　1. Go to the installation directory, default / usr / local /, you can also customize

# cd /usr/local

　　2. Download the installation package zookeeper, to select the appropriate version of the official website of the installation package:  http://mirror.bit.edu.cn/apache/zookeeper

# wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

　　3. Extract

tar -zxvf zookeeper-3.4.14.tar.gz 

Modify the configuration file

　　1. Go to the conf directory:

# cd zookeeper-3.4.14/conf/

　　2. Copy this file to the zoo_sample.cfg zoo.cfg (must be the file name)

# cp  zoo_sample.cfg  zoo.cfg

　　3. Go zoo.cfg file for editing

# Vim zoo.cfg

　　4. Press i to enter the edit mode, the following modifications:

dataDir=/tmp/zookeeper/data

dataLogDir=/tmp/zookeeper/log

　　

Note: If you want to configure a cluster, please add the server ip in clientPort below. Such as

server.1=192.168.180.132:2888:3888
server.2=192.168.180.133:2888:3888

If the computer memory is relatively small, zookeeper can also be set to a pseudo-cluster. That is, all servers using the same ip, but use a different port.

　　5. Create a directory in the tmp directory.

# mkdir /tmp/zookeeper

# mkdir /tmp/zookeeper/data

# mkdir /tmp/zookeeper/log

 

 　　6. If you are configuring a cluster, you also need to add myid configuration file in front of the path had dataDir

# cd /tmp/zookeeper/data

# touch myid

# vim myid

 

Create a file in the data directory, a file named "myid", edit the "myid" file and enter the corresponding number on the corresponding IP of the machine.
As in 192.168.180.132, "myid" file content is 1. On 192.168.180.133, content is 2.

Modify environment variables

　　1. Set Environment Variables

# export ZOOKEEPER_INSTALL=/usr/local/zookeeper-3.4.14/

# export PATH=$PATH:$ZOOKEEPER_INSTALL/bin

Start zookeeper

　　1. Start zk

# Cd ../bin/

# ./zkServer.sh start

 

 

 

Start successfully effect:

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

 

　　After 2.zookeeper the server side, start, start the client needs the zookeeper:

# ./zkCli.sh

 

 

If the connection of a plurality of different host node, with the following command:

# ./zkCli.sh -server 192.168.180.132:2888

　　3. Check the status:

# ./zkServer.sh status

 

 

Returns the following:

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg

Mode: standalone

other

　　Some of Use 1. zk

　　See Reference: https://www.cnblogs.com/expiator/p/9853378.html

 

Kafka AnSo

Download extract

　　1 into the installation directory, the default / usr / local /, may be defined (zk process and the like)

# cd /usr/local

　　2. Download the installation package kafka, to select the appropriate version of the official website of the installation package:  http://kafka.apache.org/downloads

# wget http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.4.0/kafka_2.12-2.4.0.tgz

　　3. Extract

# Tar -zxvf kafka_2.12-2.4.0.tgz 

Start Service

　　1. Start zk

If the manual installation mode zk, where you can skip 
bin / zookeeper-server-start.sh config / zookeeper.properties

　　2. Start Kafka

# bin/kafka-server-start.sh config/server.properties

Creating topic

　　1. Create topic 

Note here that a need to re-open the window, then the cd to /usr/local/kafka_2.12-2.4.0/

# bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

 

View topic just created by the list command

# bin/kafka-topics.sh -list -zookeeper localhost:2181

Start producer

　　1. Start the producer

# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

Start consumer

　　1. Start consumer

Note here that a need to re-open the window and cd to /usr/local/kafka_2.12-2.4.0/

# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

use

　　1 can then send a message producer unpleasant side view of the end of the message consumer

　　2. Reference Source: https://www.cnblogs.com/expiator/p/9990171.html