Kafka monitoring commonly used by old drivers-eagle

Kafka monitoring commonly used by old drivers-eagle

Waves waves talk big data
in front of the article when it comes to the management tool called kafka kafka manager of this tool management kafka really strong, but there is no security certification, can easily create, delete, modify topic, and alarm systems, flow fluctuations You did not do well. So, here at the top of the wave, I would like to recommend a Kafka alarm monitoring and management tool, kafka-eagle.

Kafka-eagle mainly has a few points that we pay attention to but kafkamanager does not exist, it is worth mentioning:

• Flow, you can view the flow fluctuation graph for the last seven days at most

• lag size email alert

• Can be analyzed with kafkasql

1. Download and unzip

Download the compressed package directly

http://download.smartloli.org/

Source address

https://github.com/smartloli/kafka-eagle

It is recommended to download the compressed package directly instead of compiling with source code, which is prone to errors during the compilation process. Those who are interested can download the source code and read it.

Unzip

tar -zxf ~/Downloads/kafka-eagle-bin-1.2.4.tar.gz -C ./

Add the name to the file as kafka-eagle.

2. Configuration

Configure environment variables.

vi /etc/profile

export KE_HOME=/opt/hadoop/kafka-eagle
export PATH=$PATH:$KE_HOME/bin

3. Configure system-config.properties

Langjian runs multiple versions of kafka here to test multiple versions and multiple clusters. For specific configuration methods, please refer to the following configuration.

######################################
# multi zookeeper&kafka cluster list
######################################
kafka.eagle.zk.cluster.alias=cluster1,cluster2
cluster1.zk.list=localhost:2181/kafka010
cluster2.zk.list=localhost:2181/kafka082
######################################
# zk client thread limit
######################################
kafka.zk.limit.size=25

######################################
# kafka eagle webui port
######################################
kafka.eagle.webui.port=8048

######################################
# kafka offset storage
######################################
cluster1.kafka.eagle.offset.storage=kafka
cluster2.kafka.eagle.offset.storage=zookeeper

######################################
# enable kafka metrics
######################################
kafka.eagle.metrics.charts=false

######################################
# alarm email configure
######################################
kafka.eagle.mail.enable=false
kafka.eagle.mail.sa=alert_sa
[email protected]
kafka.eagle.mail.password=mqslimczkdqabbbh
kafka.eagle.mail.server.host=smtp.163.com
kafka.eagle.mail.server.port=25

######################################
# delete kafka topic token
######################################
kafka.eagle.topic.token=keadmin

######################################
# kafka sasl authenticate
######################################
kafka.eagle.sasl.enable=false
kafka.eagle.sasl.protocol=SASL_PLAINTEXT
kafka.eagle.sasl.mechanism=PLAIN

######################################
# kafka jdbc driver address
######################################
#kafka.eagle.driver=com.mysql.jdbc.Driver
#kafka.eagle.url=jdbc:mysql://127.0.0.1:3306/ke?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
#kafka.eagle.username=root
#kafka.eagle.password=mt2018@#

kafka.eagle.driver=org.sqlite.JDBC
kafka.eagle.url=jdbc:sqlite:/opt/hadoop/kafka-eagle/db/ke.db
kafka.eagle.username=root
kafka.eagle.password=smartloli

Start and run

Before starting the operation, do the following step,

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home

Otherwise, an error will be reported as follows:

Officially launched kafka-eagle

bin/ke.sh start

After the startup is successful, there will be the following log, pay attention to the red box.

Open the browser and enter the url indicated in the red box in the screenshot above

Click to log in and enter

For specific use, please refer to the manual

https://ke.smartloli.org/1.Overview/1.Overview.html