kafka Web UI visual monitoring structures (kafka eagle, attachment of parameters JVM)

0, eagle github address attached: https://github.com/smartloli/kafka-eagle , here is kafka-eagle-web-1.3.7

1、修改KAFKA_HOME/bin/kafka-server-start.sh

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:MetaspaceSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=4 -XX:ConcGCThreads=2 -XX:InitiatingHeapOccupancyPercent=70"
    export JMX_PORT="9999"
    #export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

JVM parameters explained:
see some small partners to use on their machines is java8, while using the above parameter settings or PermSize (permanent space on behalf of the setting), in fact, from the beginning java8 has no permanent generation, transfer from the element space (based on memory, if not set MaxMetaspaceSize theoretically limited only by the physical machine resource constraints). Used on their own collector is java8, so I used XX: MetaspaceSize = 128m. -Xms2G -Xmx2G upcoming initial JVM heap space is set to 2G, these two parameters specifies the initial application heap space and maximum heap space applications, consistent when these two parameters can improve the performance of virtual machine program to a certain extent, thereby improve data throughput efficiency of kafka. And on GC Collector, here opt G1 (Garbage First), a current mainstream Hotspot Vm at an excellent collector; high hopes to replace CMS Hotspot team. I want to say here are interested in small partners not to know the meaning of the last three parameters.

2, unzip to bin / ke.sh grant execute permission (default is none) convenient and direct running, the way to add the eagle path to the / etc / profile to facilitate direct start

chmod u+x ke.sh

/ Etc / profile (remember to change it after the source)

export KE_HOME=/usr/SFT/kafka-eagle-web-1.3.7
export PATH=$PATH:$KE_HOME/bin

3, modify the configuration file /conf/system-config.properties (note configured according to the actual situation on their machines)

# multi zookeeper&kafka cluster list
kafka.eagle.zk.cluster.alias=cluster1
cluster1.zk.list=chdp11:2181,chdp11:2181,chdp11:2181

# kafka eagle webui port
kafka.eagle.webui.port=8048

# kafka offset storage
cluster1.kafka.eagle.offset.storage=kafka
#cluster2.kafka.eagle.offset.storage=zookeeper

# enable kafka metrics
kafka.eagle.metrics.charts=true
kafka.eagle.sql.fix.error=false

# kafka jdbc driver address
kafka.eagle.driver=com.mysql.jdbc.Driver
kafka.eagle.url=jdbc:mysql://chdp11:3306/ke?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
kafka.eagle.username=root
kafka.eagle.password=root

So kafka eagle basic configuration is complete, and will be distributed to more than a cluster of modified files. Then turn starts zk, kafka cluster, and finally turned KE

ke.sh start

Follow the prompts to enter the site to open the corresponding account password, account password here very friendly color look less clear (see inside the red box); initial account is: admin, password: 123456

Home

data monitoring under the topic is still very good, you can clearly see to each topic, partition data load. This problem can detect a data transfer process and adjust resolved.