Introduction to kafka UI tool deployment

Deployment testing of four UI tools: kafka Map, kafka eagle, Know Streaming, and Kafka-UI

1、kafka Map

Multi-cluster management
Cluster status monitoring (number of partitions, number of copies, storage size, offset)
Topic creation, deletion, and expansion (delete requires configuration delete.topic.enable = true)
broker status monitoring
View and delete consumer groups
reset offset
Message query (supports String and json display)
Send a message (supports sending string messages to the specified topic and partition)
Delayed messages (expanded to enable kafka to support 18 levels of delayed messages)

Can be installed directly using docker

docker run -d \
    -p 8080:8080 \
    -v /opt/kafka-map/data:/usr/local/kafka-map/data \
    -e DEFAULT_USERNAME=admin \
    -e DEFAULT_PASSWORD=admin \
    --name kafka-map \
    --restart always dushixiang/kafka-map:latest

Access address: http://ip:8080

Account/password: admin/admin

2、kafka-eagle

Fully functional, an open source product made by a domestic company

Download the installation package of kafka-eagle

https://github.com/smartloli/kafka-eagle-bin/releases

After the download is completed, kafka-eagleit will be decompressed to the specified directory;
```
cd /mydata/kafka/
tar -zxvf kafka-eagle-web-2.0.5-bin.tar.gz
```

/etc/profileAdd environment variables to the file KE\_HOME;

vi /etc/profile
#在profile文件中添加
export KE_HOME=/mydata/kafka/kafka-eagle-web-2.0.5
export PATH=$PATH:$KE_HOME/bin

#使修改后的profile文件生效
source /etc/profile

Install MySQL and add a database ke, kafka-eaglewhich will be used later;

Modify the configuration file $KE\_HOME/conf/system-config.properties, mainly modify the Zookeeper configuration and database configuration, comment out the sqlite configuration, and use MySQL instead;

######################################
# multi zookeeper & kafka cluster list
######################################
kafka.eagle.zk.cluster.alias=cluster1
cluster1.zk.list=localhost:2181
######################################
# kafka eagle webui port
######################################
kafka.eagle.webui.port=8048
######################################
# kafka sqlite jdbc driver address
######################################
# kafka.eagle.driver=org.sqlite.JDBC
# kafka.eagle.url=jdbc:sqlite:/hadoop/kafka-eagle/db/ke.db
# kafka.eagle.username=root
# kafka.eagle.password=www.kafka-eagle.org
######################################
# kafka mysql jdbc driver address
######################################
kafka.eagle.driver=com.mysql.cj.jdbc.Driver
kafka.eagle.url=jdbc:mysql://localhost:3306/ke?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
kafka.eagle.username=root
kafka.eagle.password=root

Use the following command to start kafka-eagle;
```
$KE_HOME/bin/ke.sh start
```
After the command is executed, the following information will be displayed, but it does not mean that the service has been started successfully. You still need to wait for a while;

Here are a few more useful kafka-eaglecommands:

# 停止服务
$KE_HOME/bin/ke.sh stop

# 重启服务
$KE_HOME/bin/ke.sh restart

# 查看服务运行状态
$KE_HOME/bin/ke.sh status

# 查看服务状态
$KE_HOME/bin/ke.sh stats

# 动态查看服务输出日志
tail -f $KE_HOME/logs/ke_console.out

After successful startup, you can access directly and enter your account and password admin:123456.
Access address: http://IP:8048/

3. Didi Know Streaming

There are community edition, enterprise edition and cloud service edition, with full functions

https://gitcode.net/mirrors/didi/LogiKM/-/tree/master

version: "2"
services:
  # *不要调整knowstreaming-manager服务名称，ui中会用到
  knowstreaming-manager:
    image: knowstreaming/knowstreaming-manager:latest
    container_name: knowstreaming-manager
    privileged: true
    restart: always
    depends_on:
      - elasticsearch-single
      - knowstreaming-mysql
    expose:
      - 80
    command:
      - /bin/sh
      - /ks-start.sh
    environment:
      TZ: Asia/Shanghai
      # mysql服务地址
      SERVER_MYSQL_ADDRESS: knowstreaming-mysql:3306
      # mysql数据库名
      SERVER_MYSQL_DB: know_streaming
      # mysql用户名
      SERVER_MYSQL_USER: root
      # mysql用户密码
      SERVER_MYSQL_PASSWORD: admin2022_
      # es服务地址
      SERVER_ES_ADDRESS: elasticsearch-single:9200
      # 服务JVM参数
      JAVA_OPTS: -Xmx1g -Xms1g
      # 对于kafka中ADVERTISED_LISTENERS填写的hostname可以通过该方式完成
#    extra_hosts:
#      - "hostname:x.x.x.x"
      # 服务日志路径
#    volumes:
#      - /ks/manage/log:/logs
  knowstreaming-ui:
    image: knowstreaming/knowstreaming-ui:latest
    container_name: knowstreaming-ui
    restart: always
    ports:
      - '80:80'
    environment:
      TZ: Asia/Shanghai
    depends_on:
      - knowstreaming-manager
#    extra_hosts:
#      - "hostname:x.x.x.x"
  elasticsearch-single:
    image: docker.io/library/elasticsearch:7.6.2
    container_name: elasticsearch-single
    restart: always
    expose:
      - 9200
      - 9300
#    ports:
#      - '9200:9200'
#      - '9300:9300'
    environment:
      TZ: Asia/Shanghai
      # es的JVM参数
      ES_JAVA_OPTS: -Xms512m -Xmx512m
      # 单节点配置，多节点集群参考 https://www.elastic.co/guide/en/elasticsearch/reference/7.6/docker.html#docker-compose-file
      discovery.type: single-node
      # 数据持久化路径
#    volumes:
#      - /ks/es/data:/usr/share/elasticsearch/data

  # es初始化服务，与manager使用同一镜像
  # 首次启动es需初始化模版和索引,后续会自动创建
  knowstreaming-init:
    image: knowstreaming/knowstreaming-manager:latest
    container_name: knowstreaming-init
    depends_on:
      - elasticsearch-single
    command:
      - /bin/bash
      - /es_template_create.sh
    environment:
      TZ: Asia/Shanghai
      # es服务地址
      SERVER_ES_ADDRESS: elasticsearch-single:9200

  knowstreaming-mysql:
    image: knowstreaming/knowstreaming-mysql:latest
    container_name: knowstreaming-mysql
    restart: always
    environment:
      TZ: Asia/Shanghai
      # root 用户密码
      MYSQL_ROOT_PASSWORD: admin2022_
      # 初始化时创建的数据库名称
      MYSQL_DATABASE: know_streaming
      # 通配所有host,可以访问远程
      MYSQL_ROOT_HOST: '%'
    expose:
      - 3306
#    ports:
#      - '3306:3306'
      # 数据持久化路径
#    volumes:
#      - /ks/mysql/data:/data/mysql

Deployment starts

docker-compose -f docker-compose.yml up -d

Access: Address: http://IP/

Account/password: admin/admin

4、Kafka UI

The UI is characterized by being simple and easy to use, without too many bells and whistles, and the interface is user-friendly. Although the function is relatively simple, it is sufficient for development and testing. Recommended

https://gitcode.net/mirrors/provectus/kafka-ui

Can be deployed directly using docker

docker run -p 8080:8080 \
	-e KAFKA_CLUSTERS_0_NAME=local \
	-e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092 \
	-d provectuslabs/kafka-ui:latest

Access address: http://localhost:8080 .