Kakfa manager 使用教程

作者：石文
时间：2018-11-15

---

Kafka manager 版本： kafka-manager 1.3.3.14

配置文件解读：

路径：kafka-manager-1.3.3.14/conf

application.conf

play.crypto.secret="^<csmm5Fx4d=r2HEX8pelM3iBkFVv?k[mc;IZE<_Qoq8EkX_/7@Zt6dP05Pzea3U"
play.crypto.secret=${?APPLICATION_SECRET}

# The application languages
# ~~~~~
play.i18n.langs=["en"]

play.http.requestHandler = "play.http.DefaultHttpRequestHandler"
play.http.context = "/"
play.application.loader=loader.KafkaManagerLoader
# ZK地址，这个指kafka manager ZK 地址
kafka-manager.zkhosts="10.237.49.121:2181,10.237.49.148:2181,10.237.49.23:2181,10.237.54.239:2181,10.237.54.244:2181"
pinned-dispatcher.type="PinnedDispatcher"
pinned-dispatcher.executor="thread-pool-executor"
#集群管理功能的开启
#KMClusterManagerFeature 启动增添删集群
#KMTopicManagerFeature 集群topic相关操作
#KMPreferredReplicaElectionFeature topic 的选举功能
#KMReassignPartitionsFeature topic 分片设置修改的功能
application.features=["KMClusterManagerFeature","KMTopicManagerFeature","KMPreferredReplicaElectionFeature","KMReassignPartitionsFeature"]

akka {
  loggers = ["akka.event.slf4j.Slf4jLogger"]
  loglevel = "INFO"
}


basicAuthentication.enabled=false
basicAuthentication.enabled=${?KAFKA_MANAGER_AUTH_ENABLED}
basicAuthentication.username="admin"
basicAuthentication.username=${?KAFKA_MANAGER_USERNAME}
basicAuthentication.password="password"
basicAuthentication.password=${?KAFKA_MANAGER_PASSWORD}
basicAuthentication.realm="Kafka-Manager"
basicAuthentication.excluded=["/api/health"] # ping the health of your instance without authentification

kafka-manager.consumer.properties.file="/home/hadoop/kafka-manager-1.3.3.14/conf/consumer.properties"

logback.xml

日志相关设置

<conversionRule conversionWord="coloredLevel" converterClass="play.api.Logger$ColoredLevel" />

<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${application.home}/logs/application.log</file>
    <encoder>
       <pattern>%date - [%level] - from %logger in %thread %n%message%n%xException%n</pattern>
    </encoder>

日志路径：

${application.home}/logs/application.log

常见故障处理：

1. kafka manager 打开速度很慢

处理过程：

a.查看kafka manager 日志发现出现如下 报错：

大量的ZK连接超时

2018-11-15 11:20:06,773 - [ERROR] - from org.apache.curator.ConnectionState

in Curator-Framework-0 
Connection timed out for connection string (10.160.99.3:2181,10.160.110.3:2181,10.160.97.2:2181/hb-security-kafka10) and timeout (15000) / elapsed (47950)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
	at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:197) [org.apache.curator.curator-client-2.10.0.jar:na]
	at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88) [org.apache.curator.curator-client-2.10.0.jar:na]
	at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:116) [org.apache.curator.curator-client-2.10.0.jar:na]
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) [org.apache.curator.curator-framework-2.10.0.jar:na]
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) [org.apache.curator.curator-framework-2.10.0.jar:na]
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) [org.apache.curator.curator-framework-2.10.0.jar:na]
	at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) [org.apache.curator.curator-framework-2.10.0.jar:na]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_172]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_172]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_172]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_172]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_172]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_172]

查看ZK的情况，发现ZK集群正常

b. 重启kafka manager

关闭进程：kill -9 (其他方式无法关闭)

重启应用发现无法启动。查看原因为：

RUNNING_PID 文件存在（这个文件记录了 程序运行的进程PID）。只要是这个文件存在，进程就认为是启动的。

删除此文件后。启动