kafka维护工具使用指南

其他 2018-10-10 00:13:47 阅读次数: 0
版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/xhpscdx/article/details/76670209

一.前言

  1. kafka-manager是Yahoo开源的Kafka管理工具，并在持续提交。是目前最好用kafka监控开源软件。
  2. 由于文档较少，scala开发，对于有些术语的确无法理解，请抱歉。
  3. 本文也介绍运维常用kafka命令。
  4. 文档根据大家反馈，会持续更新，请关注.

二.kafka-manager介绍

  1. 管理多个集群
  2. 方便查看集群状态
  3. 执行preferred replica election
  4. 批量为多个Topic生成并执行Partition分配方案
  5. 创建Topic
  6. 删除Topic（只支持0.8.2及以上版本，同时要求在Broker中将delete.topic.enable设置为true）
  7. 为已有Topic添加Partition
  8. 更新Topic配置
  9. 在Broker JMX Reporter开启的前提下，轮询Broker级别和Topic级别的Metrics
  10. 监控Consumer Group及其消费状态
  11. 支持添加和查看LogKafka

三. topic 监控管理

topic list

broker

  1. topic支持Search属性，重点解释属性意思。
  2. partitions:创建的partition的个数。
  3. brokers：使用到broker个数。

  4. brokers spread： Spread : As you can see in the source code again app/kafka/manager/model/ActorModel.scala#L400, brokers spread is the percentage of brokers in the cluster that has partitions for the given topic.
    Example: 3 brokers share a topic that has 2 partitions, so 66% of the brokers have partitions for this topic
    对于该topic的partition分布情况，只使用到66%的broker。

  5. brokers skew: Skewed : As you can see in the source code app/kafka/manager/model/ActorModel.scala#L380, a broker is skewed if its number of partitions is greater that the average of partitions per broker on the given topic.
    Example: 2 brokers share 4 partitions, if one of them has 3 partitions, it is skewed (3 > 2)

  6. brokers Leader skew :Percentage of brokers having more partitions as leader than the average

  7. replicas: 复制数
  8. under replicated: unknown
  9. Producer Message/Sec :每秒钟生产消息数量
  10. Summed Recent OffSet: 所有partition的offset的总和。

topic view

broker

broker

  1. TopicSummary:类似列表的属性描述。
  2. Metrics：Kafka使用Yammer Metrics来报告服务端和客户端的Metric信息。Yammer Metrics 3.1.0提供6种形式的Metrics收集——Meters，Gauges，Counters，Histograms，Timers，Health Checks。与此同时，Yammer Metrics将Metric的收集与报告（或者说发布）分离，可以根据需要自由组合。目前它支持的Reporter有Console Reporter，JMX Reporter，HTTP Reporter，CSV Reporter，SLF4J Reporter，Ganglia Reporter，Graphite Reporter。因此，Kafka也支持通过以上几种Reporter输出其Metrics信息
  3. 采集producer，consumer的Metrics，发送JMX里。有每秒消息数量，流入流出byte，
  4. partitionInformation：partition复制，leader，及最后偏移量值。
  5. add partition:只能够增大数量，不能够减少。
  6. delete topic:可以删除topic。
  7. 对于Reassign Partitions；Generate Partition Assignments;Manual Partition Assignments; 后面将深入介绍。

consumer group

broker

  1. logSize: partition总的数量。
  2. consumer offset：对该partition当前的已消费的量。
  3. Lag：对该partition剩余的消息量。
  4. Consumer Instance Owner：可以根据ip知道是那些机器线程绑定partition的。

三. Preferred-replica-election 优先复制选举，

  1. 官方文档这样描述这个功能，balancing leadership，重新平衡partitionLeader。意思当一个broker停止或者崩溃，在broker上的leader将转移到partition其它复制的broker上去。当该broker重启后broker上所有的partition仅仅做为复制的follower来使用，不能支持client的读写操作，造成资源浪费。
  2. 在使用0.8的kafka时，遇到过该问题。这样对于复制来说是不平衡的。kafka支持采用下面的commod来重新平衡关系。
  3. kafka 10.0后重启broker后自动触发rebanacing leadership。由增加auto.leader.rebalance.enable=true控制机制，默认为true。

-- 直接平衡所有的topic

 ./bin/kafka-preferred-replica-election.sh --zookeeper 172.16.30.13:2181/kafkanew

-- 也可以指定某个topic的平衡，先创建一个test_two_leader json文件。


 {"version":1,
 "partitions":[{"topic":"dev_infra_andre_test_two","partition":0}]}

 ./bin/kafka-preferred-replica-election.sh --zookeeper 172.16.30.13:2181/kafkanew  -path-to-json-file  test_two_leader


--- 重启broker日志

 [2017-08-03 14:13:51,240] INFO [ReplicaFetcherThread-0-3], Starting  (kafka.server.ReplicaFetcherThread)
[2017-08-03 14:13:51,241] INFO [ReplicaFetcherManager on broker 5] Added fetcher for partitions List([[dev_infra_andre_test_two,0], initOffset 127425 to broker BrokerEndPoint(3,172.16.30.13,9095)] , [[dev_infra_andre_test_two,2], initOffset 130260 to broker BrokerEndPoint(4,172.16.30.13,9096)] , [[ci_online_pay_log,0], initOffset 0 to broker BrokerEndPoint(3,172.16.30.13,9095)] , [[ci_login,0], initOffset 0 to broker BrokerEndPoint(3,172.16.30.13,9095)] ) (kafka.server.ReplicaFetcherManager)
[2017-08-03 14:14:53,197] INFO [ReplicaFetcherManager on broker 5] Removed fetcher for partitions [dev_infra_andre_test_two,2],[ci_login,0],[ci_online_pay_log,0] (kafka.server.ReplicaFetcherManager)
[2017-08-03 14:14:53,248] INFO [ReplicaFetcherThread-0-4], Shutting down (kafka.server.ReplicaFetcherThread)
[2017-08-03 14:14:53,251] INFO [ReplicaFetcherThread-0-4], Stopped  (kafka.server.ReplicaFetcherThread)
[2017-08-03 14:14:53,252] INFO [ReplicaFetcherThread-0-4], Shutdown completed (kafka.server.ReplicaFetcherThread)

  1. 点击kafka-manager header上面的Preferred Replica Election，点击Run Preferred Replica Election 按钮也能触发。

  2. 测试环境的例子，先关闭broker5的auto.leader.rebalance.enable=false

1.重启broker 5前，topic的partition分布情况

broker

2.重启broker 5后，topic的partition分布情况

broker

3 执行上面的Preferred Replica Election，后leader关系被调整。

broker

三. kafka-reassign-partitions操作

  1. 当kafka集成添加新的broker节点时候，新broker会没有任何topic与负载。需要手动迁移一些topic到新broker，即用到该工具。
  2. reassignment tools提供三种模式
  3. –generate: 它是用来非常便捷的生产分配方案json的工具，用来迁移一些topic到指定的broker上，它用来生产reassign的json文件，不会实际的执行。
  4. –execute： 它是用来执行用户自定reassignment计划的，它通过执行一个json配置文件来实现。
  5. –verify： In this mode, the tool verifies the status of the reassignment for all partitions listed during the last –execute. The status can be either of successfully completed, failed or in progress 。它是用来验证上一次执行的commod，可以查看到每个partition的处理状态。

1.reassign 自动迁移topic的数据到新的broker机器

  1. 直接采用官网的例子，移动topic f001，foo2到新的brokers5,6。移动后topic f001，foo2的partition只在broker5,6中。
  2. 官网例子文档 

--- 创建分配json 

> cat topics-to-move.json
{"topics": [{"topic": "foo1"},
            {"topic": "foo2"}],
 "version":1
}



--- 生成一个候选任务，候选任务expand-cluster-reassignment.json。

> bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --topics-to-move-json-file topics-to-move.json --broker-list "5,6" --generate
Current partition replica assignment

{"version":1,
 "partitions":[{"topic":"foo1","partition":2,"replicas":[1,2]},
               {"topic":"foo1","partition":0,"replicas":[3,4]},
               {"topic":"foo2","partition":2,"replicas":[1,2]},
               {"topic":"foo2","partition":0,"replicas":[3,4]},
               {"topic":"foo1","partition":1,"replicas":[2,3]},
               {"topic":"foo2","partition":1,"replicas":[2,3]}]
}

Proposed partition reassignment configuration

{"version":1,
 "partitions":[{"topic":"foo1","partition":2,"replicas":[5,6]},
               {"topic":"foo1","partition":0,"replicas":[5,6]},
               {"topic":"foo2","partition":2,"replicas":[5,6]},
               {"topic":"foo2","partition":0,"replicas":[5,6]},
               {"topic":"foo1","partition":1,"replicas":[5,6]},
               {"topic":"foo2","partition":1,"replicas":[5,6]}]
}



-------  实际执行该分配任务，

> bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file expand-cluster-reassignment.json --execute



----- 校验任务的执行情况：

> bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file expand-cluster-reassignment.json --verify
Status of partition reassignment:
Reassignment of partition [foo1,0] completed successfully
Reassignment of partition [foo1,1] is in progress
Reassignment of partition [foo1,2] is in progress
Reassignment of partition [foo2,0] completed successfully
Reassignment of partition [foo2,1] completed successfully
Reassignment of partition [foo2,2] completed successfully

2. 自定义topic的partition的复制移动指定broker上。

1.例子移动topic foo1的partition 0移到 5,6 broker上，移动topic foo2的partition 1移动到brokers 2,3


> cat custom-reassignment.json
{"version":1,"partitions":[{"topic":"foo1","partition":0,"replicas":[5,6]},{"topic":"foo2","partition":1,"replicas":[2,3]}]}



> bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file custom-reassignment.json --execute
Current partition replica assignment

{"version":1,
 "partitions":[{"topic":"foo1","partition":0,"replicas":[1,2]},
               {"topic":"foo2","partition":1,"replicas":[3,4]}]
}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions
{"version":1,
 "partitions":[{"topic":"foo1","partition":0,"replicas":[5,6]},
               {"topic":"foo2","partition":1,"replicas":[2,3]}]
}

3. 为topic的partition调整复制节点数。

1.例子：为topic foo的partition 0把复制从1个调整为3个节点，以前节点只在broker5上，现在将增加brokers 6,7.

-- 构建json文件

> cat increase-replication-factor.json
{"version":1,
 "partitions":[{"topic":"foo","partition":0,"replicas":[5,6,7]}]}


--- 执行

> bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file increase-replication-factor.json --execute
Current partition replica assignment

{"version":1,
 "partitions":[{"topic":"foo","partition":0,"replicas":[5]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions
{"version":1,
 "partitions":[{"topic":"foo","partition":0,"replicas":[5,6,7]}]}


-- 查询result

> bin/kafka-topics.sh --zookeeper localhost:2181 --topic foo --describe
Topic:foo   PartitionCount:1    ReplicationFactor:3 Configs:
    Topic: foo  Partition: 0    Leader: 5   Replicas: 5,6,7 Isr: 5,6,7

猜你喜欢

转载自blog.csdn.net/xhpscdx/article/details/76670209
今日推荐
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
报告：Django 仍然是 74% 开发者的首选
《2024 年一季度互联网投融资运行情况》研究报告
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
周排行
BPM为企业带来的实际利益
好程序员web前端分享css常用属性缩写
Java文件下载（excel）
css样式的动态添加及显示和隐藏等零碎用法
axios全局配置以及拦截器
使用Logstash来实时同步MySQL和log日志数据到ES
C++获取当前时间（年月日、时分秒、毫秒）
Odoo产品分析 (四) -- 工具板块(11) -- 网站即时聊天(1)
Java环境配置正确，但是java、javac、java -version均返回“不是内部或外部命令，也不是可运行的程序或批处理文件”？
01 官网下载各种CentOS教程（超详细版）
每日归档
更多
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022