Kafka write to introduce several important concepts (brief Bowen Kafka before you can reference):
Broker：消息中间件处理结点，一个Kafka节点就是一个broker，多个broker可以组成一个Kafka集群；
Topic：一类消息，例如page view日志、click日志等都可以以topic的形式存在，Kafka集群能够同时负责多个topic的分发；
Partition：topic物理上的分组，一个topic可以分为多个partition，每个partition是一个有序的队；
Segment：每个partition又由多个segment file组成；
offset：每个partition都由一系列有序的、不可变的消息组成，这些消息被连续的追加到partition中。partition中的每个消息都有一个连续的序列号叫做offset，用于partition唯一标识一条消息；
message：这个算是kafka文件中最小的存储单位，即是 a commit log。
topic：创建topic名称
partition：分区编号
offset：表示该partition已经消费了多少message
logsize：表示该paritition生产了多少的message
lag：表示有多少条message未被消费
owner：表示消费者
create：表示该partition创建时间
last seen：表示消费状态刷新最新时间
参考链接：
能查看到kafka中生产了，消费了，还剩下多少message中我们用的是kafkaoffsetmonitor这个监控插件Kafka configuration and monitoring tools KafkaOffsetMonitor use: https://www.cnblogs.com/dadonggg/p/8242682.html
topics是什么？partition是什么？kafka topics is the basic unit of data storage in
the write data, specify which topic you want to read data is written, read from the specified topic which
we can understand such a simple
topic is similar to a table in the database, you can create any number of topic each topic name is the only
For example:
Program A produces a type of message, and the message in such kafka group, which is generated by the program A is called a news topic
application B needs to subscribe to this message, the topic can become consumer
Each topic will have inside one or more partitions (partition)
data written to you, he is actually written into every topic in which a partition, and the current data is written to the paritition ordered in.
Within each partition will maintain a growing ID, every time you write a new data when the ID will increase, this id will be called offset this paritition, each partition is written in the message It will correspond to an offset.
Different partition will correspond to their own offset we can use the offset to determine the order of the current internal paritition, but we can not compare two different order from the partition, which is not meaningful
data partition is ordered, different data partition between the order of the data is lost. If there are multiple topic partition, when consumption data can not guarantee the order of the data. In scene requires strict order to ensure that the message of consumption, the number of partition needs to be set to 1.
//
Each topic is divided into a plurality of partition (zone)
Each topic is divided into a plurality of partition (zone), in addition kafka you can configure the number of partitions to be backed up (Replicas)
基于replicated方案,那么就意味着需要对多个备份进行调度;每个partition都有一个server为"leader";leader负责所有的读写操作,如果leader失效,那么将会有其他follower来接管(成为新的leader);follower只是单调的和leader跟进,同步消息即可..由此可见作为leader的server承载了全部的请求压力,因此从集群的整体考虑,有多少个partitions就意味着有多少个"leader",kafka会将"leader"均衡的分散在每个实例上,来确保整体的性能稳定.
其中partition leader的位置(host:port)注册在zookeeper中
当你讲数据写入kafka中，改数据默认情况下会在kafka中保存2个星期。当然，我们可以去配置的。如果是默认的2个星期，超过2个星期的话，kafka里面的数据就会被无效化。这个时候，该数据对应的offset就没有其他的意义了。
从kafka读取数据后 数据会自动删除吗
不会，kafka中数据的删除跟有没有消费者消费完全无关。数据的删除，只跟kafka broker上面上面的这两个配置有关：
log.retention.hours=48 #数据最多保存48小时
log.retention.bytes=1073741824 #数据最多1G
Note: the data written to kafka, is not changed. He is a familiar immutability. In other words, you have no way to change the data kafka has been written to.
If you want to update a data memssage, you can only re-written memssage to kafka, and the new message will have a new offset, message in order to distinguish it from previously written.
For each data written to kafka in, they will randomly written to the current topic in one partition, with one exception, you provide a key to the current data, this time, you can go with the current key current data should be passed to the control which the partition.
Each topic in this can be determined by multiple parititions of your