Broker end parameters:
kafka broker end parameter is also known as static parameters, the so-called static parameters, means you have to set the parameters in the configuration file serve.properties kafka, as long there are changes, you must restart the broker process to make them effective.
Storing information related parameters:
log.dirs: specifies the number of files in the directory path broker to use.
log.dir: represents a single path.
Generally you can just set log.dirs, if using multiple csv format, separated by commas. If able to do the best to mount these directories on different physical disks, this has two advantages:
1, to enhance read and write performance: Compared to single disks, while the multiple physical disk reads and writes data with a higher throughput.
2, it is possible failover: i.e. Failover. Data on the disk will be broken automatically transferred to the other normal disk, and the broker still work.
ZooKeeper related settings:
zookeeper.connect: csv format of a parameter, such as zk1: 2181, zk2: 2181, zk3: 2181
Kafka plurality ZooKeeper cluster use the same set, two sets of such clusters can be specified as kafka1 and kafka2 zk1: 2181, zk2: 2181, zk3: 2181 / kafka1 and zk1: 2181, zk2: 2181, zk3: 2181 / kafka2
Borker connection related configuration
listeners: listeners, who want to specify an external connection to access the specified host name and port open through what protocol kafka service. Several comma-separated triplets, each triplet format is <protocol name, the hostname, port number>,
For example CONTROLLER: // localhost: 9092. Once their own definition of the protocol name, you must also specify listener.security.protocol.map parameter tells the underlying protocol which uses security protocols, such as: listener.security.protocol.map = CONTROLLER: PLAINTEXT expressed CONTROLLER
This custom underlying plaintext without encryption protocol used to transmit data.
advertised.listeners: Advertised meaning representation claimed, announced, that this group of listeners is released for the broker.
host.name/port: These two key parameters set with expired parameters.
All the best host name that broker and client-side applications to fill all of the hostname configuration. broker source code is also used in the host name, if you use the IP address to connect in some places, you may find that the problem can not be connected.
Topic management-related parameters:
auto.create.topic.enable: whether to allow automatic creation topic. Recommended set to false, which does not allow automatic creation Topic.
unclean.leader.election.enable: whether to allow Unclean Leader election. Set to false, determined not to let too many copies of those behind the campaign Leader. The consequences of doing this partition is not available because there is no leader. Is set to true, then kafka allows you to select from a copy of those "slow runner" out of a
When the leader. A consequence of this is data may be lost, because the preservation of these data should not have a copy of the whole, this parameter is false by default in the latest version of kafka, it is recommended set to false.
auto.leader.rebalance.enable: whether to allow regular leader election. Set to true to allow kafka on a regular basis for some topic partition leader re-election, it is not an election leader, but a change leader! For example leaderA has been doing very well, but if this parameter is set to true, then there is likely some time after leaderA would be forced to step down into leaderB. Leader of a high cost of changing all client sends a request to the original A sends a request to be switched to B, and the changing nature leader, no performance gain, it is recommended to set false.
Data retention parameters
log.retention {hours | minutes | ms}:. From the highest priority for ms setting, minutes, and the lowest hours. For example log.eatention.hours = 7 indicates the default save seven days of data.
log.retention.bytes: This is a designated broker for the message to save the total disk capacity size. The default is -1, which means how much data you want to save can be on this broker. This parameter really play the role of a scene is actually built on the cloud kafka cluster: Imagine you have to do a Kafka service on the cloud, each tenant can only use 100GB of disk space, in order to avoid a "malicious" use of excessive disk space, set this parameter is vital.
message.max.bytes: Control broker maximum message size that can be received.