bootstrap.servers |
The host/port list is used to initialize the connection to the Kafka cluster. The list format is host1:port1,host2:port2,…. There is no need to add all cluster addresses. Kafka will discover other addresses based on the provided addresses (you can provide a few more in case the provided server is shut down) |
list |
|
|
high |
key.serializer |
The Serializer class that implements the key of the org.apache.kafka.common.serialization.Serializer interface. |
class |
|
|
high |
value.serializer |
Serializer class that implements value of the org.apache.kafka.common.serialization.Serializer interface. |
class |
|
|
high |
acks |
The producer requires the leader to confirm the number of responses received before the request is completed. This configuration controls the durability of sent messages and supports the following configurations: |
string |
1 |
[all, -1, 0, 1] |
high |
|
acks=0 If set to 0, the producer will not wait for any message acknowledgments. The message is immediately added to the socket buffer and considered for delivery. In this case there is no guarantee that the message will be received by the server. And the retry mechanism will not take effect (because the client does not know whether there is a fault or not). The offset returned for each message is always set to -1. |
|
|
|
|
|
acks=1, which means that the leader writes a message to the local log and responds immediately without waiting for all followers to respond. In this case, if the leader fails immediately after responding to the message but before the followers have replicated it, the message will be lost. |
|
|
|
|
|
acks=all This means that the leader will wait for all replicas to synchronize before acknowledging the message. This configuration guarantees that messages will not be lost (as long as there is at least one synchronized replica). This is the strongest availability guarantee. Equivalent to acks=-1. |
|
|
|
|
buffer.memory |
The total number of bytes of memory used by the producer to cache messages waiting to be sent to the server. If messages are sent faster than can be delivered to the server, the producer will block after max.block.ms and throw an exception. |
long |
33554432 |
[0,…] |
high |
|
This setting should roughly correspond to the total memory the producer will use, but is not a hard constraint since all memory used by the producer is used for buffering. Some additional memory will be used for compression (if compression is enabled) and for keeping requests in progress. |
|
|
|
|
compression.type |
Type of data compression. The default is empty (that is, no compression). Valid values are none, gzip, snappy, or lz4. Compresses the entire batch of data, so the effect of the batch will also affect the compression ratio (more batches means better compression). |
string |
none |
|
high |
retries |
Set a value greater than zero and the client will resend if the transmission fails. Note that this retry function is no different from the client resending after receiving an error. If max.in.flight.requests.per.connection is not set to 1, it is possible to change the order in which messages are sent, because if 2 batches are sent to a partition, and the first one fails and retries, but the second If successful, then the second batch will exceed the first. |
int |
0 |
[0,…,2147483647] |
high |
ssl.key.password |
The password for the private key in the keystore file. |
password |
null |
|
high |
ssl.keystore.location |
The location of the keystore file. Can be used for two-way authentication of clients. |
string |
null |
|
high |
ssl.keystore.password |
The repository password for the keystore file. Only required if ssl.keystore.location is configured. |
password |
null |
|
high |
ssl.truststore.location |
The location of the trust repository |
string |
null |
|
high |
ssl.truststore.password |
Password for trust repository file |
password |
null |
|
high |
batch.size |
When multiple messages are sent to the same partition, the producer attempts to batch messages together to reduce request interaction. This helps improve client and server performance. Default batch size for this configuration in bytes: |
int |
16384 |
[0,…] |
medium |
|
Messages larger than this configured size will not be packed. |
|
|
|
|
|
Requests sent to the broker will contain multiple batches, one for each partition, used to send the data. |
|
|
|
|
|
Smaller batch sizes have the potential to reduce throughput (a batch size of 0 disables batching entirely). A very large batch size may be more memory wasteful. Because we will pre-allocate this resource. |
|
|
|
|
client.id |
当发出请求时传递给服务器的id字符串。这样做的目的是允许服务器请求记录记录这个【逻辑应用名】,这样能够追踪请求的源,而不仅仅只是ip/prot。 |
string |
“” |
|
medium |
connections.max.idle.ms |
多少毫秒之后关闭闲置的连接。 |
long |
540000 |
|
medium |
linger.ms |
生产者组将发送的消息组合成单个批量请求。正常情况下,只有消息到达的速度比发送速度快的情况下才会出现。但是,在某些情况下,即使在适度的负载下,客户端也可能希望减少请求数量。此设置通过添加少量人为延迟来实现。- 也就是说,不是立即发出一个消息,生产者将等待一个给定的延迟,以便和其他的消息可以组合成一个批次。这类似于Nagle在TCP中的算法。此设置给出批量延迟的上限:一旦我们达到分区的batch.size值的记录,将立即发送,不管这个设置如何,但是,如果比这个小,我们将在指定的“linger”时间内等待更多的消息加入。此设置默认为0(即无延迟)。假设,设置 linger.ms=5,将达到减少发送的请求数量的效果,但对于在没有负载情况,将增加5ms的延迟。 |
long |
0 |
[0,…] |
medium |
max.block.ms |
该配置控制 KafkaProducer.send() 和 KafkaProducer.partitionsFor() 将阻塞多长时间。此外这些方法被阻止,也可能是因为缓冲区已满或元数据不可用。在用户提供的序列化程序或分区器中的锁定不会计入此超时。 |
long |
60000 |
[0,…] |
medium |
max.request.size |
请求的最大大小(以字节为单位)。此设置将限制生产者的单个请求中发送的消息批次数,以避免发送过大的请求。这也是最大消息批量大小的上限。请注意,服务器拥有自己的批量大小,可能与此不同。 |
int |
1048576 |
[0,…] |
medium |
partitioner.class |
实现Partitioner接口的的Partitioner类。 |
class |
org.apache.kafka.clients.producer.internals.DefaultPartitioner |
|
medium |
receive.buffer.bytes |
读取数据时使用的TCP接收缓冲区(SO_RCVBUF)的大小。如果值为-1,则将使用OS默认值。 |
int |
32768 |
[-1,…] |
medium |
request.timeout.ms |
该配置控制客户端等待请求响应的最长时间。如果在超时之前未收到响应,客户端将在必要时重新发送请求,如果重试耗尽,则该请求将失败。 这应该大于replica.lag.time.max.ms,以减少由于不必要的生产者重试引起的消息重复的可能性。 |
int |
30000 |
[0,…] |
medium |
sasl.jaas.config |
JAAS配置文件使用的格式的SASL连接的JAAS登录上下文参数。这里描述JAAS配置文件格式。该值的格式为:‘(=)*;’ |
password |
null |
|
medium |
sasl.kerberos.service.name |
Kafka运行的Kerberos主体名称。可以在Kafka的JAAS配置或Kafka的配置中定义。 |
string |
null |
|
medium |
sasl.mechanism |
SASL机制用于客户端连接。这是安全提供者可用与任何机制。GSSAPI是默认机制。 |
string |
GSSAPI |
|
medium |
security.protocol |
用于与broker通讯的协议。 有效值为:PLAINTEXT,SSL,SASL_PLAINTEXT,SASL_SSL。 |
string |
PLAINTEXT |
|
medium |
send.buffer.bytes |
发送数据时,用于TCP发送缓存(SO_SNDBUF)的大小。如果值为 -1,将默认使用系统的。 |
int |
131072 |
[-1,…] |
medium |
ssl.enabled.protocols |
启用SSL连接的协议列表。 |
list |
TLSv1.2,TLSv1.1,TLSv1 |
|
medium |
ssl.keystore.type |
密钥存储文件的文件格式。对于客户端是可选的。 |
string |
JKS |
|
medium |
ssl.protocol |
最近的JVM中允许的值是TLS,TLSv1.1和TLSv1.2。 较旧的JVM可能支持SSL,SSLv2和SSLv3,但由于已知的安全漏洞,不建议使用SSL。 |
string |
TLS |
|
medium |
ssl.provider |
用于SSL连接的安全提供程序的名称。默认值是JVM的默认安全提供程序。 |
string |
null |
|
medium |
ssl.truststore.type |
信任仓库文件的文件格式。 |
string |
JKS |
|
medium |
enable.idempotence |
当设置为‘true’,生产者将确保每个消息正好一次复制写入到stream。如果‘false’,由于broker故障,生产者重试。即,可以在流中写入重试的消息。此设置默认是‘false’。请注意,启用幂等式需要将max.in.flight.requests.per.connection设置为1,重试次数不能为零。另外acks必须设置为“全部”。如果这些值保持默认值,我们将覆盖默认值。 如果这些值设置为与幂等生成器不兼容的值,则将抛出一个ConfigException异常。如果这些值设置为与幂等生成器不兼容的值,则将抛出一个ConfigException异常。 |
boolean |
FALSE |
|
low |
interceptor.classes |
实现ProducerInterceptor接口,你可以在生产者发布到Kafka群集之前拦截(也可变更)生产者收到的消息。默认情况下没有拦截器。 |
list |
null |
|
low |
max.in.flight.requests.per.connection |
阻塞之前,客户端单个连接上发送的未应答请求的最大数量。注意,如果此设置设置大于1且发送失败,则会由于重试(如果启用了重试)会导致消息重新排序的风险。 |
int |
5 |
[1,…] |
low |
metadata.max.age.ms |
在一段时间段之后(以毫秒为单位),强制更新元数据,即使我们没有看到任何分区leader的变化,也会主动去发现新的broker或分区。 |
long |
300000 |
[0,…] |
low |
metric.reporters |
用作metrics reporters(指标记录员)的类的列表。实现MetricReporter接口,将受到新增加的度量标准创建类插入的通知。 JmxReporter始终包含在注册JMX统计信息中。 |
list |
“” |
|
low |
metrics.num.samples |
维护用于计算度量的样例数量。 |
int |
2 |
[1,…] |
low |
metrics.recording.level |
指标的最高记录级别。 |
string |
INFO |
[INFO, DEBUG] |
low |
metrics.sample.window.ms |
度量样例计算上 |
long |
30000 |
[0,…] |
low |
reconnect.backoff.max.ms |
重新连接到重复无法连接的代理程序时等待的最大时间(毫秒)。 如果提供,每个主机的回退将会连续增加,直到达到最大值。 计算后退增加后,增加20%的随机抖动以避免连接风暴。 |
long |
1000 |
[0,…] |
low |
reconnect.backoff.ms |
尝试重新连接到给定主机之前等待的基本时间量。这避免了在循环中高频率的重复连接到主机。这种回退适应于客户端对broker的所有连接尝试。 |
long |
50 |
[0,…] |
low |
retry.backoff.ms |
尝试重试指定topic分区的失败请求之前等待的时间。这样可以避免在某些故障情况下高频次的重复发送请求。 |
long |
100 |
[0,…] |
low |
sasl.kerberos.kinit.cmd |
Kerberos kinit 命令路径。 |
string |
/usr/bin/kinit |
|
low |
sasl.kerberos.min.time.before.relogin |
Login线程刷新尝试之间的休眠时间。 |
long |
60000 |
|
low |
sasl.kerberos.ticket.renew.jitter |
添加更新时间的随机抖动百分比。 |
double |
0.05 |
|
low |
sasl.kerberos.ticket.renew.window.factor |
登录线程将睡眠,直到从上次刷新ticket到期时间的指定窗口因子为止,此时将尝试续订ticket。 |
double |
0.8 |
|
low |
ssl.cipher.suites |
密码套件列表。这是使用TLS或SSL网络协议来协商用于网络连接的安全设置的认证,加密,MAC和密钥交换算法的命名组合。默认情况下,支持所有可用的密码套件。 |
list |
null |
|
low |
ssl.endpoint.identification.algorithm |
使用服务器证书验证服务器主机名的端点识别算法。 |
string |
null |
|
low |
ssl.keymanager.algorithm |
用于SSL连接的密钥管理因子算法。默认值是为Java虚拟机配置的密钥管理器工厂算法。 |
string |
SunX509 |
|
low |
ssl.secure.random.implementation |
用于SSL加密操作的SecureRandom PRNG实现。 |
string |
null |
|
low |
ssl.trustmanager.algorithm |
用于SSL连接的信任管理因子算法。默认值是JAVA虚拟机配置的信任管理工厂算法。 |
string |
PKIX |
|
low |
transaction.timeout.ms |
生产者在主动中止正在进行的交易之前,交易协调器等待事务状态更新的最大时间(以ms为单位)。如果此值大于broker中的max.transaction.timeout.ms设置,则请求将失败,并报“InvalidTransactionTimeout”错误。 |
int |
60000 |
|
low |
transactional.id |
用于事务传递的TransactionalId。这样可以跨多个生产者会话的可靠性语义,因为它允许客户端保证在开始任何新事务之前使用相同的TransactionalId的事务已经完成。如果没有提供TransactionalId,则生产者被限制为幂等传递。请注意,如果配置了TransactionalId,则必须启用enable.idempotence。 默认值为空,这意味着无法使用事务。 |
string |
null |
non-empty string |
low |