ACKs (default is 1)
Before the message is considered to be "Submitted", producer take several responses leader confirmation request. This parameter is used to control persistent messages currently provides three values:
acks = 0: indicates producer request returns immediately, without waiting for any confirmation of the leader. This program has the highest throughput, but does not guarantee that the message really sent successfully.
acks = -1: leader represents the partition must wait message is successfully written to all of the ISR replica (synchronization of copies) of the request that was successful. This approach provides the highest message persistence to ensure that, in theory, but throughput is the worst.
acks = 1: represents a leader must respond to the request message and writes the request to the local log is considered to succeed. If at this time leader hangs up after answering requests, messages will be lost. The program offers a good compromise between durability and guarantee throughput.
buffer.memory (default value 33554432)
This parameter specifies the size of the buffer for buffering the message producer end, in bytes, the default value is: 33554432, total 32M. kafka architecture uses a message sent asynchronously, will first create a memory buffer for saving the message to be sent when the producer starts and is responsible for reading the message from the buffer consists of a dedicated thread Send true. Continuing the process of sending a message, when the buffer is filled, producer immediately enter the blocking state until free memory is released, this time can not exceed the value max.blocks.ms set more than once, producer will throw an exception TimeoutException because the Producer is thread-safe, if the report has been TimeoutException, the need to consider raising buffer.memory. Users in the use of multiple threads share kafka producer, it is easy to buffer.memory played.
max.block.ms (default value 60000)
KafkaProducer.send () and KafkaProducer.partitionsFor () method of maximum blocking time, when the buffer is full, or a cluster of Metadata is unavailable, these two methods will be blocked.
batch.size (default value 16384)
Producer batch are transmitted in accordance with, so the size of the batch is critical to select producer performance. A plurality of messages destined for the same producer will partition into a batch in a package, when the batch is full, the message will be sent producer. But not necessarily wait until full, and this is another argument linger.ms related. The default value is 16384 batch.size, total 16K. If the target topic producer transmitted 10 partitions, the need 16K * 10 = 160K of memory to cache the data, this value can not be greater than the specified value buffer.memory.
linger.ms (default is 0)
Producer is sent in accordance with the batch, but depends linger.ms value, default is 0, which indicates not stay. In this case, may have batch does not contain enough produce request was sent out, resulting in a large number of small batch, the network IO brings great pressure. If the speed of data generated during the time T can reach values batch.size size, linger.ms set should not be smaller than T, otherwise batch.size would be meaningless.
compression.type (default is none)
Kafka end is compressed, the open end of Producer compression, and then the data is sent in compressed Broker, Broker end is stored compressed, Consumer Broker got me from the end of the data is compressed. This not only reduces bandwidth but also reduces the Broker end disk usage. Currently supports none (no compression), gzip, snappy and lz4. We recommend using gzip or lz4.
retries (default is 2147483647)
Producer set number of retries. Retry, producer messages will fail due to transient reasons before re-sending. The reason instantaneous failure may include: metadata information failure, low number of copies, overtime, shift out of bounds or unknown partitions. If set retries> 0, then the producer will try these cases and try again. producer as well as parameters: max.in.flight.requests.per.connection. If the parameter is greater than 1, then it may cause disposed retries sending the message out of order. Version 0.11.1.0 of kafka has support "to a precise semantics" and therefore will not cause retry message repeatedly sent the message.
retry.backoff.ms (default is 100)
Two retry time interval, depending on the network server and the case where the adjustment, the adjustment is not necessary under normal circumstances.
max.in.flight.requests.per.connection (default is 5)
Producer IO thread can send the maximum number of outstanding requests on a single response Socket Connection. That is, the number of requests the client to the server network allows a maximum of. Increasing this value should be increased IO throughput thread, so as to enhance the overall performance of producer. But just like to say, if before. Open the retry mechanism, set the parameter larger than 1 it may cause out of order messages. The default value of 5 is a good starting point, if we find bottlenecks producer in the IO thread, as well as individual broker end load is not high, you can try to appropriately increase the value of this parameter is too large will result in increased overall burden producer of memory, while It may also cause unnecessary lock contention but will reduce the TPS.
max.request.size (default is 1048576)
Producer single request sent to a maximum value of Borker. Sender ProducerBatch thread belonging to a plurality of packaged as a Broker ClientRequest, a plurality ProducerBatch size can not exceed the set value max.request.size. message.max.bytes max.request.size set value should not be set larger than the end of the Broker.
enable.idempotence (the default is false)
Whether idempotent. If set to true, indicates producer will ensure that each message happens to have a copy; If set to false, it indicates that the transmission data to the producer because of failure retry broker to make, it may be written into the multisection retry message data stream. Enable.idempotence If true, the required value of the configuration item max.in.flight.requests.per.connection must be less than or equal to 5; CI retries value must be greater than 0; acks CI must be set to All . If these values are not explicitly set by the user, the system will automatically select the appropriate values. If the value is not appropriate, then throws ConfigException exception.
request.timeout.ms (default value 30000)
After sending a request to Producer Broker, the maximum time to wait for a response.
