1. interceptor principle
Producer interceptor (Interceptor) is introduced Kafka 0.10 version, the main control logic for implementing customized end clients.
For the producer concerned, interceptor allows the user to have the opportunity to send a message before callback logic former producer of news and do some customization requirements, such as modifying the message and so on. Meanwhile, Producer interceptor allow users to specify a plurality of sequentially applied to the same message interception to form a chain (interceptor chain). The interface is implemented Intercetpor org.apache.kafka.clients.producer.ProducerInterceptor, its definition method comprising:
（1）configure(configs)
When the call to get configuration information and initialization data.
（2）onSend(ProducerRecord)：
The method encapsulated KafkaProducer.send method, i.e., it runs in the main user thread. Producer to ensure that messages are serialized and partitioned computing before calling this method. Users can do anything in the message this method, but the best guarantee not to modify the message belongs topic and partitions, otherwise it will affect the calculation of the target partition
（3）onAcknowledgement(RecordMetadata, Exception)：
This method is called when a response or message failed to send a message , and usually are in before producer callback logic trigger. onAcknowledgement run producer in the IO thread, so do not put heavy logic in this method, otherwise it will slow down the producer of the message transmitting efficiency
（4）close：
Close interceptor, is mainly used to perform some cleanup resources
As described above, Interceptor may be executed in multiple threads, so when the user needs a specific implementation to ensure the security thread itself. Also if a plurality of designated interceptor, the producer calls them in a specified order , and each capture only interceptor that may be thrown to the error log rather than in the transfer direction. We should pay special attention during use.
2. interceptor example
(1) increasing the time stamp interceptor
Import a java.util.Map;
Import org.apache.kafka.clients.producer.ProducerInterceptor;
Import org.apache.kafka.clients.producer.ProducerRecord;
Import org.apache.kafka.clients. producer.RecordMetadata;
public class TimeInterceptor the implements ProducerInterceptor <String, String> {
@Override
public void Configure (the Map <String,?> configs) {
}
@Override
public ProducerRecord <String, String> onSend (ProducerRecord <String, String> Record) {
// Create a new record, the most front portion of the writing time stamp message body
return new new ProducerRecord (record.topic (), record.partition (), record.timestamp (), record.key (),
System.currentTimeMillis () + "," + record.value () toString ());.
}
@Override
public void onAcknowledgement (RecordMetadata Metadata, Exception Exception) {
}
@Override
public void Close () {
}
}
(2) Statistical successfully send messages and sending failure message number, and prints the two counter upon producer closed
import java. util.Map;
Import org.apache.kafka.clients.producer.ProducerInterceptor;
Import org.apache.kafka.clients.producer.ProducerRecord;
Import org.apache.kafka.clients.producer.RecordMetadata;
public class CounterInterceptor the implements ProducerInterceptor <String , String> {
Private errorCounter int = 0;
Private successCounter int = 0;
@Override
public void Configure (? the Map <String,> configs) {
}
@Override
ProducerRecord public <String, String> onSend (ProducerRecord <String, String> Record) {
return Record;
}
@Override
public void onAcknowledgement (RecordMetadata Metadata, Exception Exception) {
// count the number of successful and failed
if (exception == null) {
successCounter ++;
} the else {
errorCounter ++;
}
}
@Override
public void Close () {
// save the result
System.out.println ( "Successful Sent:" + successCounter);
System.out.println ( "Sent the Failed:" + errorCounter );
}
}
(. 3) Producer main
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
public class InterceptorProducer {
public static void main(String[] args) throws Exception {
// 1 设置配置信息
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
// 2 构建拦截链
List<String> interceptors = new ArrayList<>();
interceptors.add("com.atguigu.kafka.interceptor.TimeInterceptor"); interceptors.add("com.atguigu.kafka.interceptor.CounterInterceptor");
props.put(ProducerConfig.INTERCEPTOR_CLASSES_CONFIG, interceptors);
String topic = "first";
Producer<String, String> producer = new KafkaProducer<>(props);
// 3 发送消息
for (int i = 0; i < 10; i++) {
ProducerRecord<String, String> record = new ProducerRecord<>(topic, "message" + i);
producer.send (Record);
}
// must be closed. 4 producer, this method will be called close the interceptor
producer.close ();
}
}
3) Testing
( 1) consumers start on kafka, then run the client java program.
> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic first 1501904047034,message0 1501904047225,message1 1501904047230,message2 1501904047234,message3 1501904047236,message4 1501904047240,message5 1501904047243,message6 1501904047246,message7 1501904047249,message8 1501904047252,message9
( 2) Observation internet java console output data was as follows:
Successful sent: 10
Failed sent: 0