The KafkaProducer send method both returns a Future and accepts a Callback.
Is there any fundamental difference between using one mechanism over the other to execute an action upon completion of the sending?
The asynchronous approach
producer.send(record, new Callback(){
@Override
onComplete(RecordMetadata rm, Exception ex){...}
})
gives you better throughput comparing to synchronous
RecordMetadata rm = producer.send(record).get();
since you don't wait for acknowledgements in first case.
Also in asynchronous way ordering is not guaranteed, whereas in synchronous it is - message is sent only after acknowledgement received.
Another difference could be that in synchronous call in case of exception you can stop sending messages straightaway after the exception occurs, whereas in second case some messages will be sent before you discover that something is wrong and perform some actions.
Also note that in asynchronous approach the number of messages which are "in fligh" is controlled by max.in.flight.requests.per.connection parameter.
Apart from synchronous and asynchronous approaches you can use Fire and Forget approach, which is almost the same as synchronous, but without processing the returned metadata - just send the message and hope that it will reach the broker (knowing that most likely it will happen, and producer will retry in case of recoverable errors), but there is a chance that some messages will be lost:
RecordMetadata rm = producer.send(record);
To summarize:
- Fire and Forget - fastest one, but some messages could be lost;
- Synchronous - slowest, use it if you cannot afford to lose messages;
- Asynchronous - something in between.