How to print KStream to console?

Vicky :

I have created a Kafka Topic and pushed a message to it.

bin/kafka-console-consumer --bootstrap-server abc.xyz.com:9092 --topic myTopic --from-beginning --property print.key=true --property key.separator="-"

prints

key1-customer1

on the command line.

I want to create a Kafka Stream out of this topic and want to print this key1-customer1 on the console.

I wrote the following for it:

final Properties streamsConfiguration = new Properties();

streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "app-id");
streamsConfiguration.put(StreamsConfig.CLIENT_ID_CONFIG, "client-id");
streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "abc.xyz.com:9092");

streamsConfiguration.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
streamsConfiguration.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());

// Records should be flushed every 10 seconds. This is less than the default
// in order to keep this example interactive.
streamsConfiguration.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 10 * 1000);
// For illustrative purposes we disable record caches
streamsConfiguration.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 0);

final StreamsBuilder builder = new StreamsBuilder();
final KStream<String, String> customerStream = builder.stream("myTopic");
customerStream.foreach(new ForeachAction<String, String>() {
    public void apply(String key, String value) {
        System.out.println(key + ": " + value);
    }
});

final KafkaStreams streams = new KafkaStreams(builder.build(), streamsConfiguration);

streams.start();   

Runtime.getRuntime().addShutdownHook(new Thread(streams::close));

This does not fail. However, this does not print anything on the console as this answer suggests.

I am new to Kafka. So any suggestions to make this work would help me a lot.

Jacek Laskowski :

TL;DR Use Printed.

import org.apache.kafka.streams.kstream.Printed
val sysout = Printed
  .toSysOut[String, String]
  .withLabel("customerStream")
customerStream.print(sysout)

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=469136&siteId=1