Alex P. :
I have grouped my kafka events:
private static void createImportStream(final StreamsBuilder builder, final Collection<String> topics) {
final KStream<byte[], GraphEvent> stream = builder.stream(topics, Consumed.with(Serdes.ByteArray(), new UserEventThriftSerde()));
stream.filter((key, request) -> {
return Objects.nonNull(request);
}).groupBy(
(key, value) -> Integer.valueOf(value.getSourceType()),
Grouped.with(Serdes.Integer(), new UserEventThriftSerde()))
.aggregate(ArrayList::new, (key, value, aggregatedValue) -> {
aggregatedValue.add(value);
return aggregatedValue;
},
Materialized.with(Serdes.Integer(), new ArrayListSerde<UserEvent>(new UserEventThriftSerde()))
).toStream();
}
how can I add a window but not based on time, but based on number of events.
The reason is that the events will be a bulk dump, a time windowed aggregation would not fit since all events could appear in the same few seconds.
Matthias J. Sax :
Kafka Streams does not support count-based windows out-of-the box because those are non-deterministic and it's hard to handle out-of-order data.
Instead of using the DSL, you can use the Processor API to build a custom operator for your use case though.
Guess you like
Origin http://10.200.1.11:23101/article/api/json?id=378434&siteId=1