Kafka Streams concepts described herein will be briefly related, including the definition of a data stream, the concept of stream processing Kafka Streams.
Stream (stream) is the most important abstract Kafka Streams offer is unlimited it represents a constantly updated data set. A stream is ordered by one, can be reproduced, the data recording support failover immutable (data record) sequence, wherein each data record is defined as a key-value pair. The basic structure shown in FIG Kafka stream.
A stream processor (stream processor) is a process node topology, which represents the processing in step topology. A receiving data stream processor topology upstream from it, the basic stream processing method provided by Kafka Streams, such as a map (), filter (), join () and a polymerization method, process the data, then after treatment one or more output structures sent to a downstream processor. A topology data processor among processors have Source and Sink two special processor stream processor.
(1) Source Processor: a process in the processor does not have any topology upstream processor. The one or more processor Kafka consumption data relating to the topology of a processing from the input stream, the input stream is transmitted to a downstream processor.
(2) Sink processor to: process a topology that the processor does not have any downstream processor. The processor sending any data received from the upstream processor to which the subject specified.
Reproduced in: https: //www.jianshu.com/p/fb1c4c4c3e24