Foreword
Kafka is a developed by linkedln (LinkedIn) company by the Apache Software Foundation open source middleware.
Kafka written in Scala and Java, is mainly responsible for distributed message subscription, it can handle the action flows between producers and consumers, the industry used as a message queue.
Kafka's presentation (according to the official website)
Kafka as a distributed workflow platform, what does that mean?
A workflow platform has the following three capabilities:
- Publish, subscribe to news flow, as if a message queue or enterprise information system
- In a lasting way of saving fault-tolerant data streams
- Process stream occurs.
kafka often used two types of applications.
- Establishing real-time streaming data pipe, the connection data between systems and applications.
Establishing real-time streaming data pipe, or conversion of the corresponding data stream.
Several important concepts
- Kafka is operating in accordance with the cluster approach, each node across multiple data centers within the cluster.
- Kafka stream cluster recording data in a different topic, topic different recording different data streams.
Each record includes a key, a value, a timestamp.
Kafka's four core API:
- Product API allows applications to publish a data stream of information to one or more topic
- Consumer API to allow applications to subscribe to this topic, and or a process of generating data stream information to them.
- Streams API allows an application as a stream processor, the one or more topic use from an input stream, and generates one or more output streams of the topic.
- Connector API allow the establishment and operation of reusable producers or consumers, and connect to the current application and data systems Kafka topic. For example, connect to a relational database connector may capture changes to the data table.
Kafka ensure the communication during the client and the service is simple, highly available, TCP connection (free language). This agreement has been versioned, and backward compatibility.
Specifically, as shown:
