Kakfa Introduction

What is Kafka

Kafka was originally LinkedIn internal internal infrastructure system. It is considered a workflow platform, you can publish on Kafka and subscription streaming data, and save them up for processing. But we use Kafka, the most is to use it as a messaging system, similar to ActiveMQ, RabbitMQ and so on. But Kafka with these traditional messaging system also has many differences, these differences make it different from the messaging system.

Kafka is a distributed system to a cluster (support free expansion) is run. (We always called distributed message queue)
Kafka can be used to store data, the length of time the data is stored is defined by your own (fault-tolerant persistent stored). And further provided that the data stored in Kafka, you can read repeatedly.
Streaming will enhance the level of data processing to a new level.

The traditional message system, only used for passing messages.
Kafka can also be seen as a real-time version of Hadoop (which is also designed one of Kafka's original intention). Hadoop to store and process large amounts of data files on a regular basis, and Kafka and persistent store and process large data streams. Hadoop is mainly used in the data analysis, and Kafka because of its low latency characteristics more suitable used in the core business, when business event occurs, Kafka can make the appropriate response to these events. Meanwhile kafka can also be compared and ETL, because they are good at moving data.

Kafka belongs to the news release system - a subscription model messaging system. The message sender does not send messages directly to the recipients in a message, but the message is first classified (topic), and then publish messages to the message system. Select message recipients subscribe message type (topic), then the system can be received from the message the message is subscribed.

Instructor Profile