Distributed publish-subscribe messaging system Kafka

Kafka is a high-throughput distributed publish-subscribe messaging system. It has the following characteristics: Provides message persistence through an O(1) disk data structure, which can keep long for even terabytes of message storage. time stability. High throughput: Even the very common




Kafka is a high-throughput distributed publish-subscribe messaging system. It has the following characteristics:

    It provides message persistence through an O(1) disk data structure, which is suitable for even numbers of TB of message storage can also maintain long-term stable performance.

    High throughput: Even very common hardware kafka can support hundreds of thousands of messages per second.

    Support for partitioning messages through kafka server and consumer machine clusters.

    Support Hadoop parallel data loading.

The purpose of Kafka is to provide a publish-subscribe solution that can handle all action streaming data in a consumer-scale website. Such actions (web browsing, searching, and other user actions) are a key factor in many social functions on the modern web. This data is usually addressed by processing logs and log aggregation due to throughput requirements. This is a viable solution for log data and offline analysis systems like Hadoop, but with the constraints of real-time processing. The purpose of Kafka is to unify online and offline message processing through Hadoop's parallel loading mechanism, and to provide real-time consumption through cluster machines.

Git address:
?
1

http://git-wip-us.apache.org/repos/asf/kafka.git