About Pulsar
Apache Pulsar is an enterprise-class distributed messaging system, originally developed by Yahoo and open source in 2016, is currently under the Apache Incubator Foundation. Plusar Yahoo has been used in a production environment more than three years, primarily serving Mail, Finance, Sports, Flickr, the Gemini Ads platform, Sherpa and Yahoo's KV storage.
Pulsar was able to be called the next generation of message queues, mainly because of the following characteristics:
-
Linear expansion. It can be silky expansion to hundreds of nodes (Kafka expansion take up a lot of system resources required to copy data between nodes, but not completely Plusar)
-
High throughput. Yahoo has stood the test in the production environment, millions of messages per second
-
Low latency. In the massive amount of messages still able to maintain low latency (<5ms)
-
Persistence mechanism. Plusar persistence mechanism built on top of Apache BookKeeper, IO provides isolation before writing and reading
-
Copy locations based. Plusar copy multi-regional / area available as a primary feature support. Users simply configure the available area, the message will be copied to a steady stream of other available areas. When one of the available area of a network partition or hang, plusar will continue after retries.
-
Diversification of deployment. Both can run on bare metal, but also support the current example Docker, some of K8S container schemes and different cloud vendors, while local developers only need to start the entire command line environment.
-
Topic supports a variety of consumption patterns: exclusive, shared, failover
Architecture Overview
From the top view, a Plusar unit composed of several clusters, the clusters within each unit may copy the data before, plusar generally have the following components:
-
Broker: Producer responsible for handling the message sent and distributed to consumers. To handle a variety of tasks through a collaborative global ZK cluster, for example, said location-based replication. And the message is stored BookKeeper, but also need to have a single cluster ZK cluster to store some metadata.
-
BookKeeper Cluster: comprising a plurality of internal bookies, for persistent messages.
-
ZooKeeper集
Broker
在Kafka和RocketMQ中，Broker负责消息数据的存储以及consumer消费位移的存储等，而Plusar中的broker和他们两个有所不同，plusar中的broker是一个无状态的节点，主要负责三件事情：
-
暴露REST接口用于执行管理员的命令以及topic所有者的查询等
-
一个用于节点间通讯的异步的TCP服务器，协议目前采用的是Google之前开源的Protocol Buffer
-
为了支持地域复制，broker会将自己 集群所在的消息发布到其他可用区。
消息会被先发布到BookKeeper中，然后会在Broker本地内存中缓存一份，因此一般来说消息的读取都会从从内存中读取，因此第一条中所说的查找topic所有者就是说，因为BookKeeper中的一个ledger只允许一个writer，因此我们可以调用rest接口获取到某一个topic当前的所有者。
BookKeeper
BookKeeper是一个可横向扩展的、错误容忍的、低延迟的分布式存储服务，BookKeeper中最基本的单位是记录，实际上就一个字节数组，而记录的数组称之为ledger，BK会将记录复制到多个bookies，存储ledger的节点叫做bookies，从而获得更高的可用性和错误容忍性。从设计阶段BK就考虑到了各种故障，Bookies可以宕机、丢数据、脏数据，但是主要整个集群中有足够的Bookies服务的行为就是正确的。
在Pulsar中，每个分区topic是由若干个ledger组成的，而ledger是一个append-only的数据结构，只允许单个writer，ledger中的每条记录会被复制到多个bookies中，一个ledger被关闭后(例如broker宕机了或者达到了一定的大小)就只支持读取，而当ledger中的数据不再需要的时候(例如所有的消费者都已经消费了这个ledger中的消息)就会被删除.
Bookkeeper的主要优势在于它可以保证在出现故障时在ledger的读取一致性。因为ledger只能被同时被一个writer写入，因为没有竞争，BK可以更高效的实现写入。在Broker宕机后重启时，Plusar会启动一个恢复的操作，从ZK中读取最后一个写入的Ledger并读取最后一个已提交的记录，然后所有的消费者也都被保证能看到同样的内容。
我们知道Kafka在0.8版本之前是将消费进度存储到ZK中的，但是ZK本质上基于单个日志的中心服务，简单来讲，ZK的性能不会随着你增加更多的节点而线性增加，会只会相反减少，因为更多的节点意味着需要将日志同步到更多的节点，性能也会随之下降，因此QPS也会受单机性能影响，因此0.8版本之后就将消费进度存储到了Kafka的Topic中，而RocketMQ最初的版本也类似，有几种不同的实现例如ZK、数据库等，目前版本采用的是存储到本机文件系统中，而Plusar采用了和Kafka类似的思想，Plusar将消费进度也存储到了BK的ledger中。
元数据
Plusar中的元数据主要存储到ZK中，例如不同可用区相关的配置会存在全局的ZK中，集群内部的ZK用于存储例如某个topic的数据写入到了那些Ledger、Broker目前的一些埋点数据等等
Plusar核心概念
Topic
发布订阅系统中最核心的概念是topic，简单来说，topic可以理解为一个管道，producer可以往这个管道丢消息，consumer可以从这个管道的另一端读取消息，但是这里可以有多个consumer同时从这个管道读取消息。
Each topic may be divided into a plurality of partitions, with different partitions messages under a topic included it is different. Is added to each message after a partition is assigned a unique offset, in the same partition are ordered message, the client user can so that the entire message for example according to a user ID are hashed modulo sent to the entire partition, so as to avoid the race condition problem to some extent.
By partitioning, the plurality of message processing into different nodes to achieve high throughput. By default, plusar the topic are non-partitioned, but the interface to create or support a certain number of partitions topic by cli.
Plusar automatically balanced Producer and Consumer, but sometimes the client wants to be carried out by default according to their own business rules routing, Plusar default supports the following rules: a single partition, polling, hash, custom (that is, to achieve their own relevant interface to customize the routing rules)
Consumption patterns
Consumer determine exactly how the message is delivered to the consumer, Plusar support several different patterns of consumption: exclusive, shared, failover. Illustrated as follows:
-
Exclusive: a topic can only be consumed by a consumer. This is the default mode Plusar
-
Shared: Share call mode or polling mode, multiple consumers can be connected to a different topic, the message is in turn distributed to the consumer, when a consumer goes down or take the initiative to disconnect, then sent to the consumer has not ack message will be rescheduled distributed to other consumers.
-
Failover: multiple consumers can be connected to the same topic and follow the lexicographical ordering, consumers will start spending the first message, called the master, when the master disconnect, not ack and all the rest of the message queue distribution to another consumer.
Plusar now also supports another Reader interface, support incoming message ID, for example, say Message.Earliest to start spending from the earliest messages.
to sum up
Plusar distributed message queue as a next generation, has very much attractive properties, but also makes up for some shortcomings of other competing products, such as copy area, multi-tenant, scalability, isolation and the like to read and write.