High performance queue Disruptor Tutorial

What is Disruptor

Disruptor is a British foreign exchange company developed high-performance LMAX queue, developed the original intention was to solve the problem of memory latency queue (found in performance tests even with the I / O operation is in the same order of magnitude). Disruptor developed based on single-threaded system can support six million orders per second, in 2010, after QCon speech, access to the industry's attention. In 2011, the enterprise application software specialist dedicated to Martin Fowler wrote a long article describes. The same year also won the award Oracle official Duke. From the data structure point of view, Disruptor support is a producer -> Consumer mode circular queue . Can no lock for parallel consumption under conditions, can also be carried out in accordance with the order of consumption has dependencies between consumers. This article will demonstrate how to achieve some of the classic scenes by Disruptor.

Add dependent

<dependency>
    <groupId>com.lmax</groupId>
    <artifactId>disruptor</artifactId>
    <version>3.4.2</version>
</dependency>
复制代码

Single producer single consumer model

First, create a OrderEventclass that will be placed in the circular queue as the content of the message.

@Data
public class OrderEvent {
    private String id;
}
复制代码

Create a OrderEventProducerclass that will be used as a producer.

public class OrderEventProducer {
    private final RingBuffer<OrderEvent> ringBuffer;
    public OrderEventProducer(RingBuffer<OrderEvent> ringBuffer) {
        this.ringBuffer = ringBuffer;
    }
    public void onData(String orderId) {
        long sequence = ringBuffer.next();
        try {
            OrderEvent orderEvent = ringBuffer.get(sequence);
            orderEvent.setId(orderId);
        } finally {
            ringBuffer.publish(sequence);
        }
    }
}
复制代码

Create a OrderEventHandlerclass and implement EventHandler<T>and WorkHandler<T>interfaces, as consumers.

@Slf4j
public class OrderEventHandler implements EventHandler<OrderEvent>, WorkHandler<OrderEvent> {
    @Override
    public void onEvent(OrderEvent event, long sequence, boolean endOfBatch) {
        log.info("event: {}, sequence: {}, endOfBatch: {}", event, sequence, endOfBatch);
    }
    @Override
    public void onEvent(OrderEvent event) {
        log.info("event: {}", event);
    }
}
复制代码

Once you have created the above three categories, we already have 事件类, 生产者, 消费者these three elements of. Next we demonstrate this process through a series of main method.

@Slf4j
public class DisruptorDemo {
    public static void main(String[] args) throws InterruptedException {
        Disruptor<OrderEvent> disruptor = new Disruptor<>(
                OrderEvent::new,
                1024 * 1024,
                Executors.defaultThreadFactory(),
                ProducerType.SINGLE,
                new YieldingWaitStrategy()
        );
        disruptor.handleEventsWith(new OrderEventHandler());
        disruptor.start();
        RingBuffer<OrderEvent> ringBuffer = disruptor.getRingBuffer();
        OrderEventProducer eventProducer = new OrderEventProducer(ringBuffer);
        eventProducer.onData(UUID.randomUUID().toString());
    }
}
复制代码

Single producer more consumers

If the consumer is more, only you need to call handleEventsWithupon the method of delivery into multiple consumers. The following code passes two consumers.

- disruptor.handleEventsWith(new OrderEventHandler());
+ disruptor.handleEventsWith(new OrderEventHandler(), new OrderEventHandler());
复制代码

Incoming repeat the above two consumers will consume each message, a message if you want to achieve in the case of multiple consumer, the consumer will only be a consumer, you need to call the handleEventsWithWorkerPoolmethod.

- disruptor.handleEventsWith(new OrderEventHandler());
+ disruptor.handleEventsWithWorkerPool(new OrderEventHandler(), new OrderEventHandler());
复制代码

And more producers and more consumers

In the actual development, multiple producers send messages, more consumer handling messages is the norm. This, Disruptor is also supported. And more producers and more consumers code as follows:

@Slf4j
public class DisruptorDemo {
    public static void main(String[] args) throws InterruptedException {
        Disruptor<OrderEvent> disruptor = new Disruptor<>(
                OrderEvent::new,
                1024 * 1024,
                Executors.defaultThreadFactory(),
                // 这里的枚举修改为多生产者
                ProducerType.MULTI,
                new YieldingWaitStrategy()
        );
        disruptor.handleEventsWithWorkerPool(new OrderEventHandler(), new OrderEventHandler());
        disruptor.start();
        RingBuffer<OrderEvent> ringBuffer = disruptor.getRingBuffer();
        OrderEventProducer eventProducer = new OrderEventProducer(ringBuffer);
        // 创建一个线程池，模拟多个生产者
        ExecutorService fixedThreadPool = Executors.newFixedThreadPool(100);
        for (int i = 0; i < 100; i++) {
            fixedThreadPool.execute(() -> eventProducer.onData(UUID.randomUUID().toString()));
        }
    }
}
复制代码

Consumer Priority

Disruptor can do things far more than the contents above. In the actual scene, we usually because the business logic to form a consumption chain. For example, a message must be made by 消费者A -> 消费者B -> 消费者Cthe order to consume in order. When configuring the consumer, through .thento implementation. as follows:

disruptor.handleEventsWith(new OrderEventHandler())
         .then(new OrderEventHandler())
         .then(new OrderEventHandler());
复制代码

Of course, handleEventsWithwith handleEventsWithWorkerPoolall support .then, and they can be used in combination. For example, you can follow the 消费者A -> (消费者B 消费者C) -> 消费者Dconsumer order

disruptor.handleEventsWith(new OrderEventHandler())
         .thenHandleEventsWithWorkerPool(new OrderEventHandler(), new OrderEventHandler())
         .then(new OrderEventHandler());
复制代码

to sum up

These are the common methods Disruptor high performance queue. In fact, 生成者 -> 消费者the pattern is very common, through a number of message queues can easily do the above effects. The difference is that, in the Disruptor memory to queue way to achieve, but no lock. This is also the reason why the efficient Disruptor.