Disruptor高级（五）多生产者多消费者

多生产者多消费者模式概述

• 在单消费者模式下，每个监听（EventHandler）要对应一个线程，在初始化disruptor的时候，线程池中线程的数量就要设定好，要不然消费者没法正常工作；
• 在多生产者多消费者模式下，每个监听（EventHandler）不依赖一个单独的线程，而是委托给线程池；

多生产者多消费者模式示例

Event

public class Order {

    private String id;
    private String name;
    private double price;

    public Order() {}

    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public double getPrice() {
        return price;
    }
    public void setPrice(double price) {
        this.price = price;
    }
    
}

Consumer（监听）

• 在多生产者多消费者模式下，消费者要实现WorkHandler，而不是EventHandler；

import java.util.Random;
import java.util.concurrent.atomic.AtomicInteger;

import com.lmax.disruptor.WorkHandler;

public class Consumer implements WorkHandler<Order> {

    private String consumerId;
    
    private static AtomicInteger count = new AtomicInteger(0);
    
    private Random random = new Random();
    
    public Consumer(String consumerId) {
        this.consumerId = consumerId;
    }

    public void onEvent(Order event) throws Exception {
        Thread.sleep(1 * random.nextInt(5));
        System.err.println("当前消费者: " + this.consumerId + ", 消费信息ID: " + event.getId());
        count.incrementAndGet();
    }
    
    public int getCount(){
        return count.get();
    }

}

生产者

import com.lmax.disruptor.RingBuffer;

public class Producer {
    
    private RingBuffer<Order> ringBuffer;
    
    public Producer(RingBuffer<Order> ringBuffer) {
        this.ringBuffer = ringBuffer;
    }

    public void sendData(String uuid) {
        long sequence = ringBuffer.next();
        try {
            Order order = ringBuffer.get(sequence);
            order.setId(uuid);
        } finally {
            ringBuffer.publish(sequence);
        }
    }

}

disruptor

• 这里直接初始化了一个RingBuffer，而没有初始化Disruptor；
  • 初始化RingBuffer的时候指定了多生产者模式，ProducerType.MULTI；
• 通过ringBuffer创建一个屏障，这个屏障应该是供消费者用的吧；
• 用WorkerPool将多个消费者维护起来；
• 从workerPool中拿出多个消费者的Sequences，维护进ringBuffer，供多个生产者向ringBuffer中丢Event的时候使用；
• workerPool在启动的时候需要一个线程池，多个消费者就是通过这个线程池被分配任务的；
• EventExceptionHandler是在WorkPool初始化的时候需要的；

import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.CyclicBarrier;
import java.util.concurrent.Executors;

import com.lmax.disruptor.EventFactory;
import com.lmax.disruptor.ExceptionHandler;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.SequenceBarrier;
import com.lmax.disruptor.WorkerPool;
import com.lmax.disruptor.YieldingWaitStrategy;
import com.lmax.disruptor.dsl.ProducerType;

public class Main {

    public static void main(String[] args) throws InterruptedException {
        
        //1 创建RingBuffer
        RingBuffer<Order> ringBuffer =
                RingBuffer.create(ProducerType.MULTI,
                        new EventFactory<Order>() {
                            public Order newInstance() {
                                return new Order();
                            }
                        },
                        1024 * 1024,
                        new YieldingWaitStrategy());
        
        //2 通过ringBuffer 创建一个屏障
        SequenceBarrier sequenceBarrier = ringBuffer.newBarrier();
        
        //3 创建含有10个消费者的数组:
        Consumer[] consumers = new Consumer[10];
        for(int i = 0; i < consumers.length; i++) {
            consumers[i] = new Consumer("C" + i);
        }
        
        //4 构建多消费者工作池
        WorkerPool<Order> workerPool = new WorkerPool<Order>(
                ringBuffer,
                sequenceBarrier,
                new EventExceptionHandler(),
                consumers);
        
        //5 设置多个消费者的sequence序号 用于单独统计消费进度, 并且设置到ringbuffer中
        ringBuffer.addGatingSequences(workerPool.getWorkerSequences());
        
        //6 启动workerPool
        workerPool
        .start(Executors.newFixedThreadPool(5));
        
//      final CountDownLatch latch = new CountDownLatch(1);
        CyclicBarrier barrier = new CyclicBarrier(100);
        
        for(int i = 0; i < 100; i++) {
            final Producer producer = new Producer(ringBuffer);
            new Thread(new Runnable() {
                public void run() {
                    try {
//                      latch.await();
                        barrier.await();
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                    for(int j = 0; j < 100; j++) {
                        producer.sendData(UUID.randomUUID().toString());
                    }
                }
            }).start();
        }
        
        Thread.sleep(2000);
        System.err.println("----------线程创建完毕，开始生产数据----------");
//      latch.countDown();
        
        Thread.sleep(10000);
        
        System.err.println("任务总数:" + consumers[2].getCount());
    }
    
    static class EventExceptionHandler implements ExceptionHandler<Order> {

        public void handleEventException(Throwable ex, long sequence, Order event) {
        }

        public void handleOnStartException(Throwable ex) {
        }

        public void handleOnShutdownException(Throwable ex) {
        }

    }
    
}

输出：

...
当前消费者: C0, 消费信息ID: 479847ac-3be7-4332-a4eb-b8f4ee2983b0
当前消费者: C0, 消费信息ID: 45a674f7-1cc4-4a75-b15f-82ba2cf2ec9d
当前消费者: C1, 消费信息ID: b94deb7c-7103-4140-bf68-b468693e88db
当前消费者: C2, 消费信息ID: 2204b867-84ec-454b-b2d8-62963724325a
当前消费者: C3, 消费信息ID: bad977e5-fb59-4d56-9ba1-135fe32d80b9
当前消费者: C3, 消费信息ID: 65c55380-9520-4c61-95ca-9c3828e38f4e
当前消费者: C4, 消费信息ID: fd8377f8-9e57-4a8c-bac5-2289efc5e18c
当前消费者: C2, 消费信息ID: 572d65fa-5ec4-4b55-b766-16cb362af94b
当前消费者: C1, 消费信息ID: 358302a1-98ff-478b-a7f9-b9dbe7e2c2a9
当前消费者: C0, 消费信息ID: 1a0b70e0-c3bc-4792-b93a-ee7783289092
任务总数:10000