A brief introduction and use of Disruptor

This article introduces the version is 3.3.6, refer to the following articles

 

http://11246272.blog.51cto.com/11236272/1745472

http://www.php.cn/java-article-370582.html

http://zhangfengzhe.blog.51cto.com/8855103/1885830

http://ifeve.com/concurrentlinkedqueue/

http://ifeve.com/disruptor/

http://wh0426.iteye.com/blog/221202

https://my.oschina.net/OutOfMemory/blog/793275

 

What is Disruptor?

Disruptor is a high-performance asynchronous processing framework, a "producer-consumer" model, a lightweight JMS, similar to BlockingQueue in JDK, but its processing speed is very fast, and won the 2011 program framework innovation The award, known as "a thread can process 600W orders in one second", and Disruptor is not only a buffer, it provides very powerful functions, such as it can help us easily build data flow processing (such as a data first handed over to A and B These two consumers are processed in parallel and then handed over to C for processing. Do you think of the stream processing of Storm? In fact, the bottom layer of strom is to use the disruptor to realize the communication of threads within the worker).

Ring buffer (tire): RingBuffer

RingBuffer, the ring buffer, plays a very important role in the disruptor. It contains a sequence number pointing to the next slot, which can transfer data between threads. Understanding the structure of the RingBuffer will help us understand why the disruptor is so fast and lock-free. methods, implementation details of the producer/consumer pattern. As shown below:



 

array

This tire-like thing is actually an array. The advantage of using an array is of course that access is much faster than a linked list due to preloading.

serial number

The elements in RingBuffer have the concept of serial number, and the serial number is always increasing. For example, if the RingBuffer size is 10, then the serial number starts to increase from 0. When it reaches 9, it is equivalent to turning a circle. If it continues to grow, it will cover 0. number element. That is to say, the element is located by the serial number %SIZE, and the set/get operation is realized. A method different from the queue is also found here, that is, there is no element deletion operation, only overwriting. In fact, this special ring storage method of RingBuffer makes it unnecessary to spend a lot of time for memory cleanup/garbage collection. Due to the modulo operation involved, the size of the RingBuffer should be 2 to the Nth power for the CPU to perform bit operations more efficiently.

lock-free mechanism

In the producer/consumer mode, the disruptor is known as a "lock-free parallel framework". Let's analyze it in detail:

one producer + one consumer

The producer maintains a production pointer P, and the consumer maintains a consumer pointer C. Of course, P and C are essentially serial numbers. Each of the two operations does not require locks. The only thing to pay attention to is the speed of producers and consumers. Of course, this has been dealt with for us inside the disruptor, that is, to judge that there can be no more than one circle between P and C. size.

One producer + multiple consumers

Of course, multiple consumers hold multiple consumption pointers C1, C2, . Beyond the concept of a circle. Locking is also not required at this time.

Multiple producers + N consumers

Obviously, no matter how many producers there are, there can only be one producer pointer P, otherwise the data will be messed up. Then a P pointer is shared between multiple producers. In the disruptor, the CAS mechanism is actually used to ensure the data security of multiple threads, and no locks are used.

Event

In the Disruptor framework, the data produced by the producer is called Event.

core object

RingBuffer: A ring-shaped data structure. When the object is initialized, it will be filled with the event Event. The size of the Buffer must be a power of 2, which is convenient for shifting operations.

  1. Event: No specific interface is specified, the user implements it by himself, and can carry any business data.

  2. EventFactory: The factory that generates the event Event, which is implemented by the user.

  3. EventTranslator: The callback interface for event publishing, implemented by the user, responsible for setting business parameters into the event.

  4. Sequencer: The sequence generator is also the core for coordinating producers and consumers and achieving high concurrency. There are two implementation classes MultiProducerSequencer and SingleProducerSequencer.

  5. SequenceBarrier: Holds the Sequence reference of the RingBuffer's publishing event and the Sequence reference that the consumer depends on. Determines the Sequence that the consumer consumes consumable.

  6. EventHandler: The handler of the event, implemented by the user himself.

  7. EventProcessor: The processor of the event, which runs in a separate thread.

  8. WorkHandler: The handler of the event, implemented by the user.

  9. WorkProcessor: The processor of the event, which runs in a thread alone.

  10. WorkerPool: A set of WorkProcessor processing.

  11. WaitStrategy: The waiting strategy of the consumer processor when the consumer is faster than the producer.

Waiting Policy:

The waiting strategy adopted by consumers when there are no events that can be consumed in the cache:

1.BlockingWaitStrategy: The default waiting strategy. Similar to the implementation of BlockingQueue, it waits for the producer to wake up (thread synchronization and wake-up) by using locks and conditions to block threads. This strategy saves CPU resources the most for thread switching, but has limited performance in high concurrency scenarios.

2.BusySpinWaitStrategy: Infinite loop strategy. The consumer thread will try its best to monitor the change of the buffer, which will occupy all CPU resources. The thread keeps spinning and waiting, which consumes more CPU.

3. LiteBlockingWaitStrategy: By blocking the thread, waiting for the producer to wake up, which is lighter than BlockingWaitStrategy, and can reduce the number of blocking in some cases.

4.PhasedBackoffWaitStrategy: Determine which waiting strategy to use according to the specified time period parameters and the specified waiting strategy. 5.SleepingWaitStrategy: CPU-friendly strategy. It will keep waiting for data in a loop. You can set the parameters to first spin and wait. If unsuccessful, use Thread.yield() to yield the CPU, and use LockSupport.parkNanos(1) to sleep the thread and reschedule through the thread scheduler; or keep spinning and waiting , so the data processing data of this strategy may have a high delay, and it is suitable for scenarios that are not sensitive to delay. The advantage is that it has little impact on the producer thread. The typical application scenario is asynchronous logging.

6.TimeoutBlockingWaitStrategy: Set the blocking time through parameters, and throw an exception if it times out.

7.YieldingWaitStrategy: Low latency strategy. The consumer thread will continuously monitor the changes of the RingBuffer in a loop, use Thread.yield() inside the loop to give up the CPU to other threads, and reschedule it through the thread scheduler.

Basic composition of the Disruptor framework

1.MyEvent: A custom entity object that acts as data in the "producer-consumer" model.

2.MyEventFactory: Implements the interface of EventFactory for data production.

3.MyEventProducerWithTranslator: Store data in a custom object and publish it, by creating a new EventTranslator class in the custom class.

4.MyEventHandler: Custom consumer, implemented through the EventHandler interface.

Comparison with ConcurrentLinkedQueue

Same point

1. They are all implementations of CAS (Compare And Swap/Set) using a lock-free algorithm

difference

1. In the case of multi-threading, there is no competition for Disruptors, and each object uses its own sequence number; because ConcurrentLinkedQueue is a queue, other threads may interrupt the queue. If a thread is enqueuing, then it must first get the tail node, and then set the next node of the tail node as the enqueue node, but at this time another thread may enter the queue, then the tail node of the queue will change, At this time, the current thread needs to suspend the enqueue operation, and then re-acquire the tail node, so it is time-consuming.

2. Disruptor is a circular buffer, an array, it will not clear the existing data, it will only update an available serial number, and then write data to the serial number, because each object has its own Therefore, there is no write conflict, and ConcurrentLinkedQueue is an unbounded queue. It will judge whether the data has been used according to the pointer, and if it has been used, the mark of the head node will be updated. Therefore, Disurptor is more time-consuming than ConcurrentLinkedQueue. few.

3.Disruptor solves the problem of false sharing.

For more introduction to Disruptor, please refer to  http://ifeve.com/disruptor/  Although the above introduction version is a bit old, the principle is basically the same.

Disurptor's java test implementation

1. Import the required jar package

<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.7.RELEASE</version>
    </parent>

    <dependencies>
        <dependency>
            <groupId>com.lmax</groupId>
            <artifactId>disruptor</artifactId>
            <version>3.3.7</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.20</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>

    </dependencies>

 

2. Custom entity object

package com.demo.disruptor.dto;

import com.alibaba.fastjson.JSONObject;

import java.util.Date;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description custom event object
 */
public class LogEvent {

    private long logId;
    private String content;
    private Date date;

    public long getLogId() {
        return logId;
    }

    public void setLogId (long logId) {
        this.logId = logId;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }

    public Date getDate() {
        return date;
    }

    public void setDate(Date date) {
        this.date = date;
    }

    @Override
    public String toString() {
        return JSONObject.toJSONString(this);
    }
}

 3. Custom factory class:

package com.demo.disruptor.factory;

import com.demo.disruptor.dto.LogEvent;
import com.lmax.disruptor.EventFactory;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description event generation factory, used to initialize the pre-allocated event object, that is, the entity object created according to the size of the RingBuffer
 */
public class LogEventFactory implements EventFactory<LogEvent> {
    public LogEvent newInstance() {
        System.out.println("New LogEvent data.....");
        return new LogEvent();
    }
}

 4. Create a new producer class

4.1 Non-Translator Producer Class

package com.demo.disruptor.producer;

import com.demo.disruptor.dto.LogEvent;
import com.lmax.disruptor.RingBuffer;

import java.util.Date;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description custom producer
 */
public class LogEventProducer {

    private RingBuffer<LogEvent> ringBuffer;

    public LogEventProducer(RingBuffer<LogEvent> ringBuffer) {
        this.ringBuffer = ringBuffer;
    }

    public void onData(long logId, String content, Date date){
        //RingBuffer is like a queue to get the next free sequence number
        long seq = ringBuffer.next();
        LogEvent logEvent = ringBuffer.get(seq);
        logEvent.setLogId (logId);
        logEvent.setContent(content);
        logEvent.setDate(date);
        //发布事件
        ringBuffer.publish(seq);
    }
}

 

package com.demo.disruptor.producer;

import com.demo.disruptor.dto.LogEvent;
import com.lmax.disruptor.RingBuffer;

import java.util.Date;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description 自定义生产者
 */
public class LogEventProducer2 {

    private RingBuffer<LogEvent> ringBuffer;

    public LogEventProducer2(RingBuffer<LogEvent> ringBuffer) {
        this.ringBuffer = ringBuffer;
    }

    public void onData(long logId, String content, Date date){
        //RingBuffer类似一个队列,获取下一个空闲的序号
        long seq = ringBuffer.next();
        LogEvent logEvent = ringBuffer.get(seq);
        logEvent.setLogId(logId);
        logEvent.setContent(content);
        logEvent.setDate(date);
        //发布事件
        ringBuffer.publish(seq);
    }
}

 LogEventProducer3的类和生产者2也一样

4.2 Translator生产者类

package com.demo.disruptor.producer;

import com.demo.disruptor.dto.LogEvent;
import com.lmax.disruptor.EventTranslatorVararg;
import com.lmax.disruptor.RingBuffer;

import java.util.Date;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description 使用translator方式到事件生产者发布事件,通常使用该方法
 */
public class LogEventProducerWithTranslator {

    private EventTranslatorVararg eventTranslatorVararg = new EventTranslatorVararg<LogEvent>() {
        public void translateTo(LogEvent logEvent, long l, Object... objects) {
            logEvent.setLogId((Long) objects[0]);
            logEvent.setContent((String)objects[1]);
            logEvent.setDate((Date)objects[2]);
        }
    };

    private RingBuffer<LogEvent> ringBuffer;

    public LogEventProducerWithTranslator(RingBuffer<LogEvent> ringBuffer) {
        this.ringBuffer = ringBuffer;
    }

    public void onData(long logId, String content, Date date){
        ringBuffer.publishEvent(eventTranslatorVararg,logId,content,date);
    }
}

 5.新建消费者类

package com.demo.disruptor.consumer;

import com.demo.disruptor.dto.LogEvent;
import com.lmax.disruptor.EventHandler;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description 自定义消费者
 */
public class LogEventConsumer implements EventHandler<LogEvent> {
    public void onEvent(LogEvent logEvent, long l, boolean b) throws Exception {
        System.out.println("消费者1-seq:" + l + ",bool:" + b + ",logEvent:" + logEvent.toString());
    }
}

 

package com.demo.disruptor.consumer;

import com.demo.disruptor.dto.LogEvent;
import com.lmax.disruptor.EventHandler;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description 自定义消费者
 */
public class LogEventConsumer2 implements EventHandler<LogEvent> {
    public void onEvent(LogEvent logEvent, long l, boolean b) throws Exception {
        System.out.println("消费者2-seq:" + l + ",bool:" + b + ",logEvent:" + logEvent.toString());
    }
}

 剩下的LogEventConsumer3、LogEventConsumer4、LogEventConsumer5的类同LogEventConsumer2类一样。

6.新建启动类

package com.demo.disruptor;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description
 */
@SpringBootApplication
public class App {

    public static void main(String [] args){
        SpringApplication.run(App.class,args);
    }

}

 7.新建测试类测试Disruptor

package com.demo.disruptor.logEvent;

import com.demo.disruptor.consumer.*;
import com.demo.disruptor.dto.LogEvent;
import com.demo.disruptor.factory.LogEventFactory;
import com.demo.disruptor.producer.LogEventProducer;
import com.demo.disruptor.producer.LogEventProducer2;
import com.demo.disruptor.producer.LogEventProducer3;
import com.demo.disruptor.producer.LogEventProducerWithTranslator;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.WaitStrategy;
import com.lmax.disruptor.YieldingWaitStrategy;
import com.lmax.disruptor.dsl.Disruptor;
import com.lmax.disruptor.dsl.EventHandlerGroup;
import com.lmax.disruptor.dsl.ProducerType;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.Date;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadFactory;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/9
 * @description
 */
@SpringBootTest(classes = LogEventMain.class)
@RunWith(SpringRunner.class)
public class LogEventMain {

    /**
     * 单个生产者和消费者的模式
     * @throws InterruptedException
     */
    @Test
    public void producer() throws InterruptedException {
        LogEventFactory logEventFactory = new LogEventFactory();
        //用于生成RingBuffer大小,其大小必须是2的n次方
        int ringBufferSize = 8;
        //定义Disruptor初始化信息
        Disruptor<LogEvent> disruptor = new Disruptor<LogEvent>(logEventFactory,ringBufferSize,Executors.defaultThreadFactory(), ProducerType.SINGLE, new YieldingWaitStrategy());
        //定义处理事件的消费者
        disruptor.handleEventsWith(new LogEventConsumer());
        //定义事件的开始
        disruptor.start();

        RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
        //进行事件的发布
        LogEventProducer logEventProducer = new LogEventProducer(ringBuffer);
        for(int i = 0; i < 10; i++){
            logEventProducer.onData(i, "c" + i, new Date());
        }
        Thread.sleep(1000);
        //关闭Disruptor
        disruptor.shutdown();
    }

    /**
     * 使用EventTranslatorVararg的单个生产者和消费者模式
     * @throws InterruptedException
     */
    @Test
    public void producerWithTranslator() throws InterruptedException {
        LogEventFactory logEventFactory = new LogEventFactory();
        //用于生成RingBuffer大小,其大小必须是2的n次方
        int ringBufferSize = 8;
        //定义Disruptor初始化信息
        Disruptor<LogEvent> disruptor = new Disruptor<LogEvent>(logEventFactory,ringBufferSize,Executors.defaultThreadFactory(), ProducerType.SINGLE, new YieldingWaitStrategy());
        //定义处理事件的消费者
        disruptor.handleEventsWith(new LogEventConsumer());
        //定义事件的开始
        disruptor.start();

        RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
        //进行事件的发布
        LogEventProducerWithTranslator producerWithTranslator = new LogEventProducerWithTranslator(ringBuffer);
        for(int i = 0; i < 10; i++){
            producerWithTranslator.onData(i, "c" + i, new Date());
        }
        Thread.sleep(1000);
        //关闭Disruptor
        disruptor.shutdown();
    }

    /**
     * 一个生产者,3个消费者,其中前面2个消费者完成后第3个消费者才可以消费
     * 也即使说当前面2个消费者把所有的RingBuffer占领完成,同时都消费完成后才会有第3个消费者的消费
     * 当发布的事件数量大于RingBuffer的大小的时候,在第3个消费者消费完RingBuffer大小的时候前面2个消费者才能继续消费,序号递增的
     * @throws InterruptedException
     */
    @Test
    public void multiConsumer() throws InterruptedException {
        LogEventFactory logEventFactory = new LogEventFactory();
        //用于生成RingBuffer大小,其大小必须是2的n次方
        int ringBufferSize = 8;
        //定义Disruptor初始化信息
        Disruptor<LogEvent> disruptor = new Disruptor<LogEvent>(logEventFactory,ringBufferSize,Executors.defaultThreadFactory(), ProducerType.SINGLE, new YieldingWaitStrategy());

        //设置多个消费者
        EventHandlerGroup<LogEvent> eventEventHandlerGroup = disruptor.handleEventsWith(new LogEventConsumer(),new LogEventConsumer2());
        eventEventHandlerGroup.then(new LogEventConsumer3());
        //启动事件的开始
        disruptor.start();
        RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
        //进行事件的发布
        LogEventProducerWithTranslator producerWithTranslator = new LogEventProducerWithTranslator(ringBuffer);
        for(int i = 0; i < 10; i++){
            producerWithTranslator.onData(i, "c" + i, new Date());
        }
        Thread.sleep(1000);
        //关闭Disruptor
        disruptor.shutdown();
    }

    /**
     * 一个生产者,多个消费者,有2条支线,其中消费者1和消费者3在同一条支线上,
     * 消费者2和消费者4在同一条支线上,消费者5是消费者3和消费者4的终点消费者
     * 这样的消费将会在消费者1和消费者2把所有的RingBuffer大小消费完成后才会执行消费者3和消费者4
     * 在消费者3和消费者4把RingBuffer大小消费完成后才会执行消费者5
     * 消费者5消费完RingBuffer大小后又按照上面的顺序来消费
     * 如果剩余的生产数据比RingBuffer小,那么还是要依照顺序来
     * @throws InterruptedException
     */
    @Test
    public void multiConsumers() throws InterruptedException {
        LogEventFactory logEventFactory = new LogEventFactory();
        //用于生成RingBuffer大小,其大小必须是2的n次方
        int ringBufferSize = 8;
        //定义Disruptor初始化信息
        Disruptor<LogEvent> disruptor = new Disruptor<LogEvent>(logEventFactory,ringBufferSize,Executors.defaultThreadFactory(), ProducerType.SINGLE, new YieldingWaitStrategy());
        LogEventConsumer consumer1 = new LogEventConsumer();
        LogEventConsumer2 consumer2 = new LogEventConsumer2();
        LogEventConsumer3 consumer3 = new LogEventConsumer3();
        LogEventConsumer4 consumer4 = new LogEventConsumer4();
        LogEventConsumer5 consumer5 = new LogEventConsumer5();
        //同时执行消费者1和消费者2
        disruptor.handleEventsWith(consumer1,consumer2);
        //消费者1后面执行消费者3
        disruptor.after(consumer1).handleEventsWith(consumer3);
        //消费者后面执行消费者4
        disruptor.after(consumer2).handleEventsWith(consumer4);
        //消费者3和消费者3执行完后执行消费者5
        disruptor.after(consumer3,consumer4).handleEventsWith(consumer5);
        //定义事件的开始
        disruptor.start();

        RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
        //进行事件的发布
        LogEventProducer logEventProducer = new LogEventProducer(ringBuffer);
        for(int i = 0; i < 10; i++){
            logEventProducer.onData(i, "c" + i, new Date());
        }
        Thread.sleep(1000);
        //关闭Disruptor
        disruptor.shutdown();
    }

    /**
     * 多个生产者,多个消费者,有2条消费者支线,其中消费者1和消费者3在同一条支线上,
     * 消费者2和消费者4在同一条支线上,消费者5是消费者3和消费者4的终点消费者
     * 这样的消费将会在消费者1和消费者2把所有的RingBuffer大小消费完成后才会执行消费者3和消费者4
     * 在消费者3和消费者4把RingBuffer大小消费完成后才会执行消费者5
     * 消费者5消费完RingBuffer大小后又按照上面的顺序来消费
     * 如果剩余的生产数据比RingBuffer小,那么还是要依照顺序来
     * 生产者只是多生产了数据
     * @throws InterruptedException
     */
    @Test
    public void multiProcedureConsumers() throws InterruptedException {
        LogEventFactory logEventFactory = new LogEventFactory();
        //用于生成RingBuffer大小,其大小必须是2的n次方
        int ringBufferSize = 8;
        //定义Disruptor初始化信息
        Disruptor<LogEvent> disruptor = new Disruptor<LogEvent>(logEventFactory,ringBufferSize,Executors.defaultThreadFactory(), ProducerType.MULTI, new YieldingWaitStrategy());
        LogEventConsumer consumer1 = new LogEventConsumer();
        LogEventConsumer2 consumer2 = new LogEventConsumer2();
        LogEventConsumer3 consumer3 = new LogEventConsumer3();
        LogEventConsumer4 consumer4 = new LogEventConsumer4();
        LogEventConsumer5 consumer5 = new LogEventConsumer5();
        //同时执行消费者1和消费者2
        disruptor.handleEventsWith(consumer1,consumer2);
        //消费者1后面执行消费者3
        disruptor.after(consumer1).handleEventsWith(consumer3);
        //消费者后面执行消费者4
        disruptor.after(consumer2).handleEventsWith(consumer4);
        //消费者3和消费者3执行完后执行消费者5
        disruptor.after(consumer3,consumer4).handleEventsWith(consumer5);
        //定义事件的开始
        disruptor.start();

        RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
        //进行事件的发布
        LogEventProducer logEventProducer = new LogEventProducer(ringBuffer);
        LogEventProducer2 logEventProducer2 = new LogEventProducer2(ringBuffer);
        LogEventProducer3 logEventProducer3 = new LogEventProducer3(ringBuffer);
        for(int i = 0; i < 10; i++){
            logEventProducer.onData(i, "1-c" + i, new Date());
            logEventProducer2.onData(i, "2-c" + i, new Date());
            logEventProducer3.onData(i, "3-c" + i, new Date());
        }
        Thread.sleep(1000);
        //关闭Disruptor
        disruptor.shutdown();
    }

}

 8.单个线程的ArrayBlockingQueue和Disruptor的性能测试对比

我们这里分别用500W,1000W,5000W的数据量来做单个生产者和消费者的测试,分别进行了10次测试,然后取他们的平均值做对比。

 

package com.demo.disruptor.test;

import com.demo.disruptor.dto.LogEvent;

import java.util.Date;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/11
 * @description
 */
public class BlockingQueueTest {
    public static int eventNum = 50000000;
//5000000:974,932,943,946,993,1073,1044,1018,1027,971  992
    //10000000:1845,1851,2433,2041,1789,1911,1953,2105,1862,1896   1969
    //50000000:9828,9595,9377,9273,9020,9450,9873,9994,8882,9695  9499
    public static void main(String[] args) {
        final BlockingQueue<LogEvent> queue = new ArrayBlockingQueue<LogEvent>(65536);
        final long startTime = System.currentTimeMillis();
        new Thread(new Runnable() {
            @Override
            public void run() {
                int i = 0;
                while (i < eventNum) {
                    LogEvent logEvent = new LogEvent();
                    logEvent.setLogId(i);
                    logEvent.setContent("c" + i);
                    logEvent.setDate(new Date());
                    try {
                        queue.put(logEvent);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    i++;
                }
            }
        }).start();

        new Thread(new Runnable() {
            @Override
            public void run() {
                int k = 0;
                while (k < eventNum) {
                    try {
                        queue.take();
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    k++;
                }
                long endTime = System.currentTimeMillis();
                System.out.println("BlockingQueue 花费时间:" + (endTime - startTime) + "ms");
            }
        }).start();

    }
}
 
package com.demo.disruptor.test;

import com.demo.disruptor.consumer.LogEventConsumer;
import com.demo.disruptor.consumer.LogEventConsumer2;
import com.demo.disruptor.dto.LogEvent;
import com.demo.disruptor.factory.LogEventFactory;
import com.demo.disruptor.producer.LogEventProducer;
import com.demo.disruptor.producer.LogEventProducerWithTranslator;
import com.lmax.disruptor.BusySpinWaitStrategy;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.dsl.Disruptor;
import com.lmax.disruptor.dsl.ProducerType;
import com.lmax.disruptor.util.DaemonThreadFactory;

import java.util.Date;

/**
 * @author liaoyubo
 * @version 1.0 2017/10/11
 * @description
 */
public class DisruptorTest {

    //5000000:542,499,550,547,605,502,743,505,657,608     576
    //10000000:1252,1048,1031,1075,1022,1207,1056,1494,1118,1258   1156
    //50000000:5489,5125,5265,5609,5201,5482,4982,4891,5351,5758  5315
    public static void main(String [] args){
        LogEventFactory factory = new LogEventFactory();
        int ringBufferSize = 65536;
        final Disruptor<LogEvent> disruptor = new Disruptor<LogEvent>(factory,
                ringBufferSize, DaemonThreadFactory.INSTANCE,
                ProducerType.SINGLE, new BusySpinWaitStrategy());

        LogEventConsumer consumer = new LogEventConsumer();
        disruptor.handleEventsWith(consumer);
        disruptor.start();
        /*new Thread(new Runnable() {
            @Override
            public void run() {
                RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
                //进行事件的发布
                LogEventProducer logEventProducer = new LogEventProducer(ringBuffer);
                for(int i = 0; i < BlockingQueueTest.eventNum; i++){
                    logEventProducer.onData(i, "c" + i, new Date());
                }
            }
        }).start();*/

        new Thread(new Runnable() {
            @Override
            public void run() {
                RingBuffer<LogEvent> ringBuffer = disruptor.getRingBuffer();
                // publish the event
                LogEventProducerWithTranslator producerWithTranslator = new LogEventProducerWithTranslator(ringBuffer);
                for(int i = 0; i < BlockingQueueTest.eventNum; i++){
                    producerWithTranslator.onData(i, "c" + i, new Date());
                }
            }
        }).start();
        //disruptor.shutdown();
    }
}
 The comparison of test results is (all according to ms statistics):

 

quantity Disruptor ArrayBlockingQueue Performance comparison
500W 576 992 1.7
1000W 1156 1969 1.7
5000W 5315 9499 1.8
测试的结果是Disruptor的性能是ArrayBlockingQueue的1.7倍左右(电脑的配置是win10,64位,i5-6500的cpu,8g的内存,jdk是1.8),但是官方提供的数据是在5倍左右:https://github.com/LMAX-Exchange/disruptor/wiki/Performance-Results ,可能也与自己的电脑有关,还有就是一些参数上面的选择,如等待策略,但是不管怎样测试的性能说明了在生产者-消费者的模式下Disruptor的性能比ArrayBlockingQueue更好。

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=326708357&siteId=291194637