CopyOnWriteList implementation details

I summed up in front of a set of Java source code details the ArrayList , which also referred to the ArrayList is not thread-safe, then what jdk thread for us to provide secure List is, that is to say the following CopyOnWriteList the concurrent safe collections, its main use is the copy-on-writeidea, this idea probably is 读写分离：读时共享、写时复制(原本的array)更新(且为独占式的加锁), but the source of our analysis below also reflects the realization of this idea

Outline

I would first affixed inheritance system CopyOnWriteList it, you can see it implements Serializable, Cloneable and RandomAccess interface, with features random access, implements List interface, with List properties.

We look at the individual main method to analyze the main properties and the following CopyOnWriteList of what. It is seen from the figure:

• Each CopyOnWriteList objects inside a particular array element array to store

• ReentrantLock exclusive use locks to ensure that only write thread array copy to be updated. About ReentrantLock I can refer to another article ReentrantLock application of the AQS .

• CopyOnWriteArrayList in traversal using ConcurrentModificationException not throw an exception, and when he traversed without additional lock

  The following is mainly to see the realization of CopyOnWriteList

Member property

//这个就是保证更新数组的时候只有一个线程能够获取lock，然后更新
final transient ReentrantLock lock = new ReentrantLock();
//使用volatile修饰的array，保证写线程更新array之后别的线程能够看到更新后的array.
//但是并不能保证实时性：在数组副本上添加元素之后，还没有更新array指向新地址之前，别的读线程看到的还是旧的array
private transient volatile Object[] array;
//获取数组，非private的，final修饰
final Object[] getArray() {
    return array;
}
//设置数组
final void setArray(Object[] a) {
    array = a;
}
复制代码

Construction method

(1) constructor with no arguments, the default is to create an array of length 0

//这里就是构造方法，创建一个新的长度为0的Object数组
//然后调用setArray方法将其设置给CopyOnWriteList的成员变量array
public CopyOnWriteArrayList() {
    setArray(new Object[0]);
}
复制代码

(2) Collection constructor parameter

//按照集合的迭代器遍历返回的顺序，创建包含传入的collection集合的元素的列表
//如果传递的参数为null，会抛出异常
public CopyOnWriteArrayList(Collection<? extends E> c) {
    Object[] elements; //一个elements数组
    //这里是判断传递的是否就是一个CopyOnWriteArrayList集合
    if (c.getClass() == CopyOnWriteArrayList.class)
        //如果是，直接调用getArray方法，获得传入集合的array然后赋值给elements
        elements = ((CopyOnWriteArrayList<?>)c).getArray();
    else {
        //先将传入的集合转变为数组形式
        elements = c.toArray();
        //c.toArray()可能不会正确地返回一个 Object[]数组，那么使用Arrays.copyOf()方法
        if (elements.getClass() != Object[].class)
            elements = Arrays.copyOf(elements, elements.length, Object[].class);
    }
    //直接调用setArray方法设置array属性
    setArray(elements);
}
复制代码

(3) create a list that contains a copy of the given array

public CopyOnWriteArrayList(E[] toCopyIn) {
    setArray(Arrays.copyOf(toCopyIn, toCopyIn.length, Object[].class));
}
复制代码

CopyOnWriteList described above is initialized, the three construction methods are relatively straightforward, the latter is also a major look at the implementation of several major method

Adding elements

Here is the add (E e) of the method, and detailed notes

public boolean add(E e) {
    //获得独占锁
    final ReentrantLock lock = this.lock;
    //加锁
    lock.lock();
    try {
        //获得list底层的数组array
        Object[] elements = getArray();
        //获得数组长度
        int len = elements.length;
        //拷贝到新数组，新数组长度为len+1
        Object[] newElements = Arrays.copyOf(elements, len + 1);
        //给新数组末尾元素赋值
        newElements[len] = e;
        //用新的数组替换掉原来的数组
        setArray(newElements);
        return true; 
    } finally {
        lock.unlock();//释放锁
    }
}
复制代码

To sum up the implementation process of the add method

• Call to add to the thread will first acquire a lock, then calls the lock method for locking of the list (to understand ReentrantLock know this is an exclusive lock, so that only a multi-threaded thread acquires the lock)
• Only thread acquires the lock, so that only one thread will go to update this array, other thread calls the add method of this process is blocked waiting for
• Get to lock the thread to continue
  • First acquires the original array and its length, will be copied and then a new element to the array (the length of the original length newArray +1)
  • Given array subscript len ​​+ 1 at the assignment
  • The new array replaces the original array
• Finally, the lock is released

So sum up, multi-threaded only one thread can acquire the lock, and then use the replication of the original array of add elements, and then replace the original array after the new array, and then release the lock (add another thread to perform ).

Last but not least is the array length is not fixed, the length of each array will be +1 after write, so CopyOnWriteList no length or size of these properties, but provides the size () method to get the actual size of the collection, size ( )Methods as below

public int size() {
    return getArray().length;
}
复制代码

Gets the element

Use get (i) i can get the element at the location, of course, if the element does not exist will throw an array bounds exception.

public E get(int index) {
    return get(getArray(), index);
}
final Object[] getArray() {
    return array;
}
private E get(Object[] a, int index) {
    return (E) a[index];
}
复制代码

Of course, the get method here also reflects the copy-on-write-listweak consistency. We use the following illustration briefly explain. FIG scenario is given: threadA access element at index = 1

• ① acquisition array array
• ② access the underlying elements of parameters passed

Since we saw not get process is locked (assuming there are three elements in the array as shown). ② After threadA assumed before performing ①, threadB performed remove (1) operation, or an exclusive lock ThreadB, and then performs a write copy, i.e. 复制一个新的数组neArray, the remove operation is then performed in the newArray (1), update array. threadB finished elements of array index = 1 is already a item3.

Then threadA continue, but because threadA operation is the original elements in the array, this time index = 1 or item2. Although the phenomenon is so the final element threadB deleted at position 1 of the elements, but threadA still access the original array. This is若一致性问题

Modify elements

Modification also belongs 写, it is necessary to obtain the lock, the following is a method to achieve set

public E set(int index, E element) {
    //获取锁
    final ReentrantLock lock = this.lock;
    //进行加锁
    lock.lock();
    try {
        //获取数组array
        Object[] elements = getArray();
        //获取index位置的元素
        E oldValue = get(elements, index);
        // 要修改的值和原值不相等
        if (oldValue != element) {
            //获取旧数组的长度
            int len = elements.length;
            //复制到一个新数组中
            Object[] newElements = Arrays.copyOf(elements, len);
            //在新数组中设置元素值
            newElements[index] = element;
            //用新数组替换掉原数组
            setArray(newElements);
        } else {
            // Not quite a no-op; ensures volatile write semantics
            //为了保证volatile 语义，即使没有修改，也要替换成新的数组
            setArray(elements);
        }
        return oldValue; //返回旧值
    } finally {
        lock.unlock();//释放锁
    }
}
复制代码

After reading set method, we discovered that in fact and add methods to achieve similar.

• Obtain an exclusive lock to ensure that only one thread can modify the array
• Get the current array, call the get method to obtain the specified location of the array elements
• Judgment and values ​​of parameters passed get Gets
  • Equal, in order to ensure volatile semantics, or the need to re-only array
  • Are not equal, the original copied to the new array element array, and then modify the index in the new array, replace the original array modification is completed with a new array
• Release the lock

Removing elements

Here is the remove method, the conclusion is that

• Acquire an exclusive lock to ensure that only one thread can to remove elements
• Count the number of array elements to be moved
  • If you delete the last element is then calculated above is 0, the original pre-len-1 substitutions array as the original array directly off the new array
  • Deleted is not the last element, then in accordance with the two-part index is divided into before and after, are copied to the new array, and then can be replaced
• Release the lock

public E remove(int index) {
    //获取锁
    final ReentrantLock lock = this.lock;
    //加锁
    lock.lock();
    try {
        //获取原数组
        Object[] elements = getArray();
        //获取原数组长度
        int len = elements.length;
        //获取原数组index处的值
        E oldValue = get(elements, index);
        //因为数组删除元素需要移动，所以这里就是计算需要移动的个数
        int numMoved = len - index - 1;
        //计算的numMoved=0，表示要删除的是最后一个元素，
        //那么旧直接将原数组的前len-1个复制到新数组中，替换旧数组即可
        if (numMoved == 0)
            setArray(Arrays.copyOf(elements, len - 1));
        //要删除的不是最后一个元素
        else {
            //创建一个长度为len-1的数组
            Object[] newElements = new Object[len - 1];
            //将原数组中index之前的元素复制到新数组
            System.arraycopy(elements, 0, newElements, 0, index);
            //将原数组中index之后的元素复制到新数组
            System.arraycopy(elements, index + 1, newElements, index,
                             numMoved);
            //用新数组替换原数组
            setArray(newElements);
        }
        return oldValue;//返回旧值
    } finally {
        lock.unlock();//释放锁
    }
}
复制代码

Iterator

The basic use of the following iterator, hashNext () method is used to determine whether there are more elements, next method returns the concrete elements.

CopyOnWriteArrayList list = new CopyOnWriteArrayList();
Iterator<?> itr = list.iterator();
while(itr.hashNext()) {
    //do sth
    itr.next();
}
复制代码

So in CopyOnWriteArrayList iterator is how to achieve it, why is it weak consistency ( 先获取迭代器的，但是如果在获取迭代器之后别的线程对list进行了修改，这对于迭代器是不可见的), said the following about the realization of CopyOnWriteArrayList

//Iterator<?> itr = list.iterator();
public Iterator<E> iterator() {
    //这里可以看到，是先获取到原数组getArray()，这里记为oldArray
    //然后调用COWIterator构造器将oldArray作为参数，创建一个迭代器对象
    //从下面的COWIterator类中也能看到，其中有一个成员存储的就是oldArray的副本
    return new COWIterator<E>(getArray(), 0);
}
static final class COWIterator<E> implements ListIterator<E> {
    //array的快照版本
    private final Object[] snapshot;
    //后续调用next返回的元素索引(数组下标)
    private int cursor;
    //构造器
    private COWIterator(Object[] elements, int initialCursor) {
        cursor = initialCursor;
        snapshot = elements;
    }
    //变量是否结束：下标小于数组长度
    public boolean hasNext() {
        return cursor < snapshot.length;
    }
    //是否有前驱元素
    public boolean hasPrevious() {
        return cursor > 0;
    }
    //获取元素
    //hasNext()返回true，直接通过cursor记录的下标获取值
    //hasNext()返回false，抛出异常
    public E next() {
        if (! hasNext())
            throw new NoSuchElementException();
        return (E) snapshot[cursor++];
    }
    //other method...
}
复制代码

In the above code, we can look at, the list iterator () method actually returns a COWIterator objects, snapshot member variables COWIterator object holds the 当前contents of the storage array's list, but it can be a snapshot of the array snapshot, why do you say

Our message is that although the current array, but there may be another thread arrayhas been modified then the original arrayreplaced, then this time the list of arrayand snapshotreferences arrayis not a, and as the original arraysnapshot exists, the iterator after the visit of the array will not be updated. This is the manifestation of weak consistency

We look at the following example

public class TestCOW {

    private static CopyOnWriteArrayList<String> list = new CopyOnWriteArrayList();

    public static void main(String[] args) throws InterruptedException {
        list.add("item1");
        list.add("item2");
        list.add("item3");

        Thread thread = new Thread() {
            @Override
            public void run() {
                list.set(1, "modify-item1");
                list.remove("item2");
            }
        };
        //main线程先获得迭代器
        Iterator<String> itr = list.iterator();
        thread.start();//启动thread线程
        thread.join();//这里让main线程等待thread线程执行完，然后再遍历看看输出的结果是不是修改后的结果
        while (itr.hasNext()) {
            System.out.println(Thread.currentThread().getName() + "线程中的list的元素:" + itr.next());
        }
    }
}

复制代码

Results are as follows. In fact the above procedure and then we Xianxiang list to add several elements, then thread the modified list, while allowing main线程先获得list的迭代器, and then wait for the thread executing the print element in the list, find the main thread did not find list of array changes in output or the original list, which is the manifestation of weak consistency.

main thread of the elements of list: item1 main thread of the list of elements: item2 main thread of the list of elements: item3

to sum up

• How CopyOnWriteArrayList is to ensure 写thread safety: Use ReentrantLock exclusive lock to ensure that only one thread of the collection 写operation
• Data are stored in the array in the array CopyOnWriteArrayList, and the array length is dynamic ( 写action updates the array)
• In modify the array when not directly operate array, but copied out a new array, modification, and then replace the old array into a new array
• Using an iterator to traverse when not locked, ConcurrentModificationException not throw an exception, because the use of iterator traversal operation is a copy of the array (of course, modify the list of which is the case in other threads)

set method details

Noting set method is a piece of code

else { //oldValue = element（element是传入的参数）
    // Not quite a no-op; ensures volatile write semantics
    //为了保证volatile 语义，即使没有修改，也要替换成新的数组
    setArray(elements);
}
复制代码

In fact, that is to specify the location you want to modify the values in the array and that position is the same, but still need to call the set method to update array, which is why, with reference to this post , just to summarize 维护happens-before原则. First look at this passage

Method in java.util.concurrent package and its sub-classes for all of these extensions ensure the synchronization of a higher level. In particular: in the object into a thread happen-before operation prior to any concurrent access or removal of the collection element from the collection of another thread ** 后续操作**.

It will be appreciated here is to ensure as set operation before the series of operations happen-before the other threads to access array (not locked) in 后续操作reference to the following examples

// 这是两个线程的初始情况
int nonVolatileField = 0; //一个不被volatile修饰的变量
//伪代码
CopyOnWriteArrayList<String> list = {"x","y","z"}

// Thread 1
// (1)这里更新了nonVolatileField
nonVolatileField = 1;
// (2)这里是set()修改（写）操作，注意这里会对volatile修饰的array进行写操作
list.set(0, "x");

// Thread 2
// (3)这里是访问（读）操作
String s = list.get(0);
// (4)使用nonVolatileField
if (s == "x") {
    int localVar = nonVolatileField;
}
复制代码

The presence of the above scenario assumes that, if only to ensure that there is a track: (1) -> (2) -> (3) -> (4) based on the java API documentation conventions

(2) happen-before and (3), in operation with a thread (. 1) happen-before and (2), (3) happen-before and (4) The transfer of the read-write happen-before nonVolatileField variables have (1) happen-before and (4)

Therefore Thread 1 writes to nonVolatileField visible Thread 2 in a read operation. If the set else CopyOnWriteArrayList's no setArray (elements) 对volatile变量的写, then, (2) happen-before and (3) no longer have the visibility it can not be guaranteed.

It is to ensure that a series of operations happen-before before the set operation with other threads access the array (unlocked) of 后续操作,