Preface
ConcurrentHashMap is a thread-safe and efficient HashMap provided in the JUC package. Therefore, ConcurrentHashMap is used more frequently in concurrent programming scenarios. Today, we will analyze how ConcurrentHashMap achieves security from the use of ConcurrentHashMap and the source code level.
The api uses
ConcurrentHashMap which is a derived class of Map, so the api is basically similar to Hashmap, mainly put and get methods. Next, based on the put and get methods of ConcurrentHashMap, we will analyze the source code implementation of ConcurrentHashMap as the entry point.
hash algorithm
Hash, generally translated as hash, hash, or transliterated as hash, is to transform an input of any length (also called a pre-mapped pre-image) into a fixed-length output through a hashing algorithm . The output is the hash value. This conversion is a compression mapping.
For example, MD5 and SHA are also hash algorithms
Solution when hash conflict occurs
- Linear exploration (open addressing method)
- Chain address method (HashMap)
- Re-hash method (through multiple hash functions) -> Bloom filter (bitMap)
- Establish a public overflow area
JDK1.7 and Jdk1.8 version changes
in JDK1.7, ConrruentHashMap of one Segment composition, in simple terms, ConcurrentHashMap is a Segment array, locking it to carry through inheritance ReentrantLock, through each lock a segment to Ensure the thread safety of operations in each segment, thereby achieving global thread safety.
The data structure diagram of jdk1.7 is as follows.
When each operation is distributed on different segments , by default, it can theoretically support concurrent writing of 16 threads at the same time.
The data structure diagram of jdk1.8 is as follows.
Compared with version 1.7, it has made two improvements
- Cancel the segment design, directly use the Node array to store data, and use the Node array element as
a lock to lock each row of data to further reduce the probability of concurrent conflicts - Change the data structure of the original array + singly linked list to: array + singly linked list + red-black tree structure.
Why introduce a red-black tree? Under normal circumstances, if the key hash can be evenly dispersed in the array, the length of each queue in the table array is mainly 0 or 1. But in actual situations, there will still be Some queue lengths are too long. If the one-way list method is also used, the time complexity of querying a node becomes O(n); therefore, for lists with a queue length exceeding 8 , JDK1.8 uses a red-black tree structure, so the query time is complicated The degree will be reduced to O(logN), which can improve the performance of search; this structure is basically the same as the implementation structure of Hashmap in the JDK1.8 version, but in order to ensure thread safety, the implementation of ConcurrentHashMap will be a little more complicated.
Next, we will understand its principle from the source code level. We can analyze its implementation based on put and get methods.
ConcurrentHashMap source code analysis
put method first stage
public V put(K key, V value) {
return putVal(key, value, false);
}
final V putVal(K key, V value, boolean onlyIfAbsent) {
if (key == null || value == null) throw new NullPointerException();
int hash = spread(key.hashCode());//计算hash值
int binCount = 0;//用来记录链表的长度
for (Node<K,V>[] tab = table;;) {
//自旋操作，当出现线程竞争时不断自旋
Node<K,V> f; int n, i, fh;
if (tab == null || (n = tab.length) == 0)
// 如果数组为空，则进行数组的初始化
// 通过 hash 值对应的数组下标得到第一个节点; 以 volatile 读的方式来读取
// table 数组中的元素，保证每次拿到的数据都是最新的
tab = initTable();
else if ((f = tabAt(tab, i = (n - 1) & hash)) == null) {
//如果该下标返回的节点为空，则直接通过 cas 将新的值封装成 node 插入即可;
//如果 cas 失败，说明存在竞争，则进入下一次循环
if (casTabAt(tab, i, null,
new Node<K,V>(hash, key, value, null)))
break; // no lock when adding to empty bin
}
else if ((fh = f.hash) == MOVED)
// 其他线程协助扩容
tab = helpTransfer(tab, f);
else {
// 这里是扩容操作
}
}
// 更新hashMap中的元素个数
addCount(1L, binCount);
return null;
If there are two threads in the above code, without locking: thread A successfully executes the casTabAt operation, the subsequent thread B can immediately see the change of table[i] through the tabAt method. The reason is as follows:
the casTabAt operation of thread A has the same memory semantics as volatile read and write. According to the happens-before rule of volatile: the casTabAt operation of thread A must be visible to the tabAt operation of thread B
initTable initializes the array
private final Node<K,V>[] initTable() {
Node<K,V>[] tab; int sc;
while ((tab = table) == null || tab.length == 0) {
if ((sc = sizeCtl) < 0)
//被其他线程抢占了初始化的操作,则直接让出自己的 CPU 时间片
Thread.yield(); // lost initialization race; just spin
//通过 cas 操作，将 sizeCtl 替换为-1，标识当前线程抢占到了初始化资格
else if (U.compareAndSwapInt(this, SIZECTL, sc, -1)) {
try {
if ((tab = table) == null || tab.length == 0) {
// 默认初始容量为16
int n = (sc > 0) ? sc : DEFAULT_CAPACITY;
@SuppressWarnings("unchecked")
//初始化数组，长度为16，或者初始化在构造 ConcurrentHashMap 的时候传入的长度
Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n];
table = tab = nt;//将这个数组赋值给table
//计算下次扩容的大小，实际就是当前容量的 0.75 倍，这里使用了右移来计算
sc = n - (n >>> 2);
}
} finally {
//设置sizeCtl为sc, 如果默认是16的话，那么这个时候sc=16*0.75=12
sizeCtl = sc;
}
break;
}
}
return tab;
}
Array initialization method. This method is relatively simple. It is to initialize a suitable size array
sizeCtl. This should be discussed separately. If you don’t understand the meaning of this attribute, you may be confused.
This sign is used when the Node array is initialized or expanded. Control bit identification, a negative number means initialization or expansion operation is in progress
- -1 means initializing
- -N means that there are N-1 threads undergoing expansion operations, here is not simply understood as n threads, sizeCtl is -N
- 0 indicates that the Node array has not been initialized, and a positive number represents the size of the initialization or next expansion
tabAt method
static final <K,V> Node<K,V> tabAt(Node<K,V>[] tab, int i) {
return (Node<K,V>)U.getObjectVolatile(tab, ((long)i << ASHIFT) + ABASE);
}
This method gets the value of the object field corresponding to the offset address in the object. In fact, the meaning of this code is equivalent to tab[i], but why not directly use tab[i] to calculate it?
getObjectVolatile, once you see the volatile keyword, it means visibility. Because volatile write operations happen-before to volatile read operations, the modifications to the table by other threads are visible to get reads;
although the table array itself adds volatile attributes, "volatile arrays are only volatile for array references Semantics, not its elements". So if other threads write to the elements of this array, the current thread may not be able to read the latest value when it comes to reading.
For performance reasons, Doug Lea directly manipulates the table through the Unsafe class.
The second stage of the put method (counting and expansion)
After the putVal method is executed, the number of elements in ConcurrentHashMap will be increased through addCount, and the expansion operation may also be triggered. There will be two very classic designs here
Count addCount
When putVal finally calls addCount, two parameters are passed, namely 1 and binCount (length of the linked list). See what operations are done in the addCount method.
x represents the number of elements that need to be added to the table this time, check The parameter indicates whether the expansion check is required, and the check is required if it is greater than or equal to 0
How to ensure the data security and performance of addCount
private transient volatile long baseCount; //在没有竞争的情况下,去通过cas操作更新元素个数
private transient volatile CounterCell[] counterCells;//在存在线程竞争的情况下，存储元素个数
addCount(1L, binCount);
return null;
private final void addCount(long x, int check) {
CounterCell[] as; long b, s;
// 判断 counterCells 是否为空
//1. 如果为空，就通过cas操作尝试修改baseCount变量,对这个变量进行原子累加操作
//(做这个操作的意义是:如果在没有竞争的情况下，仍然采用 baseCount 来记录元素个数
//2. 如果cas失败说明存在竞争，这个时候不能再采用 baseCount 来累加，而是通过
// CounterCell 来记录
if ((as = counterCells) != null ||
!U.compareAndSwapLong(this, BASECOUNT, b = baseCount, s = b + x)) {
CounterCell a; long v; int m;
boolean uncontended = true;//是否冲突标识，默认为没有冲突
if (as == null || (m = as.length - 1) < 0 ||
(a = as[ThreadLocalRandom.getProbe() & m]) == null ||
!(uncontended =
U.compareAndSwapLong(a, CELLVALUE, v = a.value, v + x))) {
/*这里有几个判断
1. 计数表为空则直接调用 fullAddCount
2. 从计数表中随机取出一个数组的位置为空，直接调用 fullAddCount
3. 通过 CAS 修改 CounterCell 随机位置的值，如果修改失败说明出现并发情况(这里又
用到了一种巧妙的方法)，调用fullAndCount。
Random 在线程并发的时候会有性能问题以及可能会产生相同的随机数,
ThreadLocalRandom.getProbe 可以解决这个问题，并且性能要比Random高 */
fullAddCount(x, uncontended);//执行fullAddCount方法
return;
}
if (check <= 1)//链表长度小于等于1，不需要考虑扩容
return;
s = sumCount();//统计ConcurrentHashMap元素个数
}
// 下面的逻辑是扩容操作
if (check >= 0) {
Node<K,V>[] tab, nt; int n, sc;
while (s >= (long)(sc = sizeCtl) && (tab = table) != null &&
(n = tab.length) < MAXIMUM_CAPACITY) {
int rs = resizeStamp(n);
if (sc < 0) {
if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 ||
sc == rs + MAX_RESIZERS || (nt = nextTable) == null ||
transferIndex <= 0)
break;
if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1))
transfer(tab, nt);
}
else if (U.compareAndSwapInt(this, SIZECTL, sc,
(rs << RESIZE_STAMP_SHIFT) + 2))
transfer(tab, null);
s = sumCount();
}
}
}
CounterCells explained:
ConcurrentHashMap uses the CounterCell array to record the number of elements. Like a general collection to record the size of a collection, you can directly define a member variable of size, and just update this variable when there is a change. Why should ConcurrentHashMap be handled in this form? The
problem is still concurrency. ConcurrentHashMap is a concurrent collection. If a member variable is used to count the number of elements, in order to ensure the security of shared variables under concurrent conditions, it will inevitably need to pass It can be achieved by locking or spinning. If the competition is fierce, there will be a relatively large conflict in the size setting, which will affect the performance. Therefore, the ConcurrentHashMap uses a fragmentation method to record the size. Look at the following code
private transient volatile int cellsBusy;// 标识当前 cell 数组是否在初始化或扩容中的CAS 标志位
private transient volatile CounterCell[] counterCells;// counterCells 数组，总数值的分值分别存在每个 cell 中
@sun.misc.Contended static final class CounterCell {
volatile long value;
CounterCell(long x) {
value = x; }
}
//看到这段代码就能够明白了，CounterCell 数组的每个元素，都存储一个元素个数，而实际我们调用
//size 方法就是通过这个循环累加来得到的
//又是一个设计精华，大家可以借鉴； 有了这个前提，再会过去看 addCount 这个方法，就容易理解一些了
final long sumCount() {
CounterCell[] as = counterCells; CounterCell a;
long sum = baseCount;
if (as != null) {
for (int i = 0; i < as.length; ++i) {
if ((a = as[i]) != null)
sum += a.value;
}
}
return sum;
}
Expansion
The third stage of put method (assisting in expansion)
- (When the number of elements is greater than the threshold)
- If the capacity is being expanded at this time, the thread coming in during the expansion phase will assist in the expansion
The fourth stage of the put method (convert the linked list into a red-black tree)
Determine whether the length of the linked list has reached the critical value 8. If the critical value is reached, at this time, it will be determined whether to expand or convert the linked list into a red-black tree according to the length of the current array. In other words, if the length of the current array is less than 64, it will be expanded first. Otherwise, the current linked list will be converted into a red-black tree
Next article
"The Design and Principle Analysis of Thread Pool" of Multithreaded Learning (8)