Why can't map be traversed while adding and deleting operations?

Some time ago, when my colleague scanned KW in the code, such a message appeared:

The reason for this above is that when using foreach to traverse the HashMap, there will be problems with the put assignment operation at the same time, and the exception ConcurrentModificationException.

So I took a brief look at it, and my impression is that the collection class needs to be cautious when deleting or adding operations while traversing. Generally, iterators are used for operations.

So I told my colleagues that iterators should be used to operate on the collection elements. Colleagues asked me why? Now I'm confused? Yes, I just remember whether it can be used like this, but it seems that I have never investigated why?

So today I decided to study this HashMap traversal operation carefully to prevent pit mining!

foreach loop?

The java foreach syntax is a new feature added in jdk1.5, mainly as an enhancement of the for syntax, so how is its bottom layer implemented? Let's take a closer look:

Inside the foreach syntax, the collection is implemented with an iterator iterator, and the array is implemented with subscript traversal. Java 5 and above compilers hide the internal implementation based on iteration and array subscript traversal.

(Note that what is said here is that the "Java compiler" or the Java language hides its implementation, not that a certain piece of Java code hides its implementation, that is, we find it in any piece of JDK Java code. Don’t see the hidden implementation here. The implementation here is hidden in the Java compiler, look at the bytecode compiled by a foreach Java code, and guess how it is implemented)

Let's write an example to study:

public class HashMapIteratorDemo {
	
	String[] arr = {"aa", "bb", "cc"};
	
	public void test1() {
		for(String str : arr) {
		}
	}
}
复制代码

Turn the above example into bytecode and decompile it (main function part):

Maybe we can't clearly understand what these instructions do, but we can compare the bytecode instructions generated by the following code:

public class HashMapIteratorDemo2 {
	
	String[] arr = {"aa", "bb", "cc"};
	
	public void test1() {
		for(int i = 0; i < arr.length; i++) {
			String str = arr[i];
		}
	} 
}
复制代码

Look at the two bytecode files, do you find that the instructions are almost the same, if you still have doubts, let's look at the foreach operation on the collection:

Traverse the collection via foreach:

public class HashMapIteratorDemo3 {
	
	List<Integer> list = new ArrayList<>();
	
	public void test1() {
		list.add(1);
		list.add(2);
		list.add(3);
		
		for(Integer var : list) {
		}
	}
}
复制代码

Traverse the collection through Iterator:

public class HashMapIteratorDemo4 {
	
	List<Integer> list = new ArrayList<>();
	
	public void test1() {
		list.add(1);
		list.add(2);
		list.add(3);
		
		Iterator<Integer> it = list.iterator();
		while(it.hasNext()) {
			Integer var = it.next();
		}
	}
}
复制代码

Compare the bytecodes of the two methods as follows:

We found that the bytecode instructions of the two methods operate almost exactly the same;

Thus we can draw the following conclusions:

For collections, since the collections all implement Iterator iterators, the foreach syntax is finally converted by the compiler into a call to Iterator.next();

For an array, it is transformed into a circular reference to each element in the array.

HashMap traverses the collection and removes, puts, and adds the elements of the collection

1. Phenomenon

According to the above analysis, we know that the underlying layer of HashMap implements the Iterator iterator, so in theory we can also use the iterator to traverse, which is true, for example as follows:

public class HashMapIteratorDemo5 {
	
	public static void main(String[] args) {
		Map<Integer, String> map = new HashMap<>();
		map.put(1, "aa");
		map.put(2, "bb");
		map.put(3, "cc");
		
		for(Map.Entry<Integer, String> entry : map.entrySet()){  
		    int k=entry.getKey();  
		    String v=entry.getValue();  
		    System.out.println(k+" = "+v);  
		}  
	} 
}
复制代码

output:

Ok, there is no problem with traversal, so how about removing, putting, and adding elements of the collection?

public class HashMapIteratorDemo5 {
	
	public static void main(String[] args) {
		Map<Integer, String> map = new HashMap<>();
		map.put(1, "aa");
		map.put(2, "bb");
		map.put(3, "cc");
		
		for(Map.Entry<Integer, String> entry : map.entrySet()){  
		    int k=entry.getKey();  
		    if(k == 1) {
		    	map.put(1, "AA");
		    }
		    String v=entry.getValue();  
		    System.out.println(k+" = "+v);  
		}  
	} 
}
复制代码

Results of the:

There is no problem with the execution, and the put operation is also successful.

but! but! but! Here comes the problem! ! !

We know that HashMap is a thread-unsafe collection class. If you use foreach to traverse, add and remove operations will cause java.util.ConcurrentModificationException. The put operation may throw this exception. (Why it is possible, we will explain this later)

Why is this exception thrown?

Let's take a look at the explanation of the HasMap operation in the java api document.

The translation roughly means that this method returns a collection view of the keys contained in this map. Collections are backed by maps, and if the map is modified while iterating over the collection (other than through the iterator's own remove operations), the results of the iteration are undefined. Collections support element removal, which removes the corresponding mapping from the map via the Iterator.remove, set.remove, removeAll, retainal, and clear operations. To put it simply, when traversing a collection through map.entrySet(), operations such as remove and add cannot be performed on the collection itself, and iterators need to be used for operations.

For the put operation, if the replacement operation modifies the first element as in the above example, no exception will be thrown, but if it is an operation to add elements using put, an exception will definitely be thrown. Let's modify the example above:

public class HashMapIteratorDemo5 {
	
	public static void main(String[] args) {
		Map<Integer, String> map = new HashMap<>();
		map.put(1, "aa");
		map.put(2, "bb");
		map.put(3, "cc");
		
		for(Map.Entry<Integer, String> entry : map.entrySet()){  
		    int k=entry.getKey();  
		    if(k == 1) {
		    	map.put(4, "AA");
		    }
		    String v=entry.getValue();  
		    System.out.println(k+" = "+v);  
		}  
 
	} 
}
复制代码

An exception occurred during execution:

This is to verify that the put operation mentioned above may throw a java.util.ConcurrentModificationException exception.

But I have doubts. We said above that the foreach loop is traversal through iterators? Why is it impossible to come here?

This is actually very simple, the reason is that the bottom layer of our traversal operation is indeed performed through an iterator, but our remove and other operations are performed by directly operating the map, as in the above example: map.put(4, "AA");// The actual operation here is directly on the collection, rather than through the iterator. So there will still be ConcurrentModificationException exceptions.

2. Study the underlying principles in detail

Let's look at the source code of HashMap again. Through the source code, we find that this method is used when the collection is traversed using Iterator:

final Node<K,V> nextNode() {
            Node<K,V>[] t;
            Node<K,V> e = next;
            if (modCount != expectedModCount)
                throw new ConcurrentModificationException();
            if (e == null)
                throw new NoSuchElementException();
            if ((next = (current = e).next) == null && (t = table) != null) {
                do {} while (index < t.length && (next = t[index++]) == null);
            }
            return e;
        }
复制代码

Here modCount indicates how many times the elements in the map have been modified (this value will be self-increased when removing and adding new elements), and expectedModCount indicates the expected number of modifications, and these two values ​​​​are equal when the iterator is constructed , if the two values ​​are out of sync during the traversal process, a ConcurrentModificationException will be thrown.

Now let's look at the collection remove operation:

(1) The remove implementation of HashMap itself:

public V remove(Object key) {
    Node<K,V> e;
    return (e = removeNode(hash(key), key, null, false, true)) == null ?
        null : e.value;
}
复制代码

(2) The remove implementation of HashMap.KeySet

public final boolean remove(Object key) {
    return removeNode(hash(key), key, null, false, true) != null;
}
复制代码

(3) The remove implementation of HashMap.EntrySet

public final boolean remove(Object o) {
    if (o instanceof Map.Entry) {
        Map.Entry<?,?> e = (Map.Entry<?,?>) o;
        Object key = e.getKey();
        Object value = e.getValue();
        return removeNode(hash(key), key, value, true, true) != null;
    }
    return false;
}
复制代码

(4) Implementation of the remove method of HashMap.HashIterator

public final void remove() {
    Node<K,V> p = current;
    if (p == null)
        throw new IllegalStateException();
    if (modCount != expectedModCount)
        throw new ConcurrentModificationException();
    current = null;
    K key = p.key;
    removeNode(hash(key), key, null, false, false);
    expectedModCount = modCount; //----------------这里将expectedModCount 与modCount进行同步
}
复制代码

The above four methods all implement the operation of deleting the key by calling the HashMap.removeNode method. As long as the key is removed in the removeNode method, modCount will perform an auto-increment operation, and modCount will be inconsistent with expectedModCount at this time;

final Node<K,V> removeNode(int hash, Object key, Object value,
                           boolean matchValue, boolean movable) {
    Node<K,V>[] tab; Node<K,V> p; int n, index;
    if ((tab = table) != null && (n = tab.length) > 0 &&
        ...
        if (node != null && (!matchValue || (v = node.value) == value ||
                             (value != null && value.equals(v)))) {
            if (node instanceof TreeNode)
                ((TreeNode<K,V>)node).removeTreeNode(this, tab, movable);
            else if (node == p)
                tab[index] = node.next;
            else
                p.next = node.next;
            ++modCount;   //------------------------这里对modCount进行了自增，可能会导致后面与expectedModCount不一致
            --size;
            afterNodeRemoval(node);
            return node;
        }
        }
        return null;
   }
复制代码

Among the above three remove implementations, only the remove method of the third iterator synchronizes the expectedModCount value to be the same as modCount after calling the removeNode method, so when traversing the next element and calling the nextNode method, the iterator method will not throw an exception.

Is there a feeling of sudden understanding here?

Therefore, if you need to perform element operations when traversing the collection, you need to use the Iterator iterator, as follows:

public class HashMapIteratorDemo5 {
	
	public static void main(String[] args) {
		Map<Integer, String> map = new HashMap<>();
		map.put(1, "aa");
		map.put(2, "bb");
		map.put(3, "cc");
		//		for(Map.Entry<Integer, String> entry : map.entrySet()){  //		    int k=entry.getKey();  //		    //		    if(k == 1) {//		    	map.put(1, "AA");//		    }//		    String v=entry.getValue();  //		    System.out.println(k+" = "+v);  //		}  
		
		Iterator<Map.Entry<Integer, String>> it = map.entrySet().iterator();
		while(it.hasNext()){
			Map.Entry<Integer, String> entry = it.next();
			int key=entry.getKey();  
	        if(key == 1){  
	            it.remove();        
	        }  
		}
	}
}