This little "pit" of HashMap's disorder caused it to overturn accidentally.

Hello everyone, I am Yihang;

Yesterday, a fan friend chatted with me and said that he encountered an inexplicable problem and asked me to help analyze it; after a round of description, he found that it was a small 'hole' in the order of HashMap elements ; but if he was not careful, the old driver It’s also easy to overturn here.

To describe his problem in one sentence: Obviously my database statements are sorted using Order by, and I can see in the log that the data is found in order, but the data received by the business layer is still out of order? ;

Throughout the whole process, there were indeed several confusions that affected his judgment of the problem; let's take a small case and source code analysis to see what happened.

Problem recurrence

In order to facilitate the explanation of the problem, a simple business scenario is used to simulate it here;

data sheet

A simple student information sheet

need

It is necessary to obtain the student information corresponding to the student number (student_id) in order by id; in order to facilitate the business level to obtain data based on the student number, this friend adopted the following data format and used HashMap to receive the data in the business code :

{
    
    
	"S000001": {
    
    
		"name": "张三",
		"student_id": "S000001",
		"age": 7
	},
	"S000002": {
    
    
		"name": "李四",
		"student_id": "S000002",
		"age": 7
	},
	"S000003": {
    
    
		"name": "王五",
		"student_id": "S000003",
		"age": 8
	},
	"S000004": {
    
    
		"name": "赵六",
		"student_id": "S000004",
		"age": 8
	}
}

Code

Dao

@MapKey("student_id")
HashMap<String, Object> getStudentId();

xml

<select id="getStudentMap" resultType="java.util.HashMap">
    select student_id , `name` , age
    from student_info
    order by id
</select>

test case

@Autowired
StudentInfoMapper studentInfoMapper;

@Test
void map() {
      
      
    HashMap<String, Object> studentId = studentInfoMapper.getStudentMap();
    studentId.forEach((id, studentInfo) -> log.info("学号：{} 信息：{}", id, JSON.toJSONString(studentInfo)));
}

Problems

Execution log of the above code;

When the student number is the following data and then queried, there is no problem in querying and using the entire data .

==>  Preparing: select student_id , `name` , age from student_info order by id
==> Parameters: 
<==    Columns: student_id, name, age
<==        Row: S000001, 张三, 7
<==        Row: S000002, 李四, 7
<==        Row: S000003, 王五, 8
<==        Row: S000004, 赵六, 8
<==      Total: 4
Closing non transactional SqlSession 
StudentTest    : 学号：S000001 信息：{
    
    "name":"张三","student_id":"S000001","age":7}
StudentTest    : 学号：S000002 信息：{
    
    "name":"李四","student_id":"S000002","age":7}
StudentTest    : 学号：S000003 信息：{
    
    "name":"王五","student_id":"S000003","age":8}
StudentTest    : 学号：S000004 信息：{
    
    "name":"赵六","student_id":"S000004","age":8}

But after changing the student number to the following data:

According to the query results and output logs, we can see that the data queried from the database is in order ; but the data returned to the HashMap of the business layer becomes unordered ;

==>  Preparing: select student_id , `name` , age from student_info order by id
==> Parameters: 
<==    Columns: student_id, name, age
<==        Row: S000001, 张三, 7
<==        Row: K000002, 李四, 7
<==        Row: M000003, 王五, 8
<==        Row: N000004, 赵六, 8
<==      Total: 4
Closing non transactional SqlSession 
StudentTest    : 学号：M000003 信息：{
    
    "name":"王五","student_id":"M000003","age":8}
StudentTest    : 学号：K000002 信息：{
    
    "name":"李四","student_id":"K000002","age":7}
StudentTest    : 学号：N000004 信息：{
    
    "name":"赵六","student_id":"N000004","age":8}
StudentTest    : 学号：S000001 信息：{
    
    "name":"张三","student_id":"S000001","age":7}

Obviously the previous query showed that it was normal, but why did it become ? The reason is just that the previous batch of data happened to be correct . Don't be fooled by the appearance. In fact, it is wrong to return both sets of data in this way.

In actual business bugs, these confusing data (user A's access is normal, user B's access is abnormal) often lead to doubts about life...

problem analysis

Since the database log obtains data sequentially, it means that there is no problem at the SQL level. The problem must lie in HashMap. Anyone who knows HashMap knows that HashMap itself is disordered , but what puzzles many novice friends the most is Yes, it doesn't matter if you are out of order, but when I insert it, I insert it in the order I need. Isn't this okay? If you are confused about this point, it means that you have not understood the meaning of disorder; there is no relationship between the insertion order of HashMap and the iterative removal order;

Unless you know all the keys during insertion and save them when getting them, you can get the keys in this order; but in the actual use process, this is rarely the case;

Simple insert and get examples:

public static void t1() {
    
    
    HashMap<String, Object> map = new HashMap<>();
    for (int i = 0; i < 5; i++) {
    
    
        String k = "key:" + i;
        String v = "va" + i;
        map.put(k, v);
        log.info("顺序插入 key:{} <-- va:{}", k, v);
    }
    log.info("-----------------------------");
    map.forEach((k, v) -> log.info("迭代获取 key:{} --> va:{}", k, v));
}

Execution log:

Main - 顺序插入 key:key:0 <-- va:va0
Main - 顺序插入 key:key:1 <-- va:va1
Main - 顺序插入 key:key:2 <-- va:va2
Main - 顺序插入 key:key:3 <-- va:va3
Main - 顺序插入 key:key:4 <-- va:va4
Main - -----------------------------
Main - 迭代获取 key:key:2 --> va:va2
Main - 迭代获取 key:key:1 --> va:va1
Main - 迭代获取 key:key:0 --> va:va0
Main - 迭代获取 key:key:4 --> va:va4
Main - 迭代获取 key:key:3 --> va:va3

Source code cause analysis

Next, let’s talk in detail about the disorder of HashMap from the perspective of source code!

HashMap data structure

The data structure of Java8 HashMap is as shown below; it adopts the method of array + joint table + red-black tree ; therefore, the subscript position of the array (yellow area) where the element is located is (n - 1) & hash])located by, when the hash values of different keys appear. At the same time, the value with the same subscript will be stored in the form of 联表or ;红黑树

HashMap insertion process

Locate the array subscript based on the AND operation of the key's hash value and the array length.
```
if ((p = tab[i = (n - 1) & hash]) == null)
    tab[i] = newNode(hash, key, value, null);
else {
      
      
	//...
}
```
Key code (n - 1) & hash]), where n is the length of the array, and the value is obtained hashthrough operationshash(key)
When the value of tab[i] subscript i is null, it is directly instantiated and placed at the subscript position.

That is, the one above
```
if ((p = tab[i = (n - 1) & hash]) == null)
    tab[i] = newNode(hash, key, value, null);
```

If a hash conflict occurs and tab[i] already has a value, it will follow the Node in a linked list.

The key exists in the joint table

Modify value

if ((e = p.next) == null)When the next

Just instantiate a new Node and follow

// ....
else {
        
        
    for (int binCount = 0; ; ++binCount) {
        
        
        if ((e = p.next) == null) {
        
        
            p.next = newNode(hash, key, value, null);
            if (binCount >= TREEIFY_THRESHOLD - 1) // -1 for 1st
                treeifyBin(tab, hash);
            break;
        }
        if (e.hash == hash &&
            ((k = e.key) == key || (key != null && key.equals(k))))
            break;
        p = e;
    }
}

If the length of the joint table is greater than 8, it will be converted into a red-black tree structure for storage.

After the above four steps, the elements are not stored in order, but are scattered under each subscript of the array; the element positions of the linked list or red-black tree have no fixed order; the keys of the same hash are inserted at different times and in different locations. The location is also different;

Sample data saving analysis

(n - 1) & hash])With the support of source code analysis, let's take a detailed look at the subscript positions of keys and values stored in hashMap after calculation :

key	value	Array subscript i = (n - 1) & hash])
key:0	wa0	6
key:1	va1	5
key:2	va2	4
key:3	va3	11
key:4	va4	10

Final key and value distribution:

subscript	4	5	6	10	11
key	key:2	key:1	key:0	key:4	key:3
value 1	va2	va1	wa0	va4	va3
Value 2

Iterative acquisition

The iterative acquisition of HashMap elements is to first traverse the array from left to right. When the array index position has a value, then traverse the joint table or red-black tree from top to bottom;

The source code is as follows:

@Override
public void forEach(BiConsumer<? super K, ? super V> action) {
    
    
    Node<K,V>[] tab;
    if (action == null)
        throw new NullPointerException();
    if (size > 0 && (tab = table) != null) {
    
    
        int mc = modCount;
        for (int i = 0; i < tab.length; ++i) {
    
    
            for (Node<K,V> e = tab[i]; e != null; e = e.next)
                action.accept(e.key, e.value);
        }
        if (modCount != mc)
            throw new ConcurrentModificationException();
    }
}

core code

First loop for (int i = 0; i < tab.length; ++i): traverse the array from left to right

The second loop for (Node<K,V> e = tab[i]; e != null; e = e.next): traverse the joint table or red-black tree from top to bottom

for (int i = 0; i < tab.length; ++i) {
    
    
    for (Node<K,V> e = tab[i]; e != null; e = e.next)
        action.accept(e.key, e.value);
}

Looking at the previous key and value distribution table, and looking back and forth at the output log, you can understand why the iterative output is as follows:

Main - 获取 key:key:2 --> va:va2
Main - 获取 key:key:1 --> va:va1
Main - 获取 key:key:0 --> va:va0
Main - 获取 key:key:4 --> va:va4
Main - 获取 key:key:3 --> va:va3

How to solve orderly problems

When you need to ensure that the insertion order and acquisition order are consistent, you can use an ordered data structure to save the data, such as List, LinkedHashMap, etc...

MyBatis data query

For example, the example listed at the beginning; when the data query needs to be returned in order, you can change the method and use List to receive data; if the business really needs to participate through Map, you can reconstruct the ordered data structure of a LinkedHashMap through conversion. Based on the needs of business logic;

Examples are as follows:

dao
```
List<StudentInfo> getStudentList();
```

xml

<select id="getStudentList" resultType="com.ehang.springboot.mybatisplus.generator.student.demain.StudentInfo">
    select student_id , `name` , age
    from student_info
    order by id
</select>

test

@Autowired
StudentInfoMapper studentInfoMapper;

@Test
void list() {
      
      
    List<StudentInfo> studentList = studentInfoMapper.getStudentList();
    studentList.forEach(studentInfo -> log.info("{}", JSON.toJSONString(studentInfo)));
}

test output

==>  Preparing: select student_id , `name` , age from student_info order by id
==> Parameters: 
<==    Columns: student_id, name, age
<==        Row: S000001, 张三, 7
<==        Row: S000002, 李四, 7
<==        Row: S000003, 王五, 8
<==        Row: S000004, 赵六, 8
<==      Total: 4
Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@ae5fa7]
StudentTest    : {
      
      "age":7,"name":"张三","studentId":"S000001"}
StudentTest    : {
      
      "age":7,"name":"李四","studentId":"S000002"}
StudentTest    : {
      
      "age":8,"name":"王五","studentId":"S000003"}
StudentTest    : {
      
      "age":8,"name":"赵六","studentId":"S000004"}

LinkedHashMap

Introduction

When the Map needs to be ordered, you only need to replace HashMap with LinkedHashMap to ensure that the order of insertion and removal is consistent;

LinkedHashMap is a subclass of HashMap

public class LinkedHashMap<K,V>
    extends HashMap<K,V>
    implements Map<K,V>
{
      
      
	//....
}

Therefore, LinkedHashMap has all the functions of HashMap; but LinkedHashMap uses a two-way joint table to save data; therefore, it can ensure the consistency of data saving order and reading order.

The following is the source code for reading data; it can be seen that the entire joint table is traversed in an orderly manner;

public void forEach(BiConsumer<? super K, ? super V> action) {
      
      
    if (action == null)
        throw new NullPointerException();
    int mc = modCount;
    for (LinkedHashMap.Entry<K,V> e = head; e != null; e = e.after)
        action.accept(e.key, e.value);
    if (modCount != mc)
        throw new ConcurrentModificationException();
}

Test example modification:

public static void t1() {
      
      
    HashMap<String, Object> map = new LinkedHashMap<>();
    for (int i = 0; i < 5; i++) {
      
      
        String k = "key:" + i;
        String v = "va" + i;
        map.put(k, v);
        log.info("插入 key:{} <-- va:{}", k, v);
    }
    log.info("-----------------------------");
    map.forEach((k, v) -> log.info("获取 key:{} --> va:{}", k, v));
}

Output:

Main - 插入 key:key:0 <-- va:va0
Main - 插入 key:key:1 <-- va:va1
Main - 插入 key:key:2 <-- va:va2
Main - 插入 key:key:3 <-- va:va3
Main - 插入 key:key:4 <-- va:va4
Main - -----------------------------
Main - 获取 key:key:0 --> va:va0
Main - 获取 key:key:1 --> va:va1
Main - 获取 key:key:2 --> va:va2
Main - 获取 key:key:3 --> va:va3
Main - 获取 key:key:4 --> va:va4

Summarize

Each data structure has its own unique characteristics; therefore, in terms of basic knowledge, you must clearly understand the principles of the differences. Only in this way can you accurately analyze the essence of the problem when you encounter it. Otherwise, it is easy to be confused by the appearance and the log and fall into confusion;

Okay, that’s it for today’s sharing. If you don’t mind, Sanlian can arrange it for you! Grateful!

Scan the QR code to join the group and share learning materials every day; waiting for you! ! !
Insert image description here

Insert image description here