数据结构与算法之美总结（数组、链表、栈、队列、递归、排序及二分）

---

title: 数据结构与算法之美总结（数组、链表、栈、队列、递归、排序及二分）
date: 2023-04-15 01:41:26
tags:

• 数据结构
• 算法
  categories:
• 数据结构与算法
  cover: https://cover.png
  feature: false

---

1. 前言

1、什么是数据结构？什么是算法？

• 从广义上讲，数据结构就是指一组数据的存储结构。算法就是操作数据的一组方法
• 从狭义上讲，是指某些著名的数据结构和算法，比如队列、栈、堆、二分查找、动态规划等。这些都是前人智慧的结晶，可以直接拿来用。这些经典数据结构和算法，都是前人从很多实际操作场景中抽象出来的，经过非常多的求证和检验，可以高效地帮助我们解决很多实际的开发问题

2、数据结构和算法有什么关系呢？

数据结构是为算法服务的，算法要作用在特定的数据结构之上。 因此，我们无法孤立数据结构来讲算法，也无法孤立算法来讲数据结构

比如，因为数组具有随机访问的特点，常用的二分查找算法需要用数组来存储数据。但如果选择链表这种数据结构，二分查找算法就无法工作了，因为链表并不支持随机访问

数据结构是静态的，它只是组织数据的一种方式。如果不在它的基础上操作、构建算法，孤立存在的数据结构就是没用的

大部分数据结构和算法知识点如下图所示：

这里面有

• 10 个数据结构：数组、链表、栈、队列、散列表、二叉树、堆、跳表、图、Trie 树
• 10 个算法：递归、排序、二分查找、搜索、哈希算法、贪心算法、分治算法、回溯算法、动态规划、字符串匹配算法

掌握了这些基础的数据结构和算法，再学更加复杂的数据结构和算法，就会非常容易、非常快

2. 复杂度分析

The data structure and algorithm itself solve the problem of "fast" and "saving", that is, how to make the code run faster and how to make the code save storage space. Therefore, execution efficiency is a very important consideration for the algorithm. So how to measure the execution efficiency of the algorithm code you write? Time and space complexity analysis is used here

2.1 Why is complexity analysis needed?

You may be a little confused, run the code once, and through statistics and monitoring, you can get the execution time of the algorithm and the memory size it occupies. Why do time and space complexity analysis? Can this analysis method be more accurate than the data obtained by actually running it again?

First of all, it is safe to say that this method of evaluating the efficiency of algorithm execution is correct. Many data structure and algorithm books also gave this method a name, called post hoc statistics . However, this statistical method has very large limitations

1. The test results are very dependent on the test environment

The difference in hardware in the test environment will have a great impact on the test results. For example, take the same piece of code and run it with an Intel Core i9 processor and an Intel Core i3 processor. Needless to say, the i9 processor executes much faster than the i3 processor. Also, for example, the execution speed of code a on this machine is faster than that of code b, and when it is switched to another machine, there may be completely opposite results

2. The test results are greatly affected by the size of the data

For the same sorting algorithm, the ordering degree of the data to be sorted is different, and the execution time of sorting will be very different. In extreme cases, if the data is already in order, the sorting algorithm does not need to do anything, and the execution time will be very short. In addition, if the test data scale is too small, the test results may not truly reflect the performance of the algorithm. For example, for small-scale data sorting, insertion sorting may actually be faster than quick sorting!

Therefore, we need a method that can roughly estimate the execution efficiency of the algorithm without using specific test data to test . This is the time and space complexity analysis method

2.2 Big O complexity notation

The execution efficiency of an algorithm, roughly speaking, is the execution time of the algorithm code. But, how to get the execution time of a piece of code with "naked eyes" without running the code?

Here is a very simple piece of code to find $1,2,3,4...\; The\; cumulative\; sum\; ofn$ , now, let’s estimate the execution time of this code

int cal(int n) {
    
    
    int sum = 0;
    int i = 1;
    for (; i <= n; ++i) {
    
    
        sum = sum + i;
    }
    return sum;
}

From the perspective of the CPU, each line of this code performs a similar operation: read data-operate-write data. Although the number of CPU executions and execution time corresponding to each line of code are different, this is only a rough estimate, so it can be assumed that the execution time of each line of code is the same, which is unit_time. Based on this assumption, what is the total execution time of this code?

The 2nd and 3rd lines of code require 1 unit_time execution time respectively, and the 4th and 5th lines are run n times, so it takes $2n\ast unit\_time$ execution time, so the total execution time of this code is$(2n+2)\ast unit\_time$ . It can be seen that the execution time of all codes is$T_{\left(n\right)}$Proportional to the number of executions per line of code

According to this analysis idea, look at this code again

int cal(int n) {
    
    
    int sum = 0;
    int i = 1;
    int j = 1;
    for (; i <= n; ++i) {
    
    
        j = 1;
        for (; j <= n; ++j) {
    
    
            sum = sum + i * j;
        }
    }
}

It is still assumed that the execution time of each statement is unit_time. Then the total execution time of this code $T_{\left(n\right)}$How much is it?

The 2nd, 3rd, and 4th lines of code each require 1 unit_time to execute, and the 5th and 6th lines of code are executed n times, requiring $2n\ast The\; execution\; time\; of\; unit\_time$ , the 7th and 8th lines of code loop executed$n^2$ times, so need$2n^2\ast unit\_timeexecutiontime$ . Therefore, the total execution time of the entire code$T_{\left(n\right)}=(2n^2+2n+3)\ast unit\_time$

Although the specific value of unit_time is not known, a very important rule can be obtained through the derivation process of the execution time of these two codes, that is, the execution time T(n) of all codes is proportional to the execution times n of each line of code

Summarize this law into a formula: $T_{\left(n\right)}=O(f_{\left(n\right)})$

Among them, $T_{\left(n \right)} $ represents the time of code execution; n represents the size of the data scale; f ( n $f_{\left(n\right)}$Indicates the sum of the number of times each line of code is executed. Since this is a formula, use $f_{\left(n\right)}$To represent. O in the formula means the execution time of the code $T_{\left(n \right)} $ and $f_{\left(n\right)}$The expression is proportional to

So, T ( n ) = O ( 2 n + 2 ) T_{\left(n \right)} = O{(2n+2)} in the first example$T_{\left(n\right)}=O(2n+2)$ , T ( n ) = O ( 2 n 2 + 2 n + 3 ) T_{\left(n \right)} = O{(2n^2 + 2n + 3)} in the secondexample$T_{\left(n\right)}=O(2n^2+2n+3)$ . This is Big O time complexity notation. Big O time complexity does not actually specifically represent the real execution time of the code, but represents the change trend of code execution time with the growth of data scale, so it is also called asymptotic time complexity, or time complexity for short

When n is large, you can think of it as 10000, 100000. However, the low-level, constant, and coefficient parts in the formula do not affect the growth trend, so they can be ignored. You only need to record a maximum magnitude. If you use big O notation to express the time complexity of the two pieces of code just mentioned, it can be recorded as: T ( n ) = O ( n ) $T_{\left(n\right)}=O(n)$； $T_{\left(n\right)}=O(n^2)$

2.3 Time Complexity Analysis

The origin and representation of Big O time complexity were introduced earlier. Now let's see, how to analyze the time complexity of a piece of code?

1. Only focus on the piece of code with the most loop execution times

The complexity representation method of Big O is only a trend of change. Usually, the constants, low order, and coefficients in the formula are ignored, and only the magnitude of the largest order needs to be recorded. Therefore, when analyzing the time complexity of an algorithm or a piece of code, only pay attention to the piece of code with the most loop execution times. The magnitude of n in the number of times this core code is executed is the time complexity of the entire code to be analyzed

Like the previous example:

int cal(int n) {
    
    
    int sum = 0;
    int i = 1;
    for (; i <= n; ++i) {
    
    
        sum = sum + i;
    }
    return sum;
}

The second and third lines of code are constant-level execution time, which has nothing to do with the size of n, so it has no effect on the complexity. The codes on the 4th and 5th lines are the ones with the most loop execution times, so this piece of code should be analyzed emphatically. These two lines of code are executed n times, so the total time complexity is $O\left(n\right)$

2. Addition rule: the total complexity is equal to the complexity of the code with the largest magnitude

For example:

int cal(int n) {
    
    
    int sum_1 = 0;
    int p = 1;
    for (; p < 100; ++p) {
    
    
        sum_1 = sum_1 + p;
    }
    int sum_2 = 0;
    int q = 1;
    for (; q < n; ++q) {
    
    
        sum_2 = sum_2 + q;
    }
    int sum_3 = 0;
    int i = 1;
    int j = 1;
    for (; i <= n; ++i) {
    
    
        j = 1;
        for (; j <= n; ++j) {
    
    
            sum_3 = sum_3 + i * j;
        }
    }
    return sum_1 + sum_2 + sum_3;
}

This code is divided into three parts, namely sum_1, sum_2, sum_3. You can analyze the time complexity of each part separately, then put them together, and then take the one with the largest magnitude as the complexity of the entire code

What is the time complexity of the first period? This code loop executes 100 times, so it is a constant execution time, which has nothing to do with the size of n

这里再强调一下，即便这段代码循环 10000 次、100000 次，只要是一个已知的数，跟 n 无关，照样也是常量级的执行时间。当 n 无限大的时候，就可以忽略。尽管对代码的执行时间会有很大影响，但是回到时间复杂度的概念来说，它表示的是一个算法执行效率与数据规模增长的变化趋势，所以不管常量的执行时间多大，都可以忽略掉。因为它本身对增长趋势并没有影响

那第二段代码和第三段代码的时间复杂度是多少呢？答案是 $O(n)$ 和 $O(n^2)$，这应该很容易就能分析出来，综合这三段代码的时间复杂度，取其中最大的量级。所以，整段代码的时间复杂度就为 $O(n)$。也就是说：总的时间复杂度就等于量级最大的那段代码的时间复杂度。那将这个规律抽象成公式就是：

如果 $T{1}_{\left(n\right)}=O(f_{\left(n\right)})$，$T2{\_}_{\left(n\right)}=O(g_{\left(n\right)})$。那么 $T_{\left(n\right)}=T{1}_{\left(n\right)}+T2{\_}_{\left(n\right)}=max(O(f_{\left(n\right)}),O(g_{\left(n\right)})=O(max(O(f_{\left(n\right)}),O(g_{\left(n\right)})))$

3. Multiplication rule: the complexity of nested code is equal to the product of the complexity of codes inside and outside the nest

Analogous to the addition rule above, if $T{1}_{\left(n\right)}=O(f_{\left(n\right)})$，$T2{\_}_{\left(n\right)}=O(g_{\left(n\right)})$。那么 $T_{\left(n\right)}=T{1}_{\left(n\right)}\ast T2{\_}_{\left(n\right)}=O(f_{\left(n\right)})\ast O\left(g_{\left(n\right)}\right)=O(f_{\left(n\right)}\ast g_{\left(n\right)})$

That is, suppose if $T{1}_{\left(n\right)}=O(n)$，$T2{\_}_{\left(n\right)}=O(n^2)$，则 $T{1}_{\left(n\right)}\ast T2{\_}_{\left(n\right)}=O(n^3)$

To implement the specific code, the multiplication rule can be regarded as a nested loop, as in the following example:

int cal(int n) {
    
    
    int ret = 0;
    int i = 1;
    for (; i < n; ++i) {
    
    
        ret = ret + f(i);
    }
}
int f(int n) {
    
    
    int sum = 0;
    int i = 1;
    for (; i < n; ++i) {
    
    
        sum = sum + i;
    }
    return sum;
}

See cal()function alone. Assuming that f()is just an ordinary operation, the time complexity of lines 4~6 is, $T{1}_{\left(n\right)}=O\left(n\right)$ . Butf()the function itself is not a simple operation, its time complexity is$T2{\_}_{\left(n\right)}=O\left(n\right)$ , so,cal()the time complexity of the whole function is,$T{1}_{\left(n\right)}\ast T2{\_}_{\left(n\right)}=O\left(n\right)\ast O\left(n\right)=O(n^2)$

2.4 Analysis of Several Common Time Complexity Examples

Common complexity scales are as follows:

• Constant order: $O(1)$
• Logarithmic order: $O(logn)$
• Linear order: $O\left(n\right)$
• Linear-log order: $O(nlogn)$
• Square order: $O(n^2)$
• Cubic order: $O(n^3)$
• kth order: $O(n^k)$
• Exponential order: $O(2^n)$
• Factorial step: $O(n!)$

For the complex scale listed above, it can be roughly divided into two categories, polynomial scale and non-polynomial scale. Among them, there are only two non-polynomial magnitudes: $O\left(2^n\right)$sum$O(n!)$

When the data size n becomes larger, the execution time of non-polynomial-level algorithms will increase sharply, and the execution time of solving problems will increase infinitely. Therefore, algorithms with non-polynomial time complexity are actually very inefficient algorithms

1、$O(1)$

First, a concept must be clarified, $O\left(1\right)$ is just a representation of constant-level time complexity, not that only one line of code is executed. For example, this code, even with 3 lines, its time complexity is$O\left(1\right)$ instead of$O(3)$

int i = 8;
int j = 6;
int sum = i + j;

As long as the execution time of the code does not increase with the increase of n, the time complexity of the code is recorded as $O\left(1\right)$ . In other words, in general, as long as there are no loop statements or recursive statements in the algorithm, even if there are thousands of lines of code, the time complexity is$O(1)$

2、$O\left(logn\right)$、$O(nlogn)$

Logarithmic time complexity is very common, and it is also the most difficult time complexity to analyze, as in the following example:

i = 1;
while (i <= n) {
    
    
    i = i * 2;
}

According to the complexity analysis method mentioned above, the third line of code is the most frequently executed loop. Therefore, as long as you can calculate how many times this line of code is executed, you can know the time complexity of the entire code

It can be seen from the code that the value of the variable i starts from 1 and is multiplied by 2 every cycle. When greater than n, the loop ends. In fact, the value of the variable i is a geometric sequence. If you list them one by one, it should look like this: $2^0{2}^1{2}^2...2^x=n$

So, as long as you know the value of x, you know the number of times this line of code is executed. by $2^x=n$ Solving x This problem should be learned in high school,$x=lo{g}_{2}n$ , so the time complexity of this code is$O(lo{g}_{2}n)$

Now, change the code a little bit and see, what is the time complexity of this code?

i = 1;
while (i <= n) {
    
    
    i = i * 3;
}

According to the idea just mentioned, it is easy to see that the time complexity of this code is $O(lo{g}_{3}n)$

In fact, regardless of base 2, base 3, or base 10, the time complexity of all logarithmic orders can be recorded as $O\left(logn\right)$ .$\_$$\_$why?

Logarithms can be converted to each other, according to the formula:

$$lo{g}_{a}b=\frac{lo{g}_{c}b}{lo{g}_{c}a}$$

Available $lo{g}_{2}n=\frac{lo{g}_{3}n}{lo{g}_{3}2}$. So $lo{g}_{3}n$ is equal to$lo{g}_{3}2\ast lo{g}_{2}n$ , because$O(lo{g}_{3}n)=O(C\ast lo{g}_{2}n)$ , where$C=lo{g}_{3}2$ is a constant. Based on a previous theory: When using big O to mark the complexity, the coefficient can be ignored, that is,$O(C{f}_{\left(n\right)})=O(f_{\left(n\right)})$ . So,$O\left(lo{g}_{3}n\right)$ is equal to$O\left(lo{g}_{2}n\right)$ . Therefore, in the expression method of logarithmic time complexity, the "base" of logarithm is ignored, and it is uniformly expressed as$O(logn)$

If you understand the O ( logn ) O(log n) mentioned earlier$O\left(logn\right)$ , N$O\left(nlogn\right)is$ easy to understand$.$If the time complexity of a piece of code is$O\left(logn\right)$ , execute the loop n times, the time complexity is$O\left(nlogn\right)too$ .$\_$Also,$O(nlogn)\; is\; also\; a\; very\; common\; algorithmtime$ complexity$.$For example, the time complexity of merge sort and quick sort is$O(nlogn)$

3、$O(m+n)$、$O(m\ast n)$

Let’s talk about a time complexity that is different from the previous one. The complexity of the code is determined by the size of the two data, as shown in the following example:

int cal(int m, int n) {
    
    
    int sum_1 = 0;
    int i = 1;
    for (; i < m; ++i) {
    
    
        sum_1 = sum_1 + i;
    }
    int sum_2 = 0;
    int j = 1;
    for (; j < n; ++j) {
    
    
        sum_2 = sum_2 + j;
    }
    return sum_1 + sum_2;
}

As can be seen from the code, m and n represent two data scales. It is impossible to evaluate in advance which of m and n has a larger magnitude, so when expressing complexity, one cannot simply use the addition rule and omit one of them. So, the time complexity of the above code is $O(m+n)$

In this case, the original addition rule is incorrect, and the addition rule needs to be changed to: $T{1}_{\left(m\right)}+T2{\_}_{\left(n\right)}=O(f_{\left(n\right)}+g_{\left(n\right)})$ . But the multiplication rule continues:$T{1}_{\left(m\right)}\ast T2{\_}_{\left(n\right)}=O(f_{\left(n\right)}\ast g_{\left(n\right)})$

2.5 Space Complexity Analysis

After understanding the time complexity analysis mentioned above, the space complexity analysis method is very simple to learn

The full name of time complexity is asymptotic time complexity, which means the growth relationship between the execution time of the algorithm and the data size . By analogy, the full name of space complexity is asymptotic space complexity (asymptotic space complexity), which represents the growth relationship between the storage space of the algorithm and the data scale.

The following example (generally no one will write like this, here is for the convenience of explanation)

void print(int n) {
    
    
    int i = 0;
    int[] a = new int[n];
    for (i; i < n; ++i) {
    
    
        a[i] = i * i;
    }
    for (i = n - 1; i >= 0; --i) {
    
    
        print out a[i]
    }
}

Similar to the time complexity analysis, we can see that in the second line of code, a space storage variable i is applied, but it is of constant order and has nothing to do with the data size n, so it can be ignored. Line 3 applies for an array of int type with size n, other than that, the rest of the code does not take up more space, so the space complexity of the whole code is O ( n ) O(n $O\left(n\right)$

The common space complexity is $O\left(1\right)$、$O\left(n\right)$、$O\left(n^2\right)$, the image$O\left(logn\right)$、$Logarithmic$ complexity such as$O$$($$n$$log$$n$$)$$is\; usually\; not\; used.$Moreover, space complexity analysis is much simpler than time complexity analysis

2.6 Best, Worst, Average Case Time Complexity

Let's look at an example first:

// n 表示数组 array 的长度
int find(int[] array, int n, int x) {
    
    
    int i = 0;
    int pos = -1;
    for (; i < n; ++i) {
    
    
        if (array[i] == x) pos = i;
    }
    return pos;
}

The function of this code is to find the position where the variable x appears in an unordered array (array). If not found, -1 is returned. According to the analysis method mentioned earlier, the complexity of this code is $O\left(n\right)$ where n is the length of the array

To find a piece of data in an array, it is not necessary to traverse the entire array every time, because it is possible to end the loop early if it is found halfway. However, this code is not written efficiently enough. You can optimize this lookup code like this:

// n 表示数组 array 的长度
int find(int[] array, int n, int x) {
    
    
    int i = 0;
    int pos = -1;
    for (; i < n; ++i) {
    
    
        if (array[i] == x) {
    
    
            pos = i;
            break;
        }
    }
    return pos;
}

At this time, the problem comes. After optimization, the time complexity of this code is still $Is\; it\; O\left(n\right)$ ? Obviously, the analysis method mentioned above cannot solve this problem.

Because, the variable x to be looked up may appear in any position of the array. If the first element in the array happens to be the variable x to be found, then there is no need to continue to traverse the remaining n-1 data, and the time complexity is O ( 1 ) $O\left(1\right)$ . But if the variable x does not exist in the array, then the entire array needs to be traversed, and the time complexity becomes$O\left(n\right)$ . Therefore, in different situations, the time complexity of this code is different

In order to represent the different time complexities of the code in different situations, three concepts need to be introduced: best case time complexity, worst case time complexity and average case time complexity

• As the name suggests, the best-case time complexity is, in the best case, the time complexity of executing this piece of code. As just mentioned, in the most ideal case, the variable x to be searched happens to be the first element of the array, and the corresponding time complexity at this time is the best case time complexity
• In the same way, the worst-case time complexity is, in the worst case, the time complexity of executing this code. Just like the example just mentioned, if there is no variable x to find in the array, the entire array needs to be traversed, so the time complexity corresponding to this worst case is the worst case time complexity

The best-case time complexity and worst-case time complexity correspond to the code complexity in extreme cases, and the probability of occurrence is actually not high. In order to better represent the complexity of the average case, another concept needs to be introduced: the average case time complexity, hereinafter referred to as the average time complexity

How to analyze the average time complexity? Still using the example of finding the variable x just now

The position of the variable x to be found in the array has n+1 cases: it is in the 0～n-1 position of the array and it is not in the array. In each case, add up the number of elements that need to be traversed, and then divide by n+1 to get the average number of elements that need to be traversed. The formula is as follows (the sum of the arithmetic sequence is equal to the first item plus the last item multiplied by the number of items divided by 2):

$$\frac{1+2+3+...+n+n}{n+1}=\frac{(1+n)\times \frac{n}{2}+n}{n+1}=\frac{\frac{n+n^2+2n}{2}}{n+1}=\frac{n(n+3)}{2(n+1)}$$

In the big O notation of time complexity, coefficients, low-orders, and constants can be omitted. Therefore, after simplifying the above formula, the average time complexity obtained is O ( n ) O(n $O\left(n\right)$

Although this conclusion is correct, the calculation process is slightly problematic. What is the problem? The n+1 situations mentioned above have different probabilities

The variable x to be looked up is either in the array or not in the array. The probability corresponding to these two situations is cumbersome to count. For the convenience of understanding, it is assumed that the probability of being in the array and not being in the array is 1/2. In addition, the probability that the data to be searched appears in the n positions from 0 to n-1 is the same, which is 1/n. Therefore, according to the probability multiplication rule, the probability that the data to be searched appears in any position between 0 and n-1 is $\frac{1}{2n}$

Therefore, the biggest problem in the previous derivation process is that the probability of occurrence of various situations is not taken into account. If the probability of occurrence of each situation is also taken into account, the calculation process of the average time complexity becomes like this:

$$1\times \frac{1}{2n}+2\times \frac{1}{2n}+3\times \frac{1}{2n}+...+n\times \frac{1}{2n}+n\times \frac{1}{2}=((1+n)\times \frac{n}{2})\times \frac{1}{2n}+\frac{n}{2}=\frac{n+n^2}{4n}+\frac{n}{2}=\frac{3n+1}{4}$$

This value is the weighted average in probability theory , also known as expected value , so the full name of average time complexity should be called weighted average time complexity or expected time complexity

After introducing probability, the weighted average of the previous piece of code is $\frac{3n+}{4}$。用大 O 表示法来表示，去掉系数和常量，这段代码的加权平均时间复杂度仍然是 $O(n)$

实际上，在大多数情况下，并不需要区分最好、最坏、平均情况时间复杂度三种情况。很多时候，使用一个复杂度就可以满足需求了。只有同一块代码在不同的情况下，时间复杂度有量级的差距，才会使用这三种复杂度表示法来区分

2.7 均摊时间复杂度

时间复杂度，听起来跟平均时间复杂度有点儿像。前面说到，大部分情况下，并不需要区分最好、最坏、平均三种复杂度。平均复杂度只在某些特殊情况下才会用到，而均摊时间复杂度应用的场景比它更加特殊、更加有限，如下例（仅为方便讲解，一般没人会这么写）：

// array 表示一个长度为 n 的数组
// 代码中的 array.length 就等于 n
int[] array = new int[n];
int count = 0;
void insert(int val) {
    
    
    if (count == array.length) {
    
    
        int sum = 0;
        for (int i = 0; i < array.length; ++i) {
    
    
            sum = sum + array[i];
        }
        array[0] = sum;
        count = 1;
    }
    array[count] = val;
    ++count;
}

这段代码实现了一个往数组中插入数据的功能。当数组满了之后，也就是代码中的 count == array.length 时，用 for 循环遍历数组求和，并清空数组，将求和之后的 sum 值放到数组的第一个位置，然后再将新的数据插入。但如果数组一开始就有空闲空间，则直接将数据插入数组

那这段代码的时间复杂度是多少呢？

最理想的情况下，数组中有空闲空间，只需要将数据插入到数组下标为 count 的位置就可以了，所以最好情况时间复杂度为 $ O(1)$。最坏的情况下，数组中没有空闲空间了，需要先做一次数组的遍历求和，然后再将数据插入，所以最坏情况时间复杂度为 $O(n)$

那平均时间复杂度是多少呢？答案是 $O(1)$。还是可以通过前面讲的概率论的方法来分析

Assuming that the length of the array is n, according to the position of data insertion, it can be divided into n cases, and the time complexity of each case is $O\left(1\right)$ . In addition, there is an "extra" situation, which is to insert a piece of data when there is no free space in the array. The time complexity at this time is$O\left(n\right)$ . Moreover, the probability of these n+1 situations is the same, they are all$\frac{1}{n+1}$. Therefore, according to the calculation method of weighted average, the average time complexity obtained is:

$$1\times \frac{1}{n+1}+1\times \frac{1}{n+1}+...+n\times \frac{1}{n+1}=O(1)$$

But the average complexity analysis in this example does not need to be so complicated, and does not need to introduce the knowledge of probability theory. Why is this? Let's first compare this insert()example with the previous find()example, you will find that there is a big difference between the two

• First of all, find()the complexity of the function is $O\left(1\right)$ . Butinsert()in most cases, the time complexity is$O\left(1\right)$ . Only in a few cases, the complexity is relatively high,$O\left(n\right)$
• For insert()the function, $O\left(1\right)$ time complexity for insertion and$The\; insertion\; of\; O\left(n\right)$ time complexity, the frequency of occurrence is very regular, and there is a certain timing relationship before and after, generally an$O\left(n\right)$ insertion followed by n-1$O\left(1\right)$ insertion operation, iterates over and over again

Therefore, for the complexity analysis of such a special scene, it is not necessary to find out all the input situations and the corresponding occurrence probabilities as in the average complexity analysis method before, and then calculate the weighted average

For this special scenario, a simpler analysis method is introduced: the amortized analysis method . The time complexity obtained through the amortized analysis is called the amortized time complexity

So how to use the amortized analysis method to analyze the amortized time complexity of the algorithm?

Look at the above example of inserting data into the array, each time $O\left(n\right)$ insertion operations will be followed by n-1 times$O\left(1\right)$ insertion operation, so the more time-consuming operation is evenly distributed to the next n-1 less time-consuming operations, and the average time complexity of this group of continuous operations is O ( 1 ) O(1$O\left(1\right)$ . This is the general idea of ​​amortized analysis

The application scenarios of amortized time complexity and amortized analysis are relatively special, so they are not often used

In a group of continuous operations on a data structure, the time complexity is very low in most cases. Only in a few cases, the time complexity is relatively high, and there is a coherent timing relationship between these operations. At this time, this group of operations can be analyzed together to see if the time consumption of the operation with higher time complexity can be amortized to other operations with lower time complexity. Moreover, where the amortized time complexity analysis can be applied, the general amortized time complexity is equal to the best case time complexity

Although many data structure and algorithm books have spent a lot of effort to distinguish between average time complexity and amortized time complexity, in fact, I personally think that amortized time complexity is a special average time complexity, and there is no need to spend too much effort to distinguish them. The most important thing to master is its analysis method, amortization analysis. As for whether the result of the analysis is called average or evenly shared, this is just a way of saying, it is not important

3. Arrays

3.1 How to achieve random access?

Array (Array) is a linear table data structure. It uses a set of contiguous memory spaces to store a set of data of the same type

There are several keywords in this definition. After understanding these keywords, you can basically grasp the concept of arrays thoroughly.

1. Linear List

As the name implies, a linear table is a structure in which data is arranged like a line. The data on each linear table has at most two directions, front and back. In fact, in addition to arrays, linked lists, queues, stacks, etc. are also linear table structures

The concept opposite to it is a nonlinear table, such as a binary tree, a heap, a graph, and so on. The reason why it is called non-linear is that in a non-linear table, there is not a simple context between data

2. Contiguous memory space and the same type of data

It is precisely because of these two limitations that it has a feature called "killer": "random access". But there are advantages and disadvantages. These two limitations also make many operations on the array very inefficient. For example, if you want to delete or insert a piece of data in the array, in order to ensure continuity, you need to do a lot of data moving.

How does the array implement random access to array elements based on subscripts?

Take an array of type int with length 10 int[] a = new int[10]as an example. In the figure below, the computer allocates a continuous memory space from 1000 to 1039 for the array a[10], where the first address of the memory block is base_address = 1000

The computer assigns an address to each memory unit, and the computer uses the address to access the data in the memory. When the computer needs to randomly access an element in the array, it will first calculate the memory address of the element through the following addressing formula:

a[i]_address = base_address + i * data_type_size

Where data_type_size represents the size of each element in the array. In the above example, int type data is stored in the array, so data_type_size is 4 bytes. This formula is very simple, so I won’t introduce too much here

There is one "mistake" to be corrected here. During interviews, the difference between arrays and linked lists is often asked, and many people answer, "Linked lists are suitable for insertion and deletion, and the time complexity is O ( 1 ) $O\left(1\right)$ ; the array is suitable for lookup, the lookup time complexity is$O(1)$”

In fact, this representation is inaccurate. Arrays are suitable for search operations, but the time complexity of search is not $O\left(1\right)$ . Even for sorted arrays, using binary search, the time complexity is$O\left(logn\right)$ .$\_$$\_$Therefore, the correct expression should be that the array supports random access, and the time complexity of random access according to the subscript is$O(1)$

3.2 Inefficient "insert" and "delete"

As mentioned earlier, in order to maintain the continuity of memory data in the array, the two operations of insertion and deletion will be relatively inefficient. Now why exactly is this causing the inefficiency? What are the ways to improve it?

First look at the insert operation

Assuming that the length of the array is n, now, if a piece of data needs to be inserted into the kth position in the array. In order to free up the kth position for the new data, the elements of the kth to nth part need to be moved back one bit in sequence. What is the time complexity of the insert operation?

If you insert elements at the end of the array, there is no need to move the data, and the time complexity is $O\left(1\right)$ . But if you insert an element at the beginning of the array, all the data needs to be moved backwards one by one, so the worst time complexity is$O\left(n\right)$ . Because the probability of inserting an element at each position is the same, the average case time complexity is$\frac{1+2+\dots n}{n}=O\left(n\right)$

If the data in the array is ordered, when a new element is inserted at a certain position, the data after k must be moved according to the method just now. However, if the data stored in the array does not have any rules, the array is only regarded as a collection of stored data. In this case, if you want to insert an array into the kth position, in order to avoid large-scale data movement, there is another simple way is to directly move the kth bit of data to the end of the array element, and put the new element directly into the kth position

For example, the following 5 elements are stored in the array a[10]: a, b, c, d, e. Now the element x needs to be inserted at the 3rd position. Just put c into a[5] and assign a[2] to x. Finally, the elements in the array are as follows: a, b, x, d, e, c

Using this processing technique, in a specific scenario, the time complexity of inserting an element at the kth position will be reduced to $O\left(1\right)$ . This processing idea will also be used in quick sorting

Let's look at the delete operation

Similar to inserting data, if you want to delete the data at the kth position, you also need to move the data for the continuity of the memory, otherwise there will be a hole in the middle, and the memory will not be continuous

Similar to insertion, if you delete the data at the end of the array, the best case time complexity is $O\left(1\right)$ ; if you delete the data at the beginning, the worst case time complexity is$O\left(n\right)$ ; average case time complexity is also$O\left(n\right)$

In fact, in some special scenarios, it is not necessary to pursue the continuity of data in the array. If multiple deletion operations are performed together, will the efficiency of deletion be improved a lot?

For example, 8 elements are stored in the array a[10]: a, b, c, d, e, f, g, h. Now, to delete a, b, c three elements in turn

In order to avoid the data of d, e, f, g, and h being moved three times, you can first record the deleted data. Each deletion operation does not actually move the data, but only records that the data has been deleted. When the array has no more space to store data, a real delete operation is triggered, which greatly reduces the data movement caused by the delete operation

If you understand the JVM, you will find that this is the core idea of ​​the JVM mark-sweep garbage collection algorithm. This is the charm of data structures and algorithms. In many cases, it is not to memorize a certain data structure or algorithm by rote, but to learn the thinking and processing skills behind it. These things are the most valuable. If you pay attention, whether in software development or architecture design, you can always find the shadow of certain algorithms and data structures

3.3 Be wary of array access out of bounds

First, let's analyze the running results of this C language code:

int main(int argc, char * argv[]) {
    
    
    int i = 0;
    int arr[3] = {
    
    0};
    for(; i <= 3; i++) {
    
    
        arr[i] = 0;
        printf("hello world\n");
    }
    return 0;
}

The result of this code is not to print three lines of "hello word", but to print "hello world" infinitely. Why?

Because the size of the array is 3, a[0], a[1], a[2], and the code is wrongly written, the end condition of the for loop is wrongly written as i<=3 instead of i<3, so when i=3, the access of the array a[3] is out of bounds

In the C language, all memory spaces are freely accessible as long as they are not access-restricted memory. According to the array addressing formula mentioned earlier, a[3] will also be located at a memory address that does not belong to the array, and this address happens to be the memory address where the variable i is stored, then a[3]=0 is equivalent to i=0, so it will lead to an infinite loop of code

Array out-of-bounds is a pending behavior in C language, and there is no regulation on how the compiler should deal with array access out-of-bounds. Because the essence of accessing an array is to access a piece of continuous memory, as long as the memory address obtained by calculating the offset of the array is available, the program may not report any errors

In this case, inexplicable logic errors will generally occur. Just like the example above, debugging is very difficult. Moreover, many computer viruses also use the loopholes in the code to access illegal addresses when the array is out of bounds to attack the system, so when writing code, you must be wary of array out of bounds

But not all languages ​​are like C, which leaves the work of array out-of-bounds checks to programmers, like Java itself will do out-of-bounds checks, such as the following lines of Java code, it will throwjava.lang.ArrayIndexOutOfBoundsException

int[] a = new int[3];
a[3] = 10;

3.4 Can containers completely replace arrays?

For array types, many languages ​​provide container classes, such as ArrayList in Java and vector in C++ STL. In project development, when is it appropriate to use arrays and when is it appropriate to use containers?

Take the Java language as an example. If you are a Java engineer and use ArrayList almost every day, you should be very familiar with it. So what advantages does it have over arrays?

Personally, the biggest advantage of ArrayList is that it can encapsulate the details of many array operations. For example, when inserting and deleting data in the array mentioned above, other data needs to be moved. In addition, it has another advantage, that is, it supports dynamic expansion

The array itself needs to specify the size in advance when it is defined, because it needs to allocate continuous memory space. If you apply for an array with a size of 10, when the 11th data needs to be stored in the array, you need to reallocate a larger space, copy the original data, and then insert the new data

If you use ArrayList, you don't need to care about the underlying expansion logic at all, as ArrayList has already been implemented. Every time the storage space is not enough, it will automatically expand the space to 1.5 times the size

However, you need to pay attention here, because the expansion operation involves memory application and data movement, which is time-consuming. Therefore, if the size of the data to be stored can be determined in advance, it is best to specify the data size in advance when creating the ArrayList

For example, you want to take out 10,000 pieces of data from the database and put them into ArrayList. Look at the following lines of code. In contrast, specifying the data size in advance can save a lot of memory application and data movement operations

ArrayList<User> users = new ArrayList(10000);
for (int i = 0; i < 10000; ++i) {
    
    
 users.add(xxx);
}

So is the array useless? Of course not, sometimes it is more appropriate to use an array

1. Java ArrayList cannot store basic types, such as int and long, and needs to be packaged as Integer and Long classes, while Autoboxing and Unboxing will consume a certain amount of performance, so if you pay special attention to performance or want to use basic types, you can use arrays
2. If the size of the data is known in advance, and the operation on the data is very simple, most of the methods provided by ArrayList are not used, and the array can also be used directly
3. Another is that when multidimensional arrays are to be represented, it is often more intuitive to use arrays. For example Object[][] array, if you use a container, you need to define it like this:ArrayList<ArrayList> array

For business development, it is enough to use the container directly, saving time and effort. After all, a loss of performance will not affect the overall performance of the system at all. But if you are doing some very low-level development, such as developing a network framework, performance optimization needs to be extreme. At this time, arrays will be better than containers and become the first choice.

3.5 Why do arrays start numbering from 0 instead of 1 in most programming languages?

From the memory model of array storage, the most accurate definition of "subscript" should be "offset". As mentioned earlier, if a is used to represent the first address of the array, a[0] is the position with an offset of 0, that is, the first address, and a[k] represents the position offset by k type_size, so you only need to use this formula to calculate the memory address of a[k]:

a[k]_address = base_address + k * type_size

However, if the array starts counting from 1, then the memory address to calculate the array element a[k] becomes:

a[k]_address = base_address + (k - 1) * type_size

Comparing the two formulas, it is not difficult to find that starting numbering from 1, each random access to an array element requires one more subtraction operation. For the CPU, it is one more subtraction instruction

Arrays are a very basic data structure, and accessing array elements randomly through subscripts is a very basic programming operation. The optimization of efficiency must be as extreme as possible. So in order to reduce one subtraction operation, the array chooses to start numbering from 0 instead of 1

However, no matter how many explanations are given above, it is not an overwhelming proof that the starting number of the array must start with 0. The most important reason may be historical reasons

C language designers start counting array subscripts with 0, and later high-level languages ​​such as Java and JavaScript have followed C language, or in other words, in order to reduce the learning cost of C language programmers learning Java to a certain extent, they continue to use the habit of counting from 0. In fact, arrays do not start counting from 0 in many languages, such as Matlab. There are even some languages ​​that support negative subscripts, such as Python

4. Linked List

4.1 Linked list structure

A linked list is a slightly more complex data structure than an array. These two very basic and commonly used data structures are often compared together. So let's see what's the difference between the two

underlying storage structure

As shown in the figure below, the array needs a continuous memory space to store, and the memory requirements are relatively high. If you apply for an array with a size of 100MB, when there is no continuous and large enough storage space in the memory, even if the remaining total available space of the memory is greater than 100MB, the application will still fail

On the contrary, the linked list does not need a continuous memory space. It connects a group of scattered memory blocks in series through "pointers", so if you apply for a linked list with a size of 100MB, there will be no problem at all.

There are various linked list structures. Here are the three most common linked list structures. They are: single linked list, double linked list and circular linked list

1. Singly linked list

As just mentioned, the linked list connects a group of scattered memory blocks together through pointers. Among them, the memory block is called the "node" of the linked list. In order to string all the nodes together, in addition to storing data, each node of the linked list also needs to record the address of the next node on the chain. As shown in the figure, the pointer that records the address of the next node is called the successor pointer next

As can be seen from the figure above, there are two special nodes, which are the first node and the last node. It is customary to call the first node the head node and the last node the tail node. Among them, the head node is used to record the base address of the linked list. With it, you can traverse to get the entire linked list. The special feature of the end node is: the pointer does not point to the next node, but points to an empty address NULL, indicating that this is the
last node on the linked list

Like arrays, linked lists also support data lookup, insertion, and deletion operations

When performing array insertion and deletion operations, in order to maintain the continuity of memory data, a large amount of data needs to be moved, so the time complexity is $O\left(n\right)$ . When inserting or deleting a piece of data in the linked list, there is no need to move nodes to maintain the continuity of the memory, because the storage space of the linked list itself is not continuous. Therefore, inserting and deleting a data in the linked list is very fast

As shown in the figure below, for the insertion and deletion operations of the linked list, only the pointer changes of adjacent nodes need to be considered, so the corresponding time complexity is $O(1)$

However, there are pros and cons. If a linked list wants to randomly access the kth element, it is not as efficient as an array. Because the data in the linked list is not stored continuously, it is impossible to directly calculate the corresponding memory address through the addressing formula according to the first address and subscript like an array. Instead, it needs to traverse one node by one according to the pointer until the corresponding node is found.

You can think of a linked list as a team. Everyone in the team only knows who the person behind them is, so when you want to know who the person in kth place is, you need to start from the first person and count down one by one. Therefore, the performance of random access of the linked list is not as good as that of the array, requiring $O\left(n\right)$ time complexity

2. Circular linked list

A circular linked list is a special kind of singly linked list. In fact, the circular linked list is also very simple. The only difference between it and the singly linked list is the tail node. The tail node pointer of the singly linked list points to an empty address, indicating that this is the last node. The tail node pointer of the circular linked list points to the head node of the linked list. As shown in the figure below, it is connected end to end like a ring, so it is called a "circular" linked list

Compared with the singly linked list, the advantage of the circular linked list is that it is more convenient to go from the end of the chain to the head of the chain. When the data to be processed has the characteristics of a ring structure, it is especially suitable to use a circular linked list. Such as the famous Joseph problem. Although it can also be implemented with a singly linked list, if implemented with a circular linked list, the code will be much simpler

3. Doubly linked list

The one-way linked list has only one direction, and the node has only one successor pointer next pointing to the following node. The doubly linked list, as the name suggests, supports two directions. Each node has more than one successor pointer next pointing to the following node, and a predecessor pointer prev pointing to the previous node.

The doubly linked list requires two additional spaces to store the address of the successor node and the predecessor node. Therefore, if the same amount of data is stored, the doubly linked list takes up more memory space than the singly linked list. Although two pointers are a waste of storage space, they can support bidirectional traversal, which also brings the flexibility of doubly linked list operations. Compared with the single linked list, what kind of problem is the doubly linked list suitable for solving?

From a structural point of view, a doubly linked list can support $O(1)$ 时间复杂度的情况下找到前驱结点，正是这样的特点，也使双向链表在某些情况下的插入、删除等操作都要比单链表简单、高效

这时可能会说，前面讲到单链表的插入、删除操作的时间复杂度已经是 $O(1)$ 了，双向链表还能再怎么高效呢？前面的分析比较偏理论，很多数据结构和算法书籍中都会这么讲，但是这种说法实际上是不准确的，或者说是有先决条件的。这里再来分析一下链表的两个操作

删除操作：在实际的软件开发中，从链表中删除一个数据无外乎下面两种情况

• 删除结点中“值等于某个给定值”的结点
• 删除给定指针指向的结点

对于第一种情况，不管是单链表还是双向链表，为了查找到值等于给定值的结点，都需要从头结点开始一个一个依次遍历对比，直到找到值等于给定值的结点，然后再通过前面讲的指针操作将其删除

尽管单纯的删除操作时间复杂度是 $O(1)$，但遍历查找的时间是主要的耗时点，对应的时间复杂度为 $O(n)$。根据时间复杂度分析中的加法法则，删除值等于给定值的结点对应的链表操作的总时间复杂度为 $O(n)$

对于第二种情况，已经找到了要删除的结点，但是删除某个结点 q 需要知道其前驱结点，而单链表并不支持直接获取前驱结点，所以，为了找到前驱结点，还是要从头结点开始遍历链表，直到 p->next=q，说明 p 是 q 的前驱结点

但是对于双向链表来说，这种情况就比较有优势了。因为双向链表中的结点已经保存了前驱结点的指针，不需要像单链表那样遍历。所以，针对第二种情况，单链表删除操作需要 O(n) 的时间复杂度，而双向链表只需要在 O(1) 的时间复杂度内就搞定了！

插入操作同理，如果希望在链表的某个指定结点前面插入一个结点，双向链表比单链表有很大的优势。双向链表可以在 $O(1)$ 时间复杂度搞定，而单向链表需要 $O(n)$ 的时间复杂度

除了插入、删除操作有优势之外，对于一个有序链表，双向链表的按值查询的效率也要比单链表高一些。因为，可以记录上次查找的位置 p，每次查询时，根据要查找的值与 p的大小关系，决定是往前还是往后查找，所以平均只需要查找一半的数据

有没有觉得双向链表要比单链表更加高效呢？这就是为什么在实际的软件开发中，双向链表尽管比较费内存，但还是比单链表的应用更加广泛的原因。如果你熟悉 Java 语言，你肯定用过 LinkedHashMap 这个容器。如果你深入研究 LinkedHashMap 的实现原理，就会发现其中就用到了双向链表这种数据结构

In fact, there is a more important knowledge point for you to master, that is, the design idea of ​​exchanging space for time . When the memory space is sufficient, if you pursue the execution speed of the code more, you can choose an algorithm or data structure with relatively high space complexity but relatively low time complexity. On the contrary, if the memory is relatively scarce, such as code running on a mobile phone or a single-chip microcomputer, at this time, it is necessary to reverse the design idea of ​​exchanging time for space

Caching is actually the design idea of ​​using space for time. If the data is stored on the hard disk, it will save memory, but every time the data is searched, the hard disk must be queried, which will be slower. However, if the data is loaded in memory in advance through caching technology, although it will consume more memory space, the speed of each data query will be greatly improved.

For programs that execute slowly, they can be optimized by consuming more memory (space for time); for programs that consume too much memory, they can reduce memory consumption by consuming more time (time for space)

After understanding the circular linked list and the doubly linked list, if these two linked lists are integrated together, it will be a new version: doubly linked list

4.2 Linked List VS Array Performance Comparison

Arrays and linked lists are two very different ways of organizing memory. It is precisely because of the difference in memory storage that the time complexity of their insertion, deletion, and random access operations is just the opposite

However, the comparison between arrays and linked lists cannot be limited to time complexity. Moreover, in actual software development, it is not possible to decide which data structure to use to store data only by complexity analysis

The array is simple and easy to use. The implementation uses a continuous memory space. The data in the array can be read in advance with the help of the CPU's cache mechanism, so the access efficiency is higher. The linked list is not stored continuously in the memory, so it is not friendly to the CPU cache, and there is no way to effectively read ahead

The disadvantage of an array is that its size is fixed, and once it is declared, it will occupy the entire continuous memory space. If the declared array is too large, the system may not have enough contiguous memory space allocated to it, resulting in "out of memory". If the declared array is too small, it may not be sufficient. At this time, we can only apply for a larger memory space and copy the original array into it, which is very time-consuming. The linked list itself has no size limit and naturally supports dynamic expansion, which is the biggest difference between it and the array

Although the ArrayList container in Java supports dynamic expansion, it is actually an array copy operation. When inserting a piece of data into an array that supports dynamic expansion, if there is no free space in the array, a larger space will be applied for and the data will be copied, and the data copy operation is very time-consuming

Take a slightly extreme example. If you use ArrayList to store 1GB of data, there is no free space at this time. When inserting data again, ArrayList will apply for a 1.5GB storage space, and copy the original 1GB of data to the newly applied space. Does that sound like a lot of time?

Besides that, if the code is very memory-critical, arrays are more suitable. Because each node in the linked list needs to consume additional storage space to store a pointer to the next node, the memory consumption will double. Moreover, frequent insertion and deletion operations on the linked list will also lead to frequent memory application and release, which will easily cause memory fragmentation. If it is Java language, it may lead to frequent GC (Garbage Collection, garbage collection). Therefore, in actual development, for different types of projects, it is necessary to weigh whether to choose an array or a linked list according to the specific situation.

4.3 How to implement the LRU cache elimination algorithm based on the linked list?

Caching is a technology to improve data reading performance. It is widely used in hardware design and software development, such as common CPU cache, database cache, browser cache, etc.

The size of the cache is limited. When the cache is full, which data should be cleared out and which data should be kept? This requires a cache elimination strategy to decide. There are three common strategies: first-in-first-out strategy FIFO (First In, First Out), least used strategy LFU (Least Frequently Used), least recently used strategy LRU (Least Recently Used)

These strategies are actually well-known by name, so how to implement the LRU cache elimination algorithm based on the linked list?

An ordered singly-linked list can be maintained, and nodes closer to the end of the linked list are accessed earlier. When a new data is accessed, traverse the linked list sequentially from the head of the linked list

1. If this data has been cached in the linked list before, traverse to get the node corresponding to this data, delete it from the original position, and then insert it into the head of the linked list

2. If the data is not in the cache linked list, it can be divided into two situations:

   • If the cache is not full at this time, insert this node directly into the head of the linked list
   • If the cache is full at this time, the tail node of the linked list is deleted, and a new data node is inserted into the head of the linked list

In this way, an LRU cache is implemented with a linked list, is it very simple?

Now let's look at the time complexity of m cache access. Because the linked list needs to be traversed regardless of whether the cache is full or not, the time complexity of the cache access is $O\left(n\right)$

In fact, you can continue to optimize this implementation idea, such as introducing a hash table (Hash table) to record the location of each data, reducing the time complexity of cache access to $O(1)$

In addition to the implementation idea based on linked list, in fact, arrays can also be used to implement the LRU cache elimination strategy

4.4 How to easily write the correct linked list code?

It is not easy to write linked list code well, especially those complicated linked list operations, such as linked list reversal, ordered linked list merging, etc., are very error-prone when writing. Why is linked list code so difficult to write? How can we write the correct linked list code more easily?

1. Understand the meaning of pointers or references

In fact, it is not difficult to understand the structure of the linked list, but once it is mixed with pointers, it is easy to be confused. Therefore, if you want to write the linked list code correctly, you must first understand the pointer

Some languages ​​have the concept of "pointers", such as C language; some languages ​​do not have pointers, but use "references" instead, such as Java and Python. Regardless of whether it is a "pointer" or a "reference", in fact, their meanings are the same, and they are all memory addresses that store the pointed object

In fact, for the understanding of pointers, you only need to remember the following sentence:

Assigning a variable to a pointer is actually assigning the address of the variable to the pointer, or conversely, the memory address of the variable is stored in the pointer, pointing to the variable, and the variable can be found through the pointer

When writing linked list code, there are often such codes: · p->next = q. This line of code means that the next pointer in the p node stores the memory address of the q node

There is also a more complicated one, which is often used when writing linked list code: p->next = p->next->next. This line of code indicates that the next pointer of the p node stores the memory address of the next node of the p node

Once you have mastered the concept of pointers or references, you should be able to easily understand the linked list code

2. Be wary of lost pointers and memory leaks

When writing linked list code, the pointer points to and fro, and I don't know where it points for a while. Therefore, when writing, be careful not to lose the pointer

How do pointers often get lost? Here is an example of the insertion operation of a singly linked list

As shown in the figure, it is hoped to insert node x between node a and adjacent node b, assuming that the current pointer p points to node a. If the code implementation is changed to the following, pointer loss and memory leaks will occur

p->next = x; // 将 p 的 next 指针指向 x 结点；
x->next = p->next; // 将 x 的结点的 next 指针指向 b 结点；

After the first step, the p->next pointer no longer points to node b, but points to node x. The second line of code is equivalent to assigning x to x->next, pointing to itself. Therefore, the entire linked list is broken into two halves, and all nodes from node b onwards cannot be accessed

For some languages, such as C language, memory management is the responsibility of the programmer. If the memory space corresponding to the node is not manually released, memory leaks will occur. Therefore, when inserting a node, you must pay attention to the order of operations. You must first point the next pointer of node x to node b, and then point the next pointer of node a to node x, so that the pointer will not be lost, resulting in memory leaks. So, for the inserted code just now, you only need to reverse the order of the code in line 1 and line 2

Similarly, when deleting a linked list node, you must also remember to manually release the memory space, otherwise, memory leaks will also occur. Of course, for a programming language like Java, where the virtual machine automatically manages memory, there is no need to consider so much

3. Use Sentinel to simplify the implementation difficulty

First, let's review the insertion and deletion operations of the singly linked list. If you insert a new node after node p, you only need the following two lines of code to get it done

new_node->next = p->next;
p->next = new_node;

However, when inserting the first node into an empty linked list, the logic just now cannot be used. The following special processing is required, where head represents the head node of the linked list. Therefore, from this code, it can be found that for the insertion operation of the singly linked list, the insertion logic of the first node and other nodes is different

if (head == null) {
    
    
   head = new_node;
}

Let's look at the delete operation of the singly linked list node. If you want to delete the successor node of node p, you only need one line of code to get it done

p->next = p->next->next;

However, if you want to delete the last node in the linked list, the previous deletion code will not work. Similar to insertion, special handling is also required for this case

if (head->next == null) {
    
    
   head = null;
}

From the previous step-by-step analysis, it can be seen that for the insertion and deletion operations of the linked list, special handling needs to be performed for the case of inserting the first node and deleting the last node. In this way, the code will be cumbersome to implement, not concise, and it is easy to make mistakes due to incomplete consideration. How to solve this problem?

Sentinel , the sentinel mentioned here is to solve the "boundary problem" and does not directly participate in business logic

head = nullIndicates that there are no more nodes in the linked list. Where head represents the head node pointer, pointing to the first node in the linked list. If a sentinel node is introduced, at any time, regardless of whether the linked list is empty or not, the head pointer will always point to this sentinel node. This kind of linked list with sentinel nodes is also called the leading linked list. Conversely, a linked list without a sentinel node is called a headless linked list

As shown in the figure below, it can be found that the sentinel node does not store data. Because the sentinel node has always existed, inserting the first node and inserting other nodes, deleting the last node and deleting other nodes can all be unified into the same code to implement logic

In fact, this technique of using sentinels to simplify programming difficulty is used in many code implementations, such as insertion sorting, merge sorting, dynamic programming, etc.

4. Pay attention to the processing of boundary conditions

In software development, the code is most prone to bugs in some borderline or abnormal situations. Linked list code is no exception. To implement linked list code without bugs, it is necessary to check whether the boundary conditions are fully considered during the writing process and after the writing is completed, and whether the code can run correctly under the boundary conditions

There are several boundary conditions that are often used to check whether the linked list code is correct:

• Does the code work correctly if the linked list is empty?
• Does the code work correctly if the linked list contains only one node?
• Does the code work correctly if the linked list contains only two nodes?
• Does the code logic work properly when dealing with head nodes and tail nodes?

After writing the linked list code, in addition to checking whether the code can work under normal conditions, it also depends on whether the code can still work correctly under the above boundary conditions. If there are no problems under these boundary conditions, it can basically be considered that there is no problem

Of course, the boundary conditions are not limited to those listed above. For different scenarios, there may be specific boundary conditions. This needs to be thought about by yourself, but the routine is the same

In fact, not only writing linked list code, but when writing any code, don’t just implement the functions under normal business conditions. You must think more about the boundary conditions or abnormal conditions that may be encountered when the code is running. How to deal with the encounter, so that the code written is robust enough!

5. Drawing examples to aid thinking

For slightly complicated linked list operations, such as the aforementioned single-linked list reversal, the pointer points here and there for a while, and then gets dizzy for a while. I always feel that my brain capacity is not enough, and I can't think clearly. So at this time, other methods can be used to assist understanding, such as: example method and drawing method

You can find a specific example, draw it on paper, release some brain capacity, and leave more for logical thinking, so that you will feel that your thinking is much clearer. For example, an operation such as inserting a piece of data into a singly linked list generally takes an example of various situations and draws the changes of the linked list before and after insertion, as shown in the figure

It is much easier to write code by looking at the picture. Moreover, after writing the code, you can also give a few examples, draw on paper, follow the code, and you can easily find bugs in the code

6. Write more and practice more, there is no shortcut

Practice makes perfect! Here are 5 common linked list operations. As long as you can write these operations proficiently, write it several times if you are not familiar with it, so that you will never be afraid to write linked list codes again.

• Singly linked list reverse
• Detection of rings in linked list
• Merge of two sorted linked lists
• Delete the last nth node of the linked list
• Find the middle node of the linked list

5. Stack

5.1 How to understand "stack"?

A very apt example of a "stack" is a stack of plates stacked on top of each other. When we usually put the plates, we put them one by one from bottom to top; when we take them out, we also take them one by one from top to bottom, and we cannot pull them out from the middle arbitrarily. Last-in first-out, advanced last-out , this is a typical "stack" structure

From the operating characteristics of the stack, the stack is a linear table with "restricted operations" , which only allows data to be inserted and deleted at one end.

In fact, from a functional point of view, an array or a linked list can indeed replace a stack, but a specific data structure is an abstraction of a specific scene. Moreover, an array or a linked list exposes too many operation interfaces. The operation is indeed flexible and free, but it is relatively uncontrollable when used, and naturally it is more error-prone.

When a data set only involves inserting and deleting data at one end, and satisfies the characteristics of last-in-first-out and first-in-last-out, the data structure of "stack" should be preferred

5.2 How to implement a "stack"?

The stack mainly includes two operations, push and pop, that is, insert a data at the top of the stack and delete a data from the top of the stack

In fact, a stack can be implemented with either an array or a linked list. A stack implemented with an array is called a sequential stack, and a stack implemented with a linked list is called a linked stack

An array-based sequential stack is implemented here

// 基于数组实现的顺序栈
public class ArrayStack {
    
    
    private String[] items; // 数组
    private int count; // 栈中元素个数
    private int n; // 栈的大小
    // 初始化数组，申请一个大小为 n 的数组空间
    public ArrayStack(int n) {
    
    
        this.items = new String[n];
        this.n = n;
        this.count = 0;
    }
    // 入栈操作
    public boolean push(String item) {
    
    
        // 数组空间不够了，直接返回 false，入栈失败。
        if (count == n) return false;
        // 将 item 放到下标为 count 的位置，并且 count 加一
        items[count] = item;
        ++count;
        return true;
    }

    // 出栈操作
    public String pop() {
    
    
        // 栈为空，则直接返回 null
        if (count == 0) return null;
        // 返回下标为 count-1 的数组元素，并且栈中元素个数 count 减一
        String tmp = items[count - 1];
        --count;
        return tmp;
    }
}

After understanding the definition and basic operations, what is the time and space complexity of its operations?

Regardless of whether it is a sequential stack or a chained stack, only an array with a size of n is enough to store data. In the process of pushing and popping, only one or two temporary variable storage spaces are needed, so the space complexity is $O(1)$

Note that storing data here requires an array of size n, not to say that the space complexity is $O\left(n\right)$ . Because these n spaces are necessary and cannot be omitted. So when we talk about space complexity, it means that in addition to the original data storage space, the algorithm operation also requires additional storage space.

Regardless of whether it is a sequential stack or a chain stack, pushing and popping only involve the operation of individual data on the top of the stack, so the time complexity is $O\left(1\right)$

5.3 Sequential stack supporting dynamic expansion

The stack implemented based on the array just now is a fixed-size stack, that is to say, the size of the stack needs to be specified in advance when initializing the stack. When the stack is full, no more data can be added to the stack. Although the size of the linked stack is not limited, it is relatively memory intensive to store the next pointer. So how to implement a stack that can support dynamic expansion based on arrays?

In the section on arrays, the way to implement an array that supports dynamic expansion is to re-apply for a larger memory when the array space is not enough, and copy all the data in the original array to the past, thus realizing an array that supports dynamic expansion

Therefore, if you want to implement a stack that supports dynamic expansion, you only need to rely on an array that supports dynamic expansion at the bottom layer. When the stack is full, apply for a larger array and move the original data to the new array, as shown in the figure below

In fact, sequential stacks that support dynamic expansion are not commonly used in development. Mainly still complexity analysis

For the pop-up operation, memory re-application and data movement will not be involved, so the time complexity of stack-popping is still $O\left(1\right)$ . However, for push operations, the situation is different. When there is free space in the stack, the time complexity of the stack operation is$O\left(1\right)$ . But when the space is not enough, you need to re-apply for memory and data movement, so the time complexity becomes$O\left(n\right)$

That is to say, for the push operation, the best case time complexity is $O\left(1\right)$ , worst case time complexity is$O\left(n\right)$ . What is the average time complexity? The average time complexity of this push operation can be analyzed by amortized analysis

For the convenience of analysis, some assumptions and definitions need to be made in advance:

• When the stack space is not enough, re-apply for an array twice the original size
• In order to simplify the analysis, it is assumed that there are only push operations and no pop operations
• Define the push operation that does not involve memory transfer as a simple-push operation, and the time complexity is $O(1)$

If the current stack size is K and it is full, when there is new data to be pushed into the stack, it is necessary to re-apply for twice the size of memory, and perform K data moving operations, and then push it into the stack. However, the next K-1 push operations do not need to re-apply for memory and move data, so these K-1 push operations can be completed with only one simple-push operation, as shown in the figure below

These K push operations involve a total of K data transfers and K simple-push operations. If K data transfers are amortized to K push operations, each push operation only requires one data transfer and one simple-push operation. By analogy, the amortized time complexity of the stack operation is $O\left(1\right)$

Through the actual analysis of this example, it also confirms what was mentioned earlier that the amortized time complexity is generally equal to the best case time complexity. Because in most cases, the time complexity O of the stack operation is $O\left(1\right)$ , degenerates to O ( n ) O(n)only at individual moments$O\left(n\right)$ , so the time-consuming stack operation is allocated to other stack operations, and the average time-consuming is close to$O(1)$

5.4 Application of the stack

5.4.1 Application of stack in function call

As a relatively basic data structure, the stack has many application scenarios. Among them, one of the more classic application scenarios is the function call stack

The operating system allocates an independent memory space for each thread, and this memory is organized into a "stack" structure, which is used to store temporary variables when the function is called. Every time a function is entered, the temporary variable will be pushed into the stack as a stack frame. When the called function is executed and returned, the stack frame corresponding to the function will be popped out of the stack. For example

int main() {
    
    
    int a = 1;
    int ret = 0;
    int res = 0;
    ret = add(3, 5);
    res = a + ret;
    printf("%d", res);
    reuturn 0;
}
int add(int x, int y) {
    
    
    int sum = 0;
    sum = x + y;
    return sum;
}

It can be seen from the code that main()the function calls add()the function, obtains the calculation result, adds it to the temporary variable a, and finally prints the value of res. add()The flow chart is as follows, and the figure shows the status of the function call stack when the function is executed

5.4.2 Application of the stack in expression evaluation

Let's look at another common application scenario of the stack, how the compiler uses the stack to implement expression evaluation

For the convenience of explanation, the arithmetic expression is simplified to only include the four arithmetic operations of addition, subtraction, multiplication and division, for example: $34+13\ast 9+44-12/3$ . For these four arithmetic operations, the human brain can quickly solve the answer, but for a computer, it is very difficult to understand the expression itself. How to implement such an expression evaluation function?

In fact, the compiler is implemented through two stacks. One holds the operand stack and the other holds the operator stack. Traversing expressions from left to right, when a number is encountered, it is directly pushed into the operand stack; when an operator is encountered, it is compared with the top element of the operator stack

If it is higher than the priority of the top element of the operator stack, push the current operator onto the stack; if it is lower or the same as the priority of the top element of the operator stack, take the top operator from the operator stack, take 2 operands from the top of the operand stack, and then perform calculations, then push the calculated results into the operand stack, and continue to compare

$3+5\ast 8-6$ The calculation process of this expression is as follows:

5.4.3 Application of the stack in parenthesis matching

You can use the stack to check whether the parentheses in the expression match

Simplify the background. Assume that the expression contains only three kinds of brackets, parentheses (), square brackets [] and curly brackets {}, and they can be nested arbitrarily. For example, {[{}]} or [{()}([])] are legal formats, but {[}()] or [({)] are illegal formats. Now there is an expression string containing three kinds of brackets, how to check whether it is legal?

Stacks can also be used here. Use a stack to store unmatched left parentheses, and scan the string from left to right. When a left parenthesis is scanned, it is pushed onto the stack; when a right parenthesis is scanned, a left parenthesis is taken from the top of the stack. If it can match, for example, "(" matches ")", "[" matches "]", "{" matches "}", then continue to scan the remaining strings. If during the scanning process, an unmatched right parenthesis is encountered, or there is no data in the stack, it means that the format is illegal

After all the brackets are scanned, if the stack is empty, it means that the string is in a legal format; otherwise, it means that there is an unmatched left bracket, which is an illegal format

5.5 How to implement the forward and backward functions of the browser?

After visiting a series of pages abc in turn, click the back button of the browser to view the previously browsed pages b and a. When you go back to page a and click the forward button, you can view pages b and c again. However, if you go back to page b and click on a new page d, you can no longer view page c through the forward and back functions

How? In fact, using two stacks can perfectly solve this problem

Use two stacks, X and Y, to push the pages browsed for the first time into stack X one by one. When the back button is clicked, they will be popped from stack X in turn, and the popped data will be put into stack Y in turn. When the forward button is clicked, the data is sequentially taken out from stack Y and put into stack X. When there is no data in the stack X, it means that there is no page to browse back. When there is no data in stack Y, it means that there is no page to browse by clicking the forward button

For example, after viewing the three pages a, b, and c in sequence, push a, b, and c onto the stack in turn. At this time, the data of the two stacks looks like this:

After going back from page c to page a through the back button of the browser, pop c and b from stack X in turn, and put them into stack Y in turn. At this time, the data of the two stacks looks like this:

At this time, I want to see page b again, so I click the forward button to return to page b, and then pop b from stack Y and put it into stack X. At this time, the data of the two stacks looks like this:

At this time, page b jumps to a new page d, and page c can no longer be viewed repeatedly through the forward and back buttons, so stack Y needs to be cleared. At this time, the data of the two stacks looks like this:

6. Queue

6.1 How to understand "queue"?

The concept of queue is very easy to understand. It can be thought of as queuing up to buy tickets, first come first buy, and those who come later can only stand at the end and are not allowed to jump in line. First in first out, this is a typical "queue"

The stack supports only two basic operations: push push()and pop pop(). The queue is very similar to the stack, and the supported operations are also very limited. The most basic operations are two: enqueue enqueue(), put a piece of data to the end of the queue; dequeue dequeue(), take an element from the head of the queue

Therefore, the queue, like the stack, is also a linear table data structure with limited operations.

The concept of queue is easy to understand, and the basic operations are easy to master. As a very basic data structure, queues are also widely used, especially some queues with some additional features, such as circular queues, blocking queues, and concurrent queues. They play a key role in the development of many low-level systems, frameworks, and middleware. For example, the high-performance queue Disruptor and the Linux ring cache all use circular concurrent queues; Java concurrent contract delivery uses ArrayBlockingQueue to implement fair locks, etc.

6.2 Sequential queues and chained queues

Like a stack, a queue is also an abstract data structure. It has the feature of first-in-first-out, and supports inserting elements at the end of the queue and deleting elements at the head of the queue. So how to implement a queue?

Like stacks, queues can be implemented using arrays or linked lists. A stack implemented with an array is called a sequential stack, and a stack implemented with a linked list is called a linked stack. Similarly, a queue implemented with an array is called a sequential queue, and a queue implemented with a linked list is called a chained queue

Let's take a look at the implementation method based on the array

// 用数组实现的队列
public class ArrayQueue {
    
    
    // 数组：items，数组大小：n
    private String[] items;
    private int n = 0;
    // head 表示队头下标，tail 表示队尾下标
    private int head = 0;
    private int tail = 0;
    // 申请一个大小为 capacity 的数组
    public ArrayQueue(int capacity) {
    
    
        items = new String[capacity];
        n = capacity;
    }
    // 入队
    public boolean enqueue(String item) {
    
    
        // 如果 tail == n 表示队列已经满了
        if (tail == n) return false;
        items[tail] = item;
        ++tail;
        return true;
    }
    // 出队
    public String dequeue() {
    
    
        // 如果 head == tail 表示队列为空
        if (head == tail) return null;
        String ret = items[head];
        ++head;
        return ret;
    }
}

For the stack, only a pointer to the top of the stack is needed. But the queue needs two pointers: one is the head pointer, pointing to the head of the queue; the other is the tail pointer, pointing to the tail of the queue

It can be understood by combining the following figure. When a, b, c, and d are enqueued in turn, the head pointer in the queue points to the position with subscript 0, and the tail pointer points to the position with subscript 4

After calling the dequeue operation twice, the head pointer in the queue points to the position with subscript 2, and the tail pointer still points to the position with subscript 4

At this time, it must have been discovered. With the continuous entry and exit operations, the head and tail will continue to move backwards. When the tail moves to the far right, even if there is still free space in the array, it is impossible to continue adding data to the queue. How to solve this problem?

In the previous section on arrays, I also encountered a similar problem, that is, the deletion operation of the array will cause the data in the array to be discontinuous. The method used at that time was data movement! However, each dequeue operation is equivalent to deleting the data with the subscript 0 in the array, and the data in the entire queue needs to be moved, so the time complexity of the dequeue operation will change from the original O ( 1 ) O(1 $O\left(1\right)$ transforms$O\left(n\right)$ . Can it be optimized?

In fact, there is no need to move data when dequeuing. If there is no free space, you only need to trigger a data movement operation centrally when entering the queue. With this idea, the dequeue function dequeue()remains unchanged, and a little modification of the implementation of the enqueue function enqueue()can easily solve the problem just now. The following is the specific code:

// 入队操作，将 item 放入队尾
public boolean enqueue(String item) {
    
    
    // tail == n 表示队列末尾没有空间了
    if (tail == n) {
    
    
        // tail ==n && head==0，表示整个队列都占满了
        if (head == 0) return false;
        // 数据搬移
        for (int i = head; i < tail; ++i) {
    
    
            items[i - head] = items[i];
        }
        // 搬移完之后重新更新 head 和 tail
        tail -= head;
        head = 0;
    }

    items[tail] = item;
    ++tail;
    return true;
}

As can be seen from the code, when the tail pointer of the queue is moved to the rightmost of the array, if new data enters the queue, the data between head and tail can be moved to the position from 0 to tail-head in the array as a whole

Let's look at the linked list-based queue implementation method. The linked list-based implementation also requires two pointers: head pointer and tail pointer. They point to the first node and the last node of the linked list respectively. As shown in the figure, when entering the team, tail -> next = new_node, tail = tail -> next; when leaving the team, head = head->next

6.3 Circular Queue

When the array is used to implement the queue, when tail==n, there will be a data movement operation, so the performance of the enqueue operation will be affected. Is there a way to avoid data migration? Let's take a look at the solution to the circular queue

A circular queue, as the name suggests, looks like a ring. The original array has a head and a tail, which is a straight line. Now connect the end to end to form a ring, as shown in the figure below

It can be seen that the size of the queue in the figure is 8, the current head=4, tail=7. When a new element a enters the queue, put it into the position with subscript 7. But at this time, instead of updating tail to 8, it is moved back one bit in the ring to the position where the subscript is 0. When another element b enters the queue, put b into the position with subscript 0, and then add 1 to tail to update it to 1. Therefore, after a and b are enqueued in sequence, the elements in the circular queue become as follows:

Through this method, the data movement operation is successfully avoided. It seems not difficult to understand, but the code implementation of the circular queue is much more difficult than the non-circular queue mentioned above. If you want to write the implementation code of a circular queue without bugs, the most important thing is to determine the conditions for judging whether the queue is empty or full

In an acyclic queue implemented by an array, the judging condition for the full queue is tail == n, and the judging condition for the empty queue is head == tail. For the circular queue, how to judge whether the queue is empty or full?

The condition for judging that the queue is empty is still head == tail. But the judging condition of the full queue is a bit more complicated, as shown in the figure below

Just like the situation in the picture where the team is full, tail=3, head=4, n=8, so to sum up the rule is: $(3+1)\%8=4$ . Draw a few more pictures of the full team, and you will find that when the team is full,$(tail+1)\%n=head$

At this time, it will be found that when the queue is full, the location pointed to by tail in the figure does not actually store data. Therefore, the circular queue will waste the storage space of an array, the code is as follows:

public class CircularQueue {
    
    
    // 数组：items，数组大小：n
    private String[] items;
    private int n = 0;
    // head 表示队头下标，tail 表示队尾下标
    private int head = 0;
    private int tail = 0;
    // 申请一个大小为 capacity 的数组
    public CircularQueue(int capacity) {
    
    
        items = new String[capacity];
        n = capacity;
    }
    // 入队
    public boolean enqueue(String item) {
    
    
        // 队列满了
        if ((tail + 1) % n == head) return false;
        items[tail] = item;
        tail = (tail + 1) % n;
        return true;
    }
    // 出队
    public String dequeue() {
    
    
        // 如果 head == tail 表示队列为空
        if (head == tail) return null;
        String ret = items[head];
        head = (head + 1) % n;
        return ret;
    }
}

6.4 Blocking queues and concurrent queues

The blocking queue is actually adding blocking operations on the basis of the queue. Simply put, when the queue is empty, fetching data from the head of the queue will be blocked. Because there is no data available at this time, it cannot return until there is data in the queue; if the queue is full, the operation of inserting data will be blocked until there is a free position in the queue before inserting data, and then returning

The above definition is actually a "producer-consumer model"! Using blocking queues, you can easily implement a "producer-consumer model"!

This "producer-consumer model" based on blocking queues can effectively coordinate the speed of production and consumption. When the "producer" produces data too fast and the "consumer" has no time to consume it, the queue for storing data will soon be full. At this time, the producer will block and wait until the "consumer" consumes the data, and the "producer" will be awakened to continue "production"

And not only that, based on the blocking queue, the data processing efficiency can also be improved by coordinating the number of "producers" and "consumers". For example, in the previous example, you can configure several "consumers" to deal with a "producer"

In the case of multi-threading, there will be multiple threads operating the queue at the same time. At this time, there will be thread safety issues. How to implement a thread-safe queue?

A thread-safe queue is called a concurrent queue . The simplest and most direct implementation method is to directly add locks to enqueue()methods dequeue(), but the concurrency will be relatively low if the lock granularity is large, and only one store or fetch operation is allowed at the same time. In fact, array-based circular queues can implement very efficient concurrent queues by using CAS atomic operations. This is why circular queues are more widely used than chained queues

6.5 Application of queues in limited resource pools such as thread pools

CPU resources are limited, and the processing speed of tasks is not linearly positively correlated with the number of threads. On the contrary, too many threads will lead to frequent switching of CPU and decrease of processing performance. Therefore, the size of the thread pool is generally set in advance considering the characteristics of the task to be processed and the hardware environment.

When requesting a thread from a fixed-size thread pool, if there are no idle resources in the thread pool, how does the thread pool handle this request at this time? Is it rejecting the request or queuing the request? How are various processing strategies implemented?

Generally, there are two treatment strategies. The first is a non-blocking processing method, which directly rejects the task request; the other is a blocking processing method, which queues the request and waits for an idle thread to take out the queued request to continue processing. So how do you store queued requests?

We hope to process each queued request fairly, and the advanced ones will be served first, so the queue data structure is very suitable for storing queued requests. As mentioned earlier, there are two implementations of queues based on linked lists and arrays. What is the difference between these two implementations for queuing requests?

Based on the implementation of the linked list, an unbounded queue (unbounded queue) that supports infinite queuing can be implemented, but it may cause too many requests to wait in line, and the response time for request processing is too long. Therefore, for systems that are sensitive to response time, an infinitely queued thread pool based on a linked list is not suitable

The bounded queue based on arrays has a limited queue size, so when the queued requests in the thread pool exceed the queue size, the subsequent requests will be rejected. This method is relatively more reasonable for systems that are sensitive to response time. However, setting a reasonable queue size is also very particular. If the queue is too large, there will be too many waiting requests; if the queue is too small, it will not be able to fully utilize system resources and maximize performance

In addition to the above-mentioned scenarios where queues are applied to queue requests in thread pools, queues can be applied to any limited resource pool for queuing requests, such as database connection pools. In fact, for most resource-limited scenarios, when there are no idle resources, the data structure of "queue" can basically be used to implement request queuing

7. Recursion

7.1 How to understand "recursion"?

Suppose you go to the cinema to watch a movie and want to know which row you are sitting in. It is too dark inside the cinema, you can’t see clearly, you can’t count, what should you do now?

So you ask the person in the front row which row he is in, just add one to his number, and you will know which row you are in. However, the person in front couldn't see clearly either, so he also asked the person in front of him. Just ask row by row until the person in the first row is asked, saying that I am in the first row, and then pass the number back row by row. Until the person in front of you tells you which row he is in, then you know the answer

This is a very standard decomposition process of recursive problem solving. The process of going is called "recursion", and the process of returning is called "return". Basically, all recursive problems can be expressed by recursive formulas. For the example just now, express it with a recursive formula like this: $f(n)=f(n-1)+1$ , where$f(1)=1$

$f\left(n\right)$ means you want to know which row you are in,$f(n-1)$ indicates the row number of the previous row,$f\left(1\right)=1$ means the people in the first row know they are in the first row. With this recursive formula, it can be easily changed to recursive code, as follows:

int f(int n) {
    
    
    if (n == 1) return 1;
    return f(n - 1) + 1;
}

7.2 Three conditions to be met for recursion

The above example is a very typical recursion, so what kind of problems can be solved with recursion? Three conditions are summarized here, as long as the following three conditions are met at the same time, it can be solved by recursion

1. The solution of a problem can be decomposed into solutions of several sub-problems

What is a subproblem? A subproblem is a problem with a smaller data size. For example, in the example of the movie theater mentioned above, you need to know that the question of "which row you are in" can be decomposed into the sub-problem of "which row is the person in the previous row"

2. This problem is the same as the sub-problem after decomposition, except for the data scale, the solution idea is exactly the same

For example, in the example of a movie theater, your way of thinking about "which row you are in" is exactly the same as the way people in the front row figure out "which row you are in".

3. There is a recursive termination condition

Decompose the problem into sub-problems, then decompose the sub-problems into sub-sub-problems, and decompose them layer by layer. There cannot be an infinite loop, which requires a termination condition

Still the example of a movie theater, the people in the first row know which row they are in without asking anyone else, that is, $f(1)=1$ , which is the termination condition for the recursion

7.3 How to write recursive code?

The key to writing recursive code is to write the recursive formula, find the termination condition , and convert the recursive formula into code is very simple

Assuming that there are n steps here, one step or two steps can be crossed each time, how many ways are there to walk these n steps? If there are 7 steps, you can go up like 2, 2, 2, 1, or 1, 2, 1, 1, 2. In short, there are many ways to move, so how to find out how many ways there are in total by programming?

If you think about it carefully, in fact, all moves can be divided into two categories according to the first step. The first type is that the first step is 1 step, and the other is that the first step is 2 steps. So the way of walking n steps is equal to the way of walking 1 step first, then n-1 steps plus the way of walking 2 steps first, then n-2 steps. Expressed in a formula: $f(n)=f(n-1)+f(n-2)$

With the recursive formula, the recursive code is basically half done. Let's look at the termination condition again. When there is a step, there is no need to continue recursion, there is only one way to go. So $f(1)=1$ . Is this recursive termination condition sufficient? You can experiment with relatively small numbers such as n=2 and n=3

n=2 时，$f(2)=f\left(1\right)+f\left(0\right)$ . If the recursive termination condition is only one$f\left(1\right)=1$ , that$f\left(2\right)$ cannot be solved. So instead of$f\left(1\right)=$In addition to the recursive termination condition of$1$$f(0)=1$ , indicating that there is a way to walk 0 steps, but this seems to be inconsistent with normal logical thinking. So, you can put$f\left(2\right)=2$ As a termination condition, it means walking 2 steps, there are two ways to walk, one step or two steps

So, the recursion termination condition is $f(1)=1$，$f(2)=2$ . At this time, you can use n=3, n=4 to verify whether the termination condition is sufficient and correct

Put together the recursive termination condition and the recursive formula just obtained as follows:

$$f\left(1\right)=1、f(2)=2、f(n)=f(n-1)+f(n-2)$$

With this formula, it is much simpler to convert into recursive code. The final recursive code is as follows:

int f(int n) {
    
    
    if (n == 1) return 1;
    if (n == 2) return 2;
    return f(n - 1) + f(n - 2);
}

The key to writing recursive code is to find the law of how to decompose a large problem into small problems, and based on this, write a recursive formula, then deliberate on the termination condition, and finally translate the recursive formula and termination condition into code

Recursive code is more difficult to understand. In the movie theater example mentioned above, there is only one branch of recursive call, that is to say, "a problem only needs to be decomposed into a sub-problem".

However, when faced with a problem that needs to be decomposed into multiple sub-problems, the recursive code is not so easy to understand. Like the second example above, the human brain has almost no way to think clearly about the whole process of "recursion" and "recursion" step by step.

Computers are good at doing repetitive things, so recursion is right up their alley. The human brain prefers a straightforward way of thinking. When we see recursion, we always want to flatten the recursion, and we will loop in our minds, adjust down one layer at a time, and then return layer by layer, trying to figure out how each step of the computer is executed, so that it is easy to get caught up in it

For recursive code, this practice of trying to figure out the entire recursive and recursive process is actually entering a misunderstanding of thinking. In many cases, it is difficult for us to understand, the main reason is that we have created this kind of understanding barrier for ourselves. What should be the correct way of thinking?

If a problem A can be decomposed into several sub-problems B, C, and D, you can assume that the sub-problems B, C, and D have been solved, and think about how to solve problem A on this basis. Moreover, you only need to think about the relationship between problem A and sub-problems B, C, and D. You don’t need to think about the relationship between sub-problems and sub-problems, sub-sub-problems and sub-sub-problems. Masking out the recursive details makes it easier to understand

Therefore, the key to writing recursive code is to abstract it into a recursive formula whenever you encounter recursion, without thinking about the layer-by-layer call relationship, and don't try to use the human brain to decompose each step of the recursion

7.4 Be wary of stack overflow in recursive code

In actual software development, when writing recursive code, you will encounter many problems, such as stack overflow. The stack overflow will cause a system crash, the consequences will be very serious. Why is recursive code prone to stack overflow? And how to prevent stack overflow?

As mentioned in the "Stack" section, function calls use the stack to store temporary variables. Every time a function is called, the temporary variable will be encapsulated as a stack frame and pushed into the memory stack, and the stack will not be popped until the function returns after execution. The system stack or virtual machine stack space is generally not large. If the data to be solved recursively has a large scale and the call level is very deep, and it is always pushed into the stack, there will be a risk of stack overflow

For example, in the movie theater example mentioned earlier, if the system stack or JVM stack size is set to 1KB, when solving $f\left(19999\right)$ , the following stack error will appear:

Exception in thread “main” java.lang.StackOverflowError

So, how to avoid stack overflow? This problem can be solved by limiting the maximum depth of recursive calls in the code. After the recursive call exceeds a certain depth (such as 1000), it will not continue to recurse further, and directly return an error

But this approach cannot completely solve the problem, because the maximum allowable recursion depth is related to the remaining stack space of the current thread, which cannot be calculated in advance. If it is calculated in real time, the code is too complex, which will affect the readability of the code. Therefore, if the maximum depth is relatively small, such as 10, 50, you can use this method, otherwise this method is not very practical

7.5 Be wary of double counting in recursive code

In addition to this, there is also the problem of double counting when using recursion. The second recursive code example mentioned above, if the whole recursive process is decomposed, it is like this:

From the figure, it can be seen intuitively that we want to calculate $f\left(5\right)$ , need to calculate f ( 4 ) f(4)first$f\left(4\right)$ and$f\left(3\right)$ , while computing$f\left(4\right)$ also needs to calculate$f\left(3\right)$ , therefore,$f\left(3\right)$ is calculated many times, this is the double calculation problem

In order to avoid repeated calculations, a data structure (such as a hash table) can be used to save the solved $f\left(k\right)$ . When the recursive call to$f\left(k\right)$ , first check whether it has been solved. If it is, it will return the value directly from the hash table without repeated calculations, so that the problem just mentioned can be avoided

public int f(int n) {
    
    
    if (n == 1) return 1;
    if (n == 2) return 2;

    // hasSolvedList 可以理解成一个 Map，key 是 n，value 是 f(n)
    if (hasSolvedList.containsKey(n)) {
    
    
        return hasSovledList.get(n);
    }

    int ret = f(n - 1) + f(n - 2);
    hasSovledList.put(n, ret);
    return ret;
}

In addition to the two common problems of stack overflow and double counting. There are many other problems with recursive code

In terms of time efficiency, there are a lot of function calls in the recursive code. When the number of these function calls is large, it will accumulate into a considerable time cost. In terms of space complexity, because a recursive call will save the field data once in the memory stack, when analyzing the space complexity of the recursive code, this part of the overhead needs to be considered. For example, the space complexity of the movie theater recursive code mentioned above is not O ( 1 ) O(1 $O\left(1\right)$ instead of$O(n)$

7.6 怎么将递归代码改写为非递归代码？

递归有利有弊，利是递归代码的表达力很强，写起来非常简洁；而弊就是空间复杂度高、有堆栈溢出的风险、存在重复计算、过多的函数调用会耗时较多等问题。所以，在开发过程中，我们要根据实际情况来选择是否需要用递归的方式来实现

那是否可以把递归代码改写为非递归代码呢？比如刚才那个电影院的例子，抛开场景，只看 $f(x)=f(x-1)+1$ 这个递推公式。这样改写看看：

int f(int n) {
    
    
    int ret = 1;
    for (int i = 2; i <= n; ++i) {
    
    
        ret = ret + 1;
    }
    return ret;
}

同样，第二个例子也可以改为非递归的实现方式

int f(int n) {
    
    
    if (n == 1) return 1;
    if (n == 2) return 2;

    int ret = 0;
    int pre = 2;
    int prepre = 1;
    for (int i = 3; i <= n; ++i) {
    
    
        ret = pre + prepre;
        prepre = pre;
        pre = ret;
    }
    return ret;
}

那是不是所有的递归代码都可以改为这种迭代循环的非递归写法呢？

笼统地讲，是的。因为递归本身就是借助栈来实现的，只不过使用的栈是系统或者虚拟机本身提供的，我们没有感知罢了。如果我们自己在内存堆上实现栈，手动模拟入栈、出栈过程，这样任何递归代码都可以改写成看上去不是递归代码的样子

但是这种思路实际上是将递归改为了“手动”递归，本质并没有变，而且也并没有解决前面讲到的某些问题，徒增了实现的复杂度

再来看一个例子，现在很多 App 都有推荐注册返佣金这个功能，这个功能中，用户 A 推荐用户 B 来注册，用户 B 又推荐了用户 C 来注册。我们可以说，用户 C的“最终推荐人”为用户 A，用户 B 的“最终推荐人”也为用户 A，而用户 A 没有“最终推荐人”

Generally, this recommendation relationship is recorded through a database. In the database table, two rows of data can be recorded, where actor_id represents the user id, and referrer_id represents the recommender id

Given a user ID, how do I find the "last referrer" for this user?

long findRootReferrerId(long actorId) {
    
    
    Long referrerId = select referrer_id from [table] where actor_id = actorId;
    if (referrerId == null) return actorId;
    return findRootReferrerId(referrerId);
}

Is it very concise? It can be done with three lines of code, but in the actual project, the above code does not work, why? There are two problems here

First, if the recursion is deep, there may be a stack overflow problem

Second, if there is dirty data in the database, it is necessary to deal with the resulting infinite recursion problem. For example, in the database in the demo environment, the test engineer will artificially insert some data for the convenience of testing, and dirty data will appear. If A's recommender is B, B's recommender is C, and C's recommender is A, an infinite loop will occur

The first problem, as mentioned earlier, can be solved by limiting the depth of recursion. The second problem can also be solved by limiting the recursion depth. However, there is a more advanced processing method, which is to automatically detect the existence of "rings" such as ABCA

8. Sort

8.1 How to analyze a "sorting algorithm"?

To learn a sorting algorithm, in addition to learning its algorithm principle and code implementation, it is more important to learn how to evaluate and analyze a sorting algorithm. Then analyze a sorting algorithm, which aspects should we start with?

1. Execution efficiency of sorting algorithm

The analysis of the execution efficiency of the sorting algorithm is generally measured from the following aspects:

1. Best case, worst case, and average case time complexity
   When analyzing the time complexity of a sorting algorithm, the time complexity of the best case, worst case, and average case should be given respectively. In addition, it is also necessary to say what the original data to be sorted corresponds to the best and worst time complexity

   Why should we distinguish between these three time complexities? First, some sorting algorithms will distinguish. For comparison, it is best to make a distinction. Second, for the data to be sorted, some are close to order, and some are completely out of order. Data with different degrees of order will definitely have an impact on the execution time of sorting. We need to know the performance of sorting algorithms under different data

2. Coefficients, constants, and low-orders of time complexity
   Time complexity reflects a growth trend when the data size n is large, so coefficients, constants, and low-orders are ignored when it is expressed. However, in actual software development, small-scale data such as 10, 100, and 1,000 may be sorted. Therefore, when comparing the performance of sorting algorithms with the same order of time complexity, coefficients, constants, and low-level data must also be taken into consideration.

3. The number of comparisons and exchanges (or moves)
   The execution process of the comparison-based sorting algorithm involves two operations, one is element comparison, and the other is element exchange or movement. Therefore, when analyzing the execution efficiency of the sorting algorithm, the number of comparisons and the number of exchanges (or moves) should also be taken into account

2. Memory consumption of sorting algorithm

The memory consumption of an algorithm can be measured by its space complexity, and sorting algorithms are no exception. However, for the space complexity of the sorting algorithm, a new concept, Sorted in place, is introduced. The in-place sorting algorithm refers specifically to the sorting algorithm whose space complexity is O(1)

3. The stability of the sorting algorithm

It is not enough to measure the quality of a sorting algorithm only by its execution efficiency and memory consumption. For sorting algorithms, there is another important metric, stability. This concept means that if there are elements with equal values ​​in the sequence to be sorted, after sorting, the original order of the equal elements remains unchanged

For example, if there is a set of data 2, 9, 3, 4, 8, 3, after sorting by size, it will be 2, 3, 3, 4, 8, 9. There are two 3s in this set of data. After sorting by a certain sorting algorithm, if the order of the two 3s does not change, the sorting algorithm is called a stable sorting algorithm; if the order changes, the corresponding sorting algorithm is called an unstable sorting algorithm

What does it matter which of the two 3s is in front and which is in the back? What does it matter if it is stable or unstable? Why should we examine the stability of the sorting algorithm?

Many data structure and algorithm courses use integers as examples when talking about sorting, but in real software development, what is to be sorted is often not a simple integer, but a set of objects, which need to be sorted according to a certain key of the object

For example, it is now necessary to sort the "orders" in the e-commerce trading system. An order has two attributes, one is the order time and the other is the order amount. If there are 100,000 pieces of order data, you want to sort the order data from small to large. For orders with the same amount, it is hoped that they will be ordered according to the order time from morning to night. For such a sorting requirement, how to do it?

The first method that comes to mind is: first sort the order data according to the amount, and then traverse the sorted order data, and then sort according to the order time for each small area with the same amount. This kind of sorting idea is not difficult to understand, but it will be very complicated to implement

With the help of the stable sort algorithm, this problem can be solved very succinctly. The solution is as follows: first sort the orders according to the time when the order was placed, and pay attention to the time when the order was placed, not the amount. After the sorting is completed, use the stable sorting algorithm to re-sort according to the order amount. After sorting twice, the obtained order data is sorted by amount from small to large, and orders with the same amount are sorted by order time from early to late. why?

The stable sorting algorithm can keep two objects with the same amount in the same order after sorting. After the first sorting, all orders are sorted from early to late according to the order time. In the second sorting, a stable sorting algorithm is used, so after the second sorting, orders with the same amount are still ordered from early to late

8.2 Bubble Sort

Bubble sort will only operate on two adjacent data. Each bubbling operation will compare two adjacent elements to see if they meet the size relationship requirements. If not, swap the two. One bubbling will move at least one element to where it should be, repeat n times, and complete the sorting of n data

For example, to sort a set of data 4, 5, 6, 3, 2, 1, from small to large. The detailed process of the first bubbling operation is as follows:

It can be seen that after a bubbling operation, the element 6 has been stored in the correct position. To complete the sorting of all data, just perform 6 such bubbling operations.

In fact, the above bubbling process can also be optimized. When there is no data exchange for a certain bubbling operation, it means that it has reached complete order, and there is no need to continue to perform subsequent bubbling operations. Here is another example, where to sort 6 elements, only 4 bubbling operations are needed

// 冒泡排序，a 表示数组，n 表示数组大小
public void bubbleSort(int[] a, int n) {
    
    
    if (n <= 1) return;
    for (int i = 0; i < n; ++i) {
    
    
        // 提前退出冒泡循环的标志位
        boolean flag = false;
        for (int j = 0; j < n - i - 1; ++j) {
    
    
            if (a[j] > a[j + 1]) {
    
     // 交换
                int tmp = a[j];
                a[j] = a[j + 1];
                a[j + 1] = tmp;
                flag = true; // 表示有数据交换
            }
        }
        if (!flag) break; // 没有数据交换，提前退出
    }
}

Now, combining the three aspects of the previous analysis of the sorting algorithm, there are three questions

1. Is bubble sort an in-place sorting algorithm?

   The bubbling process only involves the exchange operation of adjacent data, and only requires a constant level of temporary space, so its space complexity is $O\left(1\right)$ , is an in-place sorting algorithm

2. Is bubble sort a stable sorting algorithm?

   In bubble sort, only exchange can change the order of two elements. In order to ensure the stability of the bubble sorting algorithm, when there are two adjacent elements of equal size, no exchange is performed, and the data of the same size will not change the order before and after sorting, so bubble sorting is a stable sorting algorithm

3. What is the time complexity of bubble sort?

   In the best case, the data to be sorted is already in order, and only one bubbling operation is needed to finish, so the best case time complexity is O ( n ) $O\left(n\right)$ . In the worst case, the data to be sorted happens to be in reverse order, and n times of bubbling operations are required, so the worst case time complexity is$O(n^2)$

   

The time complexity of the best and worst cases is easy to analyze, so what is the time complexity of the average case? As mentioned earlier, the average time complexity is the weighted average expected time complexity, and the analysis should be combined with the knowledge of probability theory

For an array containing n data, there are n! kinds of arrangements for the n data. For different arrangements, the execution time of bubble sort must be different. For example, in the two examples mentioned above, one of them needs 6 bubbling times, while the other only needs 4 times. If the average time complexity is quantitatively analyzed by the method of probability theory, the mathematical reasoning and calculation involved will be very complicated. There is another way of thinking here, through the two concepts of "order degree" and "reverse order degree" to analyze

The degree of order is the number of pairs of elements that have an ordered relationship in the array. Ordered pairs of elements are expressed mathematically like this:

Ordered pair of elements: a[i] <= a[j], if i < j

Similarly, for an array in reverse order, such as 6, 5, 4, 3, 2, 1, the degree of order is 0; for a completely ordered array, such as 1, 2, 3, 4, 5, 6, the degree of order is $n\ast (n-1)/2$ , which is 15. The order degree of this completely ordered array is called the full order degree

The definition of reverse order is exactly the opposite of order degree (the default is order from small to large)

Pair of elements in reverse order: a[i] > a[j], if i < j

Regarding these three concepts, a formula can also be obtained: degree of reverse order = degree of full order - degree of order . The process of sorting is a process of increasing the degree of order and reducing the degree of reverse order, and finally reaching the full degree of order means that the sorting is completed

Let's take the example of bubble sort mentioned earlier to illustrate. The initial state of the array to be sorted is 4, 5, 6, 3, 2, 1, where the ordered element pairs are (4, 5) (4, 6) (5, 6), so the ordering degree is 3. n=6, so the full order degree of the final state after sorting is $n\ast (n-1)/2=15$

Bubble sort consists of two operation atoms, comparison and exchange. The degree of order is increased by 1 every time it is exchanged. No matter how the algorithm is improved, the number of exchanges is always determined, which is the degree of reverse order, that is, $n\ast (n-1)/2\; –Initial\; degree\; of\; order$ . In this example, it is 15–3=12, and 12 exchange operations are required

For bubble sorting an array containing n data, what is the average number of exchanges? In the worst case, the order degree of the initial state is 0, so $n\ast (n-1)/2$ swaps. In the best case, the order degree of the initial state is$n\ast (n-1)/2$ , there is no need for swapping. You can take an intermediate value$n\ast (n-1)/4$ , to represent the average case where the initial degree of order is neither very high nor very low

In other words, on average, $n\ast (n-1)/4$ exchange operations, the comparison operation must be more than the exchange operation, and the upper limit of the complexity is$O\left(n^2\right)$, so the average time complexity is$O(n^2)$

This average time complexity derivation process is actually not strict, but it is very practical in many cases. After all, the quantitative analysis of probability theory is too complicated and not easy to use.

8.3 Insertion Sort

Let's look at a question first. An ordered array, after adding a new data to it, how to keep the data in order? It's very simple, just traverse the array, find the position where the data should be inserted and insert it

This is a dynamic sorting process, that is, dynamically adding data to the sorted collection. This method can keep the data in the collection in order. For a set of static data, you can also learn from the insertion method mentioned above to sort, so there is an insertion sort algorithm

So how does insertion sorting use the above ideas to achieve sorting?

First, divide the data in the array into two intervals, sorted and unsorted. The initial sorted range has only one element, which is the first element of the array. The core idea of ​​the insertion algorithm is to take the elements in the unsorted interval, find a suitable insertion position in the sorted interval and insert it, and ensure that the data in the sorted interval is always in order. Repeat this process until the elements in the unsorted interval are empty, and the algorithm ends

As shown in the figure, the data to be sorted are 4, 5, 6, 1, 3, 2, where the left side is the sorted range and the right side is the unsorted range

Insertion sort also includes two operations, one is the comparison of elements, and the other is the movement of elements. When it is necessary to insert a piece of data a into the sorted range, it is necessary to compare the size of a with the elements of the sorted range in order to find a suitable insertion position. After finding the insertion point, you need to move the order of the elements after the insertion point back by one bit, so as to make room for the insertion of element a

For different methods of finding the insertion point (from head to tail, from tail to head), the number of comparisons of elements is different. But for a given initial sequence, the number of moving operations is always fixed, which is equal to the degree of reverse order

Why do you say that the number of moves is equal to the degree of reverse order? As shown in the figure below, the degree of full order is $n\ast (n-1)/2=15$ , the degree of order of the initial sequence is 5, so the degree of reverse order is 10. In insertion sorting, the sum of the number of data movements is also equal to 10=3+3+4

// 插入排序，a 表示数组，n 表示数组大小
public void insertionSort(int[] a, int n) {
    
    
    if (n <= 1) return;
    for (int i = 1; i < n; ++i) {
    
    
        int value = a[i];
        int j = i - 1;
        // 查找插入的位置
        for (; j >= 0; --j) {
    
    
            if (a[j] > value) {
    
    
                a[j + 1] = a[j]; // 数据移动
            } else {
    
    
                break;
            }
        }
        a[j + 1] = value; // 插入数据
    }
}

Analyzing the insertion sort algorithm, there are also three problems

1. Is insertion sort an in-place sorting algorithm?

   It is obvious from the implementation process that the operation of the insertion sort algorithm does not require additional storage space, so the space complexity is $O\left(1\right)$ , that is, this is an in-place sorting algorithm

2. Is insertion sort a stable sorting algorithm?

   In insertion sorting, for elements with the same value, you can choose to insert the elements that appear later to the back of the elements that appear before, so that the original order can be kept unchanged, so insertion sorting is a stable sorting algorithm

3. What is the time complexity of insertion sort?

   If the data to be sorted is already in order, there is no need to move any data. If you search for the insertion position in the ordered data group from the end to the beginning, you only need to compare one data at a time to determine the insertion position. So in this case, the best time complexity is $O\left(n\right)$ . Note that here is to traverse the ordered data from tail to head

   If the array is in reverse order, each insertion is equivalent to inserting new data at the first position of the array, so a large amount of data needs to be moved, so the worst case time complexity is $O(n^2)$

The average time complexity of inserting a data in an array is $O\left(n\right)$ . Therefore, for insertion sorting, each insertion operation is equivalent to inserting a data in the array, and the loop performs n insertion operations, so the average time complexity is$O(n^2)$

8.4 Selection Sort

The implementation idea of ​​the selection sorting algorithm is somewhat similar to insertion sorting, and it is also divided into sorted intervals and unsorted intervals. But the selection sort will find the smallest element from the unsorted interval every time, and put it at the end of the sorted interval

Selection sort space complexity is $O\left(1\right)$ , is an in-place sorting algorithm. The best case time complexity, worst case time complexity and average case time complexity of selection sort are$O\left(n^2\right)$

Is selection sort a stable sorting algorithm?

The answer is no, selection sort is an unstable sorting algorithm. As can be seen from the above figure, the selection sort must find the minimum value among the remaining unsorted elements every time, and exchange positions with the previous elements, which destroys the stability

For example, if a group of data such as 5, 8, 5, 2, and 9 is sorted using the selection sort algorithm, the smallest element 2 is found for the first time, and the position is exchanged with the first 5, then the order of the first 5 and the middle 5 will change, so it will be unstable. Because of this, selection sort is slightly inferior to bubble sort and insertion sort

Why is insertion sort more popular than bubble sort?

The time complexity of both bubble sort and insertion sort is $O\left(n^2\right)$Both are in-place sorting algorithms. Why is insertion sorting more popular than bubble sorting?

When analyzing bubble sorting and insertion sorting, we mentioned that no matter how bubble sorting is optimized, the number of element exchanges is a fixed value, which is the reverse order of the original data. Insertion sorting is the same, no matter how optimized, the number of elements moved is equal to the reverse order of the original data

However, from the perspective of code implementation, the data exchange of bubble sorting is more complicated than the data movement of insertion sorting. Bubble sorting requires three assignment operations, while insertion sorting only needs one. Take a look at this operation:

冒泡排序中数据的交换操作：
if (a[j] > a[j + 1]) {
    
     // 交换
    int tmp = a[j];
    a[j] = a[j + 1];
    a[j + 1] = tmp;
    flag = true;
}
插入排序中数据的移动操作：
if (a[j] > value) {
    
    
    a[j + 1] = a[j]; // 数据移动
} else {
    
    
    break;
}

The time to execute an assignment statement is roughly calculated as unit time (unit_time), and then use bubble sorting and insertion sorting to sort the same array with a reverse order of K. Bubble sorting requires K times of exchange operations, each requiring 3 assignment statements, so the total time spent on exchange operations is 3*K unit time. However, the data movement operation in insertion sort only needs K unit time

This is just a very theoretical analysis. For the sake of experimentation, I wrote a performance comparison test program for the above Java codes of bubble sorting and insertion sorting, randomly generating 10,000 arrays, each containing 200 data, and then using the bubble sorting and insertion sorting algorithms to sort on my machine. The bubble sorting algorithm takes about 700ms to complete, while the insertion sorting only takes about 100ms to complete!

So, although bubble sort and insertion sort have the same time complexity, both are $O\left(n^2\right)$, but if you want to achieve the ultimate performance optimization, then insertion sort is definitely the first choice. The algorithm idea of ​​insertion sort also has a lot of room for optimization. The previous one is just the most basic one. If you are interested in the optimization of insertion sort, you can learn Hill sort by yourself

8.5 Merge Sort

If you want to sort an array, first divide the array into front and back parts from the middle, then sort the front and back parts separately, and then merge the sorted two parts together, so that the whole array is in order

Merge sort uses the idea of ​​divide and conquer. Divide and conquer, as the name suggests, is to divide and conquer, decomposing a big problem into small sub-problems to solve. When the small sub-problems are solved, the big problem is also solved

The idea of ​​divide and conquer is very similar to the recursive idea mentioned earlier. Divide and conquer algorithms are generally implemented using recursion. Divide and conquer is a processing idea for solving problems, and recursion is a programming technique, and the two do not conflict. Next, let's take a look at how to use recursive code to implement merge sort

The trick to writing recursive code is to analyze the recursive formula, then find the termination condition, and finally translate the recursive formula into recursive code. Therefore, if you want to write the code for merge sort, first write the recursive formula for merge sort

Recursion formula: merge_sort(p…r) = merge(merge_sort(p…q), merge_sort(q+1…r))
Termination condition: p >= r No need to continue to decompose

merge_sort(p…r) means to sort the array with subscripts from p to r. Transform this sorting problem into two sub-problems, merge_sort(p...q) and merge_sort(q+1...r), where the subscript q is equal to the middle position of p and r, that is, ( p + r ) / 2 (p+r)/ $(p+r)/2$ . After the two sub-arrays with subscripts from p to q and from q+1 to r are sorted, merge the two sorted sub-arrays together, so that the data between subscripts from p to r is also sorted. The translation into pseudocode is as follows:

// 归并排序算法, A 是数组，n 表示数组大小
merge_sort(A, n) {
    
    
    merge_sort_c(A, 0, n-1)
}
// 递归调用函数
merge_sort_c(A, p, r) {
    
    
    // 递归终止条件
    if p >= r then return
    // 取 p 到 r 之间的中间位置 q
    q = (p+r) / 2
    // 分治递归
    merge_sort_c(A, p, q)
    merge_sort_c(A, q+1, r)
    // 将 A[p...q] 和 A[q+1...r] 合并为 A[p...r]
    merge(A[p...r], A[p...q], A[q+1...r])
}

merge(A[p…r], A[p…q], A[q+1…r]) The function of this function is to merge the ordered A[p…q] and A[q+1…r] into an ordered array and put it into A[p…r]. So how should this process be done?

As shown in the figure, apply for a temporary array tmp with the same size as A[p…r]. Use two cursors i and j, pointing to the first element of A[p…q] and A[q+1…r] respectively. Compare these two elements A[i] and A[j], if A[i]<=A[j], put A[i] into the temporary array tmp, and move i one bit behind, otherwise put A[j] into the array tmp, and move j one bit behind

Continue the above comparison process until all the data in one of the sub-arrays is put into the temporary array, and then add the data in the other array to the end of the temporary array in turn. At this time, what is stored in the temporary array is the result of merging the two sub-arrays. Finally, copy the data in the temporary array tmp to the original array A[p...r]

Write merge()the function as pseudocode, which is as follows:

merge(A[p...r], A[p...q], A[q + 1...r]) {
    
    
    var i : = p，j : = q + 1，k : = 0 // 初始化变量 i, j, k
    var tmp : = new array[0...r - p] // 申请一个大小跟 A[p...r] 一样的临时数组
    while i <= q AND j <= r do {
    
    
        if A[i] <= A[j] {
    
    
        	tmp[k++] = A[i++] // i++ 等于 i:=i+1
        } else {
    
    
        	tmp[k++] = A[j++]
    	}
    }

    // 判断哪个子数组中有剩余的数据
    var start : = i，end : = q
    if j <= r then start : = j, end: = r
	// 将剩余的数据拷贝到临时数组 tmp
    while start <= end do {
    
    
    	tmp[k++] = A[start++]
    }

    // 将 tmp 中的数组拷贝回 A[p...r]
    for i : = 0 to r - p do {
    
    
    	A[p + i] = tmp[i]
    }
}

1. Is merge sort a stable sorting algorithm?

The key to the stability and instability of merge sort depends on merge()the function, that is, the part of the code that merges two ordered subarrays into an ordered array. During the merge process, if there are elements with the same value between A[p…q] and A[q+1…r], you can first put the elements in A[p…q] into the tmp array as in the pseudocode. This ensures that elements with the same value are in the same order before and after merging. Therefore, merge sort is a stable sorting algorithm

2. What is the time complexity of merge sort?

Merge sort involves recursion, and the analysis of time complexity is a bit complicated. The applicable scenario of recursion is that a problem a can be decomposed into multiple sub-problems b and c, then solving problem a can be decomposed into solving problems b and c. After problems b and c are solved, we merge the results of b and c into the result of a

If we define the time to solve problem a as $T\left(a\right)$ , the time to solve problems b and c are respectively$T\left(b\right)$ and$T\left(c\right)$ , then I can get the recurrence relation:$T(a)=T(b)+T(c)+K$ , where K is equal to the time it takes to combine the results of the two subproblems b and c into the result of problem a

From the above analysis, an important conclusion can be drawn: not only the recursive solution problem can be written as a recursive formula, but also the time complexity of the recursive code can be written as a recursive formula . Apply this formula to analyze the time complexity of merge sort

Assume that the time required to merge and sort n elements is $T\left(n\right)$ , the time to decompose into two sub-arrays is$T(n/2)$ . merge()The time complexity of the function to merge two sorted subarrays is$O\left(n\right)$ . Therefore, applying the previous formula, the calculation formula for the time complexity of merge sort is:

T(1) = C; when n=1, only constant-level execution time is required, so it is expressed as C
T(n) = 2 * T(n/2) + n; n>1

Through this formula, how to solve $What\; about\; T\left(n\right)$ ? Not intuitive enough? Then let's break down the calculation process further

T(n) = 2 * T(n/2) + n
= 2 *(2 * T(n/4) + n/2) + n = 4 * T(n/4) + 2 * n
= 4 * (2 * T(n/8) + n/4) + 2 * n = 8 * T(n/8) + 3 * n
= 8 *( 2 * T(n/16) + n/8) + 3 * n = 16 * T(n/16) + 4 * n
…
= 2^k * T(n/2^k) + k * n
…

By decomposing and deriving step by step, we can get $T(n)=2^{k}T(n/2^k)+kn$。当$T(n/2^k)=When\; T\left(1\right)$ , that is$n/2^k=1$ , get$k=lo{g}_{2}n$ . Substituting the value of k into the above formula yields$T(n)=Cn+nlo{g}_{2}n$ . In big O notation,$T\left(n\right)$ is equal to$O\left(nlogn\right)$ .$\_$$\_$So the time complexity of merge sort is$O(nlogn)$

It can be seen from the principle analysis and pseudo-code that the execution efficiency of merge sort has nothing to do with the ordering degree of the original array to be sorted, so its time complexity is very stable, no matter it is the best case, the worst case, or the average case, the time complexity is O ( nlogn ) O(nlogn $O\left(nlogn\right)$

3. What is the space complexity of merge sort?

The time complexity of merge sort is $O\left(nlogn\right),$ looks pretty good$.$Even with quicksort, in the worst case, the time complexity is$O\left(n^2\right)$. However, merge sort is not as widely used as quick sort. Why? Because it has a fatal "weakness", that is, merge sorting is not an in-place sorting algorithm

This is because the merge function of merge sort requires additional storage space when merging two sorted arrays into one sorted array. What is the space complexity of merge sort? is $O\left(n\right)$ ,$O\left(nlogn\right),$ how to analyze it$?$

If you continue to follow the method of analyzing recursive time complexity and solve it through the recursive formula, the space complexity required for the entire merging process is $O\left(nlogn\right)$ .$\_$$\_$However, is it right to analyze space complexity similarly to analyzing time complexity?

Actually, the space complexity of recursive code doesn't add up like the time complexity. I just forgot the most important point, that is, although each merge operation needs to apply for additional memory space, after the merge is completed, the temporarily opened memory space is released. At any time, the CPU will only have one function executing, and therefore only one temporary memory space in use. The maximum temporary memory space will not exceed the size of n data, so the space complexity is $O\left(n\right)$

8.6 Quick Sort (Quick Sort)

Quick sorting also uses the idea of ​​​​divide and conquer. At first glance, it looks a bit like merge sort, but the idea is completely different

快排的核心思想：如果要排序数组中下标从 p 到 r 之间的一组数据，选择 p 到 r 之间的任意一个数据作为 pivot（分区点）

遍历 p 到 r 之间的数据，将小于 pivot 的放到左边，将大于 pivot 的放到右边，将 pivot 放到中间。经过这一步骤之后，数组 p 到 r 之间的数据就被分成了三个部分，前面 p 到 q-1 之间都是小于 pivot 的，中间是 pivot，后面的 q+1 到 r 之间是大于 pivot 的

根据分治、递归的处理思想，可以用递归排序下标从 p 到 q-1 之间的数据和下标从 q+1 到 r 之间的数据，直到区间缩小为 1，就说明所有的数据都有序了

如果用递推公式来将上面的过程写出来的话，就是这样：

递推公式：
quick_sort(p…r) = quick_sort(p…q-1) + quick_sort(q+1, r)
终止条件：
p >= r

将递推公式转化成递归代码：

// 快速排序，A 是数组，n 表示数组的大小
quick_sort(A, n) {
    
    
 quick_sort_c(A, 0, n-1)
}
// 快速排序递归函数，p,r 为下标
quick_sort_c(A, p, r) {
    
    
 if p >= r then return
 
 q = partition(A, p, r) // 获取分区点
 quick_sort_c(A, p, q-1)
 quick_sort_c(A, q+1, r)
}

归并排序中有一个 merge() 合并函数，这里有一个 partition() 分区函数。partition() 分区函数实际上前面已经讲过了，就是随机选择一个元素作为 pivot（一般情况下，可以选择 p 到 r 区间的最后一个元素），然后对 A[p…r] 分区，函数返回 pivot 的下标

如果不考虑空间消耗的话，partition() 分区函数可以写得非常简单。申请两个临时数组 X 和 Y，遍历 A[p…r]，将小于 pivot 的元素都拷贝到临时数组 X，将大于 pivot 的 元素都拷贝到临时数组 Y，最后再将数组 X 和数组 Y 中数据顺序拷贝到 A[p…r]

但是，如果按照这种思路实现的话，partition() 函数就需要很多额外的内存空间，所以快排就不是原地排序算法了。如果希望快排是原地排序算法，那它的空间复杂度得是 O(1)，那 partition() 分区函数就不能占用太多额外的内存空间，就需要在 A[p…r] 的 原地完成分区操作

原地分区函数的实现思路非常巧妙，伪代码如下：

partition(A, p, r) {
    
    
  pivot := A[r]
  i := p
  for j := p to r-1 do {
    
    
    if A[j] < pivot {
    
    
      swap A[i] with A[j]
      i := i+1
    }
  }
  swap A[i] with A[r]
  return i
 }

The processing here is somewhat similar to selection sort. We divide A[p…r-1] into two parts by cursor i. The elements of A[p…i-1] are all smaller than pivot, we call it “processed interval” for now, and A[i…r-1] is “unprocessed interval”. Every time we take an element A[j] from the unprocessed interval A[i...r-1], compare it with the pivot, if it is smaller than the pivot, then add it to the end of the processed interval, which is the position of A[i]

Inserting an element at a certain position in the array requires moving data, which is very time-consuming. But we also talked about a processing technique, that is, exchange, and the insertion operation is completed within the time complexity of O(1). Here we also use this idea, only need to exchange A[i] and A[j], we can put A[j] in the position of subscript i within O(1) time complexity

Because the partitioning process involves exchange operations, if there are two identical elements in the array, such as the sequence 6, 8, 7, 6, 3, 5, 9, 4, after the first partition operation, the relative order of the two 6s will change. Therefore, quicksort is not a stable sorting algorithm

Performance Analysis of Quick Sort

Quicksort is also implemented using recursion. For the time complexity of recursive code, the formula summarized above is still applicable here. If each partition operation can just divide the array into two small areas with nearly equal sizes, then the time complexity recursive solution formula of quick sort is the same as that of merge. Therefore, the time complexity of quick sorting is also O(nlogn)

T(1) = C；   n=1 时，只需要常量级的执行时间，所以表示为 C。
T(n) = 2*T(n/2) + n； n>1

However, the premise of the formula is that for each partition operation, the pivot we choose is very suitable, which can just divide the large interval into two equally. But in practice, this situation is difficult to achieve

Take an extreme example. If the data in the array is already ordered, such as 1, 3, 5, 6, 8. If we choose the last element as the pivot every time, the two intervals obtained by each partition are not equal. We need to perform about n partition operations to complete the whole process of quick sort. In each partition, we need to scan about n/2 elements on average. In this case, the time complexity of quick sort degenerates from O(nlogn) to O ( n 2 ) $O(n^2)$

I just talked about the time complexity in two extreme cases, one is that the partition is extremely balanced, and the other is that the partition is extremely unbalanced. They correspond to the best-case time complexity and worst-case time complexity of quicksort respectively. What is the average time complexity of quick sorting?

Assume that each partition operation divides the interval into two small intervals with a size of 9:1. If we continue to apply the recursive formula of recursive time complexity, it will become like this:

T(1) = C;  n=1 时，只需要常量级的执行时间，所以表示为 C。
 
T(n) = T(n/10) + T(9*n/10) + n； n>1

The process of recursively solving this formula is very complicated. Although it can be solved, this method is not recommended. In fact, in addition to the recursive formula, the recursive time complexity solution method also has a recursive tree. Here is a direct conclusion for you: T(n) can achieve O(nlogn) time complexity in most cases, and only in extreme cases will it degenerate to O ( n 2 ) O(n^2 $O\left(n^2\right)$. Moreover, we also have many ways to reduce this probability to a very low

Both quicksort and merge use the idea of ​​divide and conquer, and the recursive formula and recursive code are also very similar, so what is the difference between them?

It can be found that the processing process of merge sort is from bottom to top , first processing sub-problems, and then merging. The quick sort is just the opposite. Its processing process is from top to bottom , first partitioning, and then processing sub-problems. Although merge sort is a stable sorting algorithm with a time complexity of O(nlogn), it is an out-of-place sorting algorithm. As we said earlier, the main reason why merge is an out-of-place sorting algorithm is that the merge function cannot be executed in place. Quick sort can achieve in-place sorting by designing ingenious in-place partition functions, which solves the problem of merging and sorting taking up too much memory

How to find the Kth largest element in an unsorted array in O(n) time complexity?

The core idea of ​​quicksort is divide and conquer and partition . We can use the idea of ​​partition to answer this question: find the Kth largest element in the unordered array in O(n) time complexity. For example, for a set of data such as 4, 2, 5, 12, 3, the third largest element is 4

We select the last element A[n-1] of the array interval A[0…n-1] as the pivot, and partition the array A[0…n-1] in situ, so that the array is divided into three parts, A[0…p-1], A[p], A[p+1…n-1]

If p+1=K, then A[p] is the element to be solved; if K>p+1, it means that the Kth largest element appears in the interval of A[p+1…n-1], and we recursively search in the interval of A[p+1…n-1] according to the above ideas. Similarly, if K<p+1, then we search in the interval A[0…p-1]

Let's look at it again, why is the time complexity of the above solutions O(n)?

For the first partition lookup, we need to perform a partition operation on an array of size n, and we need to traverse n elements. For the second partition lookup, we only need to perform a partition operation on an array of size n/2, and we need to traverse n/2 elements. By analogy, the number of partition traversal elements are respectively, n/2, n/4, n/8, n/16... until the interval is reduced to 1

If we add up the number of elements traversed by each partition, it is: n+n/2+n/4+n/8+...+1. This is a summation of a geometric sequence, and the final sum is equal to 2n-1. Therefore, the time complexity of the above solution is O(n)

You might say, I have a stupid way to take the minimum value in the array each time, move it to the front of the array, and then continue to find the minimum value in the remaining array, and so on, execute K times, isn’t the found data the Kth largest element?

However, the time complexity is not O(n), but O(K * n). You might say, isn't the coefficient in front of the time complexity negligible? Isn't O(K * n) equal to O(n)?

This cannot be so simple. When K is a relatively small constant, such as 1, 2, the best time complexity is indeed O(n); but when K is equal to n/2 or n, the worst case time complexity is O ( n 2 ) O(n^2 $O\left(n^2\right)$up

9. Linear sort

There are three sorting algorithms whose time complexity is O(n): bucket sort, counting sort, and radix sort. Because the time complexity of these sorting algorithms is linear, we call this type of sorting algorithm Linear sort. The main reason why linear time complexity can be achieved is that these three algorithms are non-comparison-based sorting algorithms and do not involve comparison operations between elements.

9.1 Bucket Sort

Bucket sorting, as the name suggests, uses "buckets". The core idea is to divide the data to be sorted into several ordered buckets, and then sort the data in each bucket separately. After the buckets are sorted, the data in each bucket is taken out in sequence, and the formed sequence is in order

Why is the time complexity of bucket sorting O(n)?

If there are n data to be sorted, we divide them evenly into m buckets, and there are k=n/m elements in each bucket. Quick sort is used inside each bucket, and the time complexity is O(k * logk). The time complexity of sorting m buckets is O(m * k * logk), because k=n/m, so the time complexity of sorting the entire bucket is O(n*log(n/m)). When the number m of buckets is close to the number n of data, log(n/m) is a very small constant. At this time, the time complexity of bucket sorting is close to O(n)

Bucket sorting looks very good, so can it replace the sorting algorithm we talked about before?

The answer is of course no. In fact, bucket sorting is very demanding on the data to be sorted

First of all, the data to be sorted needs to be easily divided into m buckets, and there is a natural order of size between buckets. In this way, after the data in each bucket is sorted, the data between buckets does not need to be sorted

Second, the distribution of data among buckets is relatively uniform. If after the data is divided into buckets, some buckets contain a lot of data, some are very small, and very uneven, then the time complexity of sorting the data in the bucket is not constant. In extreme cases, if the data is all divided into one bucket, it degenerates into an O(nlogn) sorting algorithm

Bucket sorting is more suitable for external sorting . The so-called external sorting means that the data is stored in an external disk. The amount of data is relatively large, and the memory is limited. It is impossible to load all the data into the memory.

For example, we have 10GB of order data, and we want to sort by the order amount (assuming the amount is a positive integer), but our memory is limited, only a few hundred MB, and there is no way to load all 10GB of data into the memory at one time. What should we do at this time?

We can scan the file first to see the data range of the order amount. Assume that after scanning, we get that the minimum order amount is 1 yuan and the maximum is 100,000 yuan. We divide all orders into 100 buckets according to the amount. In the first bucket we store orders with an amount between 1 yuan and 1,000 yuan, in the second bucket we store orders with an amount between 1,001 yuan and 2,000 yuan, and so on. Each bucket corresponds to a file, and is numbered and named according to the size of the amount range (00, 01, 02...99)

Ideally, if the order amount is evenly distributed between 10,000 and 100,000, the order will be evenly divided into 100 files, and each small file will store about 100MB of order data. We can put these 100 small files in memory one by one, and use quick sort to sort them. After all the files are sorted, we only need to read the order data in each small file in order according to the file number from small to large, and write it into a file, then this file stores the order data sorted by the amount from small to large

However, orders are not necessarily evenly distributed according to the amount between 1 yuan and 100,000 yuan, so the 10GB order data cannot be evenly divided into 100 files. It is possible that there is a lot of data in a certain amount range, and the corresponding file will be very large after division, which cannot be read into the memory at one time. What should we do?

For files that are still relatively large after these divisions, we can continue to divide them. For example, if the order amount is more between 1 yuan and 1000 yuan, we will continue to divide this range into 10 small areas, 1 yuan to 100 yuan, 101 yuan to 200 yuan, 201 yuan to 300 yuan ... 901 yuan to 1000 yuan. If after dividing, there are still too many orders between 101 yuan and 200 yuan to be read into the memory at one time, then continue to divide until all the files can be read into the memory

9.2 Counting Sort

Counting sort is actually a special case of bucket sort . When the range of n data to be sorted is not large, for example, the maximum value is k, we can divide the data into k buckets. The data values ​​in each bucket are the same, which saves the time of sorting in the bucket

We have all experienced the college entrance examination. Do you remember the scoring system of the college entrance examination? When we check the score, the system will display our score and the ranking of the province we are in. If there are 500,000 candidates in your province, how to get the ranking through quick sorting of results?

The full score of candidates is 900, and the minimum is 0. The range of this data is very small, so we can divide it into 901 buckets, corresponding to scores from 0 to 900. According to the candidates' scores, we divide the 500,000 candidates into these 901 buckets. The data in the buckets are candidates with the same score, so there is no need to sort them again. We only need to scan each bucket in turn, and output the candidates in the bucket into an array in turn, and the sorting of 500,000 candidates is realized. Because only scan traversal operations are involved, the time complexity is O(n)

The algorithm idea of ​​counting sort is so simple. It is very similar to bucket sorting, except that the size and granularity of the buckets are different. But why is this sorting algorithm called "counting" sorting? Where does the meaning of "count" come from?

To understand this problem, we have to look at the implementation of the counting sort algorithm. Take the student as an example to explain. Assume there are only 8 candidates with scores between 0 and 5. We put the scores of these 8 candidates in an array A[8], they are: 2, 5, 3, 0, 2, 3, 0, 3

考生的成绩从 0 到 5 分，我们使用大小为 6 的数组 C[6] 表示桶，其中下标对应分数。不过，C[6] 内存储的并不是考生，而是对应的考生个数。像我刚刚举的那个例子，我们只需要遍历一遍考生分数，就可以得到 C[6] 的值

从图中可以看出，分数为 3 分的考生有 3 个，小于 3 分的考生有 4 个，所以，成绩为 3 分的考生在排序之后的有序数组 R[8] 中，会保存下标 4，5，6 的位置

那我们如何快速计算出，每个分数的考生在有序数组中对应的存储位置呢？这个处理方法非常巧妙

我们对 C[6] 数组顺序求和，C[6] 存储的数据就变成了下面这样子。C[k] 里存储小于等于分数 k 的考生个数

我们从后到前依次扫描数组 A。比如，当扫描到 3 时，我们可以从数组 C 中取出下标为 3 的值 7，也就是说，到目前为止，包括自己在内，分数小于等于 3 的考生有 7 个，也就是说 3 是数组 R 中的第 7 个元素（也就是数组 R 中下标为 6 的位置）。当 3 放入到数组 R 中后，小于等于 3 的元素就只剩下了 6 个了，所以相应的 C[3] 要减 1，变成 6

以此类推，当我们扫描到第 2 个分数为 3 的考生的时候，就会把它放入数组 R 中的第 6 个元素的位置（也就是下标为 5 的位置）。当我们扫描完整个数组 A 后，数组 R 内的数据就是按照分数从小到大有序排列的了

代码如下：

// 计数排序，a 是数组，n 是数组大小。假设数组中存储的都是非负整数。
public void countingSort(int[] a, int n) {
    
    
  if (n <= 1) return;
 
  // 查找数组中数据的范围
  int max = a[0];
  for (int i = 1; i < n; ++i) {
    
    
    if (max < a[i]) {
    
    
      max = a[i];
    }
  }
 
  int[] c = new int[max + 1]; // 申请一个计数数组 c，下标大小 [0,max]
  for (int i = 0; i <= max; ++i) {
    
    
    c[i] = 0;
  }
 
  // 计算每个元素的个数，放入 c 中
  for (int i = 0; i < n; ++i) {
    
    
    c[a[i]]++;
  }
 
  // 依次累加
  for (int i = 1; i <= max; ++i) {
    
    
    c[i] = c[i-1] + c[i];
  }
 
  // 临时数组 r，存储排序之后的结果
  int[] r = new int[n];
  // 计算排序的关键步骤，有点难理解
  for (int i = n - 1; i >= 0; --i) {
    
    
    int index = c[a[i]]-1;
    r[index] = a[i];
    c[a[i]]--;
  }
 
  // 将结果拷贝给 a 数组
  for (int i = 0; i < n; ++i) {
    
    
    a[i] = r[i];
  }
}

这种利用另外一个数组来计数的实现方式是不是很巧妙呢？这也是为什么这种排序算法叫计数排序的原因

计数排序只能用在数据范围不大的场景中，如果数据范围 k 比要排序的数据 n 大很多，就不适合用计数排序了。而且，计数排序只能给非负整数排序，如果要排序的数据是其他类型的，要将其在不改变相对大小的情况下，转化为非负整数

Take the example of students. If the examinee's score is accurate to one decimal place, we need to multiply all the scores by 10, convert them into integers, and then put them into 9010 buckets. For another example, if there are negative numbers in the data to be sorted, and the range of the data is [-1000, 1000], then we need to add 1000 to each data to convert it into a non-negative integer

9.3 Radix Sort

Suppose we have 100,000 mobile phone numbers and want to sort these 100,000 mobile phone numbers from small to large. Do you have any faster sorting method?

The quick sort we talked about before can achieve O(nlogn) time complexity. Is there a more efficient sorting algorithm? Can bucket sorting and counting sorting come in handy? The mobile phone number has 11 digits, and the range is too large, so it is obviously not suitable for these two sorting algorithms. For this sorting problem, is there any algorithm whose time complexity is O(n)?

There is such a rule in the question just now: Suppose you want to compare the size of two mobile phone numbers a and b, if in the first few digits, the mobile phone number of a is already larger than the mobile phone number of b, then you don’t need to look at the last few digits

With the help of the stable sorting algorithm, here is a clever implementation idea. First sort the mobile phone numbers according to the last digit, then reorder according to the penultimate digit, and so on, and finally reorder according to the first digit. After 11 sorts, the phone numbers are all in order

Note that if the sorting algorithm for sorting by bit is stable, otherwise this implementation idea is incorrect. Because if it is an unstable sorting algorithm, the last sorting will only consider the size order of the highest bit, regardless of the size relationship of other bits, then the low bit sorting is completely meaningless

To sort according to each bit, we can use the bucket sort or count sort just mentioned, and their time complexity can be O(n). If the data to be sorted has k bits, then we need k times of bucket sorting or counting sorting, and the total time complexity is O(k*n). When k is not large, such as the example of mobile phone number sorting, the maximum k is 11, so the time complexity of radix sorting is approximately O(n)

In fact, sometimes the data to be sorted is not all of the same length. For example, when we sort the 200,000 English words in the Oxford dictionary, the shortest is only 1 letter, and the longest is 45 letters. For such data of unequal length, is radix sort still applicable?

实际上，我们可以把所有的单词补齐到相同长度，位数不够的可以在后面补“0”，因为根据 ASCII 值，所有字母都大于“0”，所以补“0”不会影响到原有的大小顺序。这样就可以继续用基数排序了

基数排序对要排序的数据是有要求的，需要可以分割出独立的“位”来比较，而且位之间有递进的关系，如果 a 数据的高位比 b 数据大，那剩下的低位就不用比较了。除此之外，每一位的数据范围不能太大，要可以用线性排序算法来排序，否则，基数排序的时间复杂度就无法做到 O(n) 了

如何根据年龄给 100 万用户排序？

实际上，根据年龄给 100 万用户排序，就类似按照成绩给 50 万考生排序。我们假设年龄的范围最小 1 岁，最大不超过 120 岁。我们可以遍历这 100 万用户，根据年龄将其划分到这 120 个桶里，然后依次顺序遍历这 120 个桶中的元素。这样就得到了按照年龄排序的 100 万用户数据

如何实现一个通用的、高性能的排序函数？

如何选择合适的排序算法？

线性排序算法的时间复杂度比较低，适用场景比较特殊。所以如果要写一个通用的排序函数，不能选择线性排序算法

如果对小规模数据进行排序，可以选择时间复杂度是 $O(n^2)$ 的算法；如果对大规模数据进行排序，时间复杂度是 O(nlogn) 的算法更加高效。所以，为了兼顾任意规模数据的排序，一般都会首选时间复杂度是 O(nlogn) 的排序算法来实现排序函数

时间复杂度是 O(nlogn) 的排序算法不止一个，比如归并排序、快速排序，还有堆排序。堆排序和快速排序都有比较多的应用，比如 Java 语言采用堆排序实现排序函数，C 语言使用快速排序实现排序函数

I don’t know if you have discovered that there are not many cases where merge sort is used. We know that the time complexity of quicksort in the worst case is $O\left(n^2\right)$, and merge sorting can achieve the time complexity of O(nlogn) in the average case and the worst case. From this point of view, it looks very attractive, so why hasn't it been "favored"?

Merge sort is not an in-place sorting algorithm, and its space complexity is O(n). So, roughly speaking, if you want to sort 100MB of data, in addition to the memory occupied by the data itself, the sorting algorithm will take up an additional 100MB of memory space, and the space consumption will double.

Quick sorting is more suitable to implement sorting functions, but we also know that the time complexity of quick sorting in the worst case is $O\left(n^2\right)$How to solve this "complexity deterioration" problem?

How to optimize quicksort?

Let's first look at why the time complexity of quick sorting in the worst case is $O\left(n^2\right)$What about? As mentioned earlier, if the data is originally ordered or close to ordered, and the last data is selected for each partition point, the quick sort algorithm will become very bad, and the time complexity will degenerate to$O\left(n^2\right)$. Actually,this$O\left(n^2\right)$The main reason for the emergence of time complexity is that our selection of partitions is not reasonable enough

So what kind of partition point is a good partition point? Or how to choose the partition point?

The ideal partition point is: the amount of data in the two partitions separated by the partition point is about the same

If you directly select the first or last data as the partition point directly, regardless of the characteristics of the data, it will definitely appear as mentioned before. In some cases, the worst case time complexity of sorting is O ( n 2 ) O(n^2 $O\left(n^2\right)$. In order to improve the performance of the sorting algorithm, we also want to make each partition as average as possible

Here are two more commonly used and simpler partition algorithms

1. Take the middle of three numbers

We take a number from the beginning, end, and middle of the interval, then compare the sizes, and take the middle value of these 3 numbers as the partition point. In this way, at intervals of a certain fixed length, the partition algorithm that takes data out for comparison and uses the intermediate value as the partition point is definitely better than simply taking a certain data. However, if the array to be sorted is relatively large, then "choose the middle of three numbers" may not be enough, you may need to "take the middle of five numbers" or "take the middle of ten numbers"

2. Random method

The random method is to randomly select an element from the interval to be sorted each time as the partition point. This method does not guarantee that each partition point is selected better, but from the perspective of probability, it is unlikely that each partition point is selected poorly, so on average, the partition points selected in this way are relatively good. Time complexity degenerates to worst $O\left(n^2\right)$the situation is unlikely to occur

Quick sort is implemented using recursion. Be wary of stack overflow with recursion. In order to avoid stack overflow caused by too deep recursion and too small stack in quick sort, we have two solutions: the first is to limit the depth of recursion. Once the recursion is too deep and exceeds the threshold we set in advance, the recursion will stop. The second is to implement a function call stack by simulating on the heap, and manually simulate the process of recursively pushing and popping the stack, so that there is no limit on the size of the system stack

Example analysis sorting function

Take the function in Glibc qsort()as an example. Although qsort()from the name, it seems to be based on the quick sort algorithm, in fact it does not only use the quick sort algorithm

If you look at the source code, you will find that qsort()merge sorting will be used first to sort the input data , because the space complexity of merge sorting is O(n), so for sorting small data volumes, such as 1KB, 2KB, etc., merge sorting requires an additional 1KB or 2KB of memory space, which is not a big problem. Nowadays, the memory of computers is quite large, and what we often pursue is speed

But if the amount of data is too large, as we mentioned earlier, sorting 100MB of data is not appropriate for us to use merge sort at this time. Therefore, when the amount of data to be sorted is relatively large, qsort()it will be sorted by the quick sort algorithm

Then qsort()how to choose the partition point of the quick sort algorithm? If you look at the source code, you will find that qsort()the method of selecting partition points is the "three-number method"

There is also the problem that too deep recursion will cause stack overflow that we mentioned earlier, qsort()which is solved by implementing a stack on the heap and manually simulating recursion

In fact, qsort()not only merge sort and quick sort are used, it also uses insertion sort. In the process of quick sorting, when the number of elements in the interval to be sorted is less than or equal to 4, it qsort()degenerates into insertion sorting, and does not continue to use recursion for quick sorting, because we have also said before that in the face of small-scale data, $O\left(n^2\right)$Algorithms with time complexity are not necessarily longer than O(nlogn) algorithms. Let us now analyze this statement

When we talked about complexity analysis, we said that the performance of the algorithm can be analyzed through time complexity. However, this kind of complexity analysis is more theoretical. If we go deeper, the time complexity is not equal to the actual running time of the code.

Time complexity represents a growth trend. If you draw a growth curve, you will find $O\left(n^2\right)$It is steeper than O(nlogn), which means that the growth trend is stronger. However, as we said before, in the big O complexity notation, we will omit the low order, coefficients and constants, that is to say, O(nlogn) may be O(knlogn + c) before the low order, coefficients and constants are omitted, and k and c may still be a relatively large number

Suppose k=1000, c=200, when we sort small-scale data (such as n=100), $n^{The\; value\; of\; 2}$ is actually smaller than knlogn+c

knlogn+c = 1000 * 100 * log100 + 200 远大于 10000
 
n^2 = 100*100 = 10000

So, for small scale data sorting, $O\left(n^2\right)$The sorting algorithm does not necessarily take longer to execute than the O(nlogn) sorting algorithm. For the sorting of small data volumes, we choose a relatively simple insertion sorting algorithm that does not require recursion

The sentinel mentioned before simplifies the code and improves execution efficiency. qsort()This programming technique is also used in the implementation of the insertion sort algorithm . Although the sentinel may just make one less judgment, after all, the sorting function is a very common and basic function, and performance optimization must be extreme

10. Binary Search

Search algorithm for ordered data sets: binary search (Binary Search) algorithm, also known as binary search algorithm

10.1 Dichotomous thinking is ubiquitous

Binary search is a very simple and easy-to-understand fast search algorithm, which can be seen everywhere in life. For example, in the classic word guessing game, randomly write a number between 0 and 99, and then guess what is written. During the guessing process, every time you guess, you will be prompted whether the guess is bigger or smaller, until you get it right

Assuming that the number written is 23, the steps are as follows (if there are even numbers in the guess range and two middle numbers, choose the smaller one)

You can guess it 7 times. This example uses the dichotomous thinking. According to this thinking, even if you guess the number from 0 to 999, you can guess it right only 10 times at most.

This is a real life example, we now return to the actual development scenario. Assume that there are 1,000 pieces of order data, which have been sorted according to the order amount from small to large. The amount of each order is different, and the smallest unit is yuan. We now want to know if there is an order with an amount equal to $19. Returns the order data if it exists, or null if it does not

The easiest way is of course to start from the first order and traverse the 1000 orders one by one until an order with an amount equal to 19 yuan is found. But it will be slower to search in this way. In the worst case, you may have to traverse the 1000 records before you can find them. Can binary search be used to solve it more quickly?

For the convenience of explanation, we assume that there are only 10 orders, and the order amounts are: 8, 11, 19, 23, 27, 33, 45, 55, 67, 98

Still use the dichotomy idea, compare the size with the intermediate data of the interval each time, and narrow the scope of the search interval. As shown in the figure below, low and high represent the subscript of the range to be searched, and mid represents the subscript of the middle element of the range to be found

Binary search is aimed at an ordered data set, and the search idea is somewhat similar to the divide and conquer idea. Each time, by comparing with the middle element of the interval, the interval to be searched is reduced to half of the previous one, until the element to be searched is found, or the interval is reduced to 0

10.2 O(logn) amazing lookup speed

Binary search is a very efficient search algorithm, how efficient is it? Let's analyze its time complexity

We assume that the data size is n, and the data will be reduced to half of the original size after each search, that is, it will be divided by 2. In the worst case, do not stop until the search range is reduced to empty

It can be seen that this is a geometric sequence. where $n/2^{When\; k}$ = 1, the value of k is the total number of reductions. And each reduction operation only involves the size comparison of two data, so after k times of interval reduction operations, the time complexity is O(k). by$n/2^k$ = 1, we can obtain k=$lo{g}_{2}n$ , so the time complexity is O(logn)

O(logn) This logarithmic time complexity . This is an extremely efficient time complexity, sometimes even more efficient than an algorithm whose time complexity is constant O(1). Why do you say that?

Because logn is a very "terrible" order of magnitude, even if n is very, very large, the corresponding logn is also very small. For example, n is equal to 2 to the 32nd power, and this number is about 4.2 billion. In other words, if we use binary search for a data in 4.2 billion data, we need to compare up to 32 times

As we said earlier, when using big O notation to express time complexity, constants, coefficients, and low-orders are omitted. For algorithms with constant time complexity, O(1) may represent a very large constant value, such as O(1000), O(10000). Therefore, algorithms with constant time complexity may sometimes not be as efficient as O(logn) algorithms

Conversely, logarithms correspond to exponents. There is a very famous "The Story of Archimedes and the King Playing Chess", which is why we say that algorithms with exponential time complexity are ineffective in the face of large-scale data

10.3 Recursive and non-recursive implementation of binary search

In fact, simple binary search is not difficult to write. The simplest case is that there are no repeated elements in the ordered array , and we use binary search for data whose value is equal to a given value. The code is as follows:

public int bsearch(int[] a, int n, int value) {
    
    
  int low = 0;
  int high = n - 1;
 
  while (low <= high) {
    
    
    int mid = (low + high) / 2;
    if (a[mid] == value) {
    
    
      return mid;
    } else if (a[mid] < value) {
    
    
      low = mid + 1;
    } else {
    
    
      high = mid - 1;
    }
  }
 
  return -1;
}

low, high, and mid all refer to the subscript of the group, where low and high indicate the range of the current search, and the initial low=0, high=n-1. mid means the middle position of [low, high]. By comparing the size of a[mid] and value, we update the interval to be searched next until we find it or the interval is reduced to 0, then exit

Here are 3 places where it is easy to go wrong

1. Loop exit condition

Note that low<=high, not low<high

2. The value of mid

In fact, mid=(low+high)/2 is problematic. Because if low and high are relatively large, the sum of the two may overflow. The improved method is to write the calculation method of mid as low+(high-low)/2. Furthermore, if we want to optimize the performance to the extreme, we can convert the division by 2 operation into a bit operation low+((high-low)>>1). Because computers can handle bit operations much faster than division

3. Update of low and high

low=mid+1, high=mid-1. Pay attention to the +1 and -1 here. If you directly write low=mid or high=mid, an infinite loop may occur. For example, when high=3, low=3, if a[3] is not equal to value, it will cause the loop to never exit

In fact, in addition to using loops to implement binary search, it can also be implemented using recursion , and the process is very simple

// 二分查找的递归实现
public int bsearch(int[] a, int n, int val) {
    
    
  return bsearchInternally(a, 0, n - 1, val);
}
 
private int bsearchInternally(int[] a, int low, int high, int value) {
    
    
  if (low > high) return -1;
 
  int mid =  low + ((high - low) >> 1);
  if (a[mid] == value) {
    
    
    return mid;
  } else if (a[mid] < value) {
    
    
    return bsearchInternally(a, mid+1, high, value);
  } else {
    
    
    return bsearchInternally(a, low, mid-1, value);
  }
}

10.4 Limitations of Binary Search Application Scenarios

1. First of all, binary search relies on the sequence table structure, which is simply an array

Can binary search rely on other data structures? For example linked list. The answer is no, the main reason is that the binary search algorithm needs to randomly access elements according to the subscript. The time complexity of randomly accessing data according to the subscript of the array is O(1), while the time complexity of random access of the linked list is O(n). Therefore, if the data is stored in a linked list, the time complexity of the binary search will become very high

Binary search can only be used on data structures where data is stored in sequential tables. Binary search cannot be applied if your data is stored via other data structures

2. Binary search is for ordered data

Binary search has strict requirements on this point, and the data must be in order. If the data is not ordered, we need to sort it first. As we mentioned in the previous chapter, the lowest time complexity of sorting is O(nlogn). Therefore, if we are targeting a set of static data without frequent insertion and deletion, we can perform one sort and multiple binary searches. In this way, the cost of sorting can be amortized, and the marginal cost of binary search will be relatively low

However, if our data set has frequent insertion and deletion operations, if we want to use binary search, we must either ensure that the data is still in order after each insertion and deletion operation, or sort it before each binary search. For this kind of dynamic data collection, no matter which method, the cost of maintaining order is very high

Therefore, binary search can only be used in scenarios where insertion and deletion operations are infrequent and one sorting is performed multiple times. For dynamically changing data sets, binary search is no longer applicable

3. The amount of data is too small to be suitable for binary search

If the amount of data to be processed is small, there is no need to use binary search at all, and sequential traversal is sufficient. For example, if we search for an element in an array with a size of 10, the search speed is similar no matter whether we use binary search or sequential traversal. Only when the amount of data is relatively large, the advantages of binary search will be more obvious

However, there is one exception here. If the comparison operation between data is very time-consuming, regardless of the amount of data, I recommend using binary search. For example, all strings with a length of more than 300 are stored in the array, and it will be very time-consuming to compare the size of two such long strings. We need to reduce the number of comparisons as much as possible, and the reduction of the number of comparisons will greatly improve performance. At this time, binary search is more advantageous than sequential traversal

4. Too much data is not suitable for binary search

The bottom layer of binary search needs to rely on the data structure of array, and in order to support the characteristics of random access, array requires continuous memory space and relatively strict memory requirements. For example, if we have 1GB of data, if we want to store it in an array, we need 1GB of continuous memory space

Pay attention to the word "continuous" here, that is to say, even if there is 2GB of memory space remaining, if the remaining 2GB of memory space is scattered and there is no continuous 1GB of memory space, then it is still impossible to apply for a 1GB array. And our binary search works on the data structure of arrays, so it is more difficult to store too large data in arrays, so binary search cannot be used.

How to quickly find an integer among 10 million integers?

Our memory limit is 100MB, and the size of each data is 8 bytes. The easiest way is to store the data in an array, and the memory usage is almost 80MB, which meets the memory limit. We can first sort the 10 million data from small to large, and then use the binary search algorithm to quickly find the desired data

10.5 Dichotomous variants

Donald E. Knuth said in The Art of Computer Programming, Volume 3, "Sorting and Searching", "Although the first binary search algorithm appeared in 1946, the first fully correct implementation of the binary search algorithm did not appear until 1962"

"Ten out of ten and nine out of nine mistakes". Although the principle of binary search is extremely simple, it is not easy to write a binary search without bugs. The simplest binary search is really not difficult to write, but the deformation problem of binary search is not so easy to write

10.5.1 Finding the first element whose value is equal to a given value

The binary search written earlier is the simplest one, that is, there is no duplicate data in the ordered data set, and we look for data whose value is equal to a given value. If we slightly modify this problem, there are duplicate data in the ordered data set, and we want to find the first data whose value is equal to the given value, so the previous binary search code can continue to work?

For example, the following is an ordered array, where the values ​​of a[5], a[6], and a[7] are all equal to 8, which is repeated data. We want to find the first data equal to 8, that is, the element whose subscript is 5

If we use the previous binary search code to implement, first compare 8 with the middle value a[4] of the interval, 8 is greater than 6, and then continue to search between subscripts 5 to 9. The middle position between the subscripts 5 and 9 is the subscript 7, a[7] is exactly equal to 8, so the code returns

Although a[7] is also equal to 8, it is not the first element equal to 8 that we want to find, because the first element whose value is equal to 8 is the element whose index is 5 in the array. Therefore, for this deformation problem, we can slightly modify the code

If 100 people write binary search, there will be 100 ways to write it. There are many implementation methods of deformed binary search on the Internet, and many of them are written very concisely, such as the following writing method. However, although it is concise, it is very brain-burning to understand, and it is easy to write wrong

public int bsearch(int[] a, int n, int value) {
    
    
  int low = 0;
  int high = n - 1;
  while (low <= high) {
    
    
    int mid = low + ((high - low) >> 1);
    if (a[mid] >= value) {
    
    
      high = mid - 1;
    } else {
    
    
      low = mid + 1;
    }
  }
 
  if (low < n && a[low]==value) return low;
  else return -1;
}

A more understandable way of writing is as follows:

public int bsearch(int[] a, int n, int value) {
    
    
  int low = 0;
  int high = n - 1;
  while (low <= high) {
    
    
    int mid =  low + ((high - low) >> 1);
    if (a[mid] > value) {
    
    
      high = mid - 1;
    } else if (a[mid] < value) {
    
    
      low = mid + 1;
    } else {
    
    
      if ((mid == 0) || (a[mid - 1] != value)) return mid;
      else high = mid - 1;
    }
  }
  return -1;
}

The relationship between a[mid] and the value to be found has three situations: greater than, less than, or equal to. For the case of a[mid]>value, we need to update high= mid-1; for the case of a[mid]<value, we need to update low=mid+1. Both of these points are easy to understand. So what should be done when a[mid]=value?

If we are looking for any element whose value is equal to a given value, when a[mid] is equal to the value we are looking for, a[mid] is the element we are looking for. However, if we are solving for the first element whose value is equal to a given value, when a[mid] is equal to the value to be found, we need to confirm whether this a[mid] is the first element whose value is equal to a given value

Focus on line 11 of the code. If mid is equal to 0, then this element is already the first element of the array, then it must be what we are looking for; if mid is not equal to 0, but the previous element a[mid-1] of a[mid] is not equal to value, it also means that a[mid] is the first element we are looking for whose value is equal to the given value

If after checking, it is found that an element a[mid-1] in front of a[mid] is also equal to value, it means that a[mid] at this time is definitely not the first element whose value is equal to the given value we are looking for. Then we update high=mid-1, because the element we are looking for must appear between [low, mid-1]

10.5.2 Finding the last element whose value is equal to a given value

The previous problem is to find the first element whose value is equal to the given value. Now I change the question a little bit to find the last element whose value is equal to the given value. How to do it?

Just change the condition of the above code to

public int bsearch(int[] a, int n, int value) {
    
    
  int low = 0;
  int high = n - 1;
  while (low <= high) {
    
    
    int mid =  low + ((high - low) >> 1);
    if (a[mid] > value) {
    
    
      high = mid - 1;
    } else if (a[mid] < value) {
    
    
      low = mid + 1;
    } else {
    
    
      if ((mid == n - 1) || (a[mid + 1] != value)) return mid;
      else low = mid + 1;
    }
  }
  return -1;
}

The focus is on line 11 of the code. If the element a[mid] is already the last element in the array, then it must be what we are looking for; if the next element a[mid+1] of a[mid] is not equal to value, it also means that a[mid] is the last element we are looking for whose value is equal to the given value

If we check and find that an element a[mid+1] after a[mid] is also equal to value, it means that the current a[mid] is not the last element whose value is equal to the given value. We will update low=mid+1, because the element we are looking for must appear between [mid+1, high]

10.5.3 Find the first element greater than or equal to a given value

In a sorted array, finds the first element greater than or equal to a given value. For example, such a sequence stored in the array: 3, 4, 6, 7, 10. If you look for the first element greater than or equal to 5, it will be 6

In fact, the implementation idea is similar to the implementation ideas of the previous two deformation problems, and the code is even simpler to write.

public int bsearch(int[] a, int n, int value) {
    
    
  int low = 0;
  int high = n - 1;
  while (low <= high) {
    
    
    int mid =  low + ((high - low) >> 1);
    if (a[mid] >= value) {
    
    
      if ((mid == 0) || (a[mid - 1] < value)) return mid;
      else high = mid - 1;
    } else {
    
    
      low = mid + 1;
    }
  }
  return -1;
}

If a[mid] is less than the value to be searched for, the value to be searched must be between [mid+1, high], so we update low=mid+1

For the case where a[mid] is greater than or equal to the given value value, we need to check whether this a[mid] is the first element whose value is greater than or equal to the given value we are looking for. If there is no element before a[mid], or the previous element is smaller than the value to be found, then a[mid] is the element we are looking for. The code corresponding to this logic is line 7

If a[mid-1] is also greater than or equal to the value value to be searched, it means that the element to be searched is between [low, mid-1], so we update high to mid-1

10.5.4 Find the last element less than or equal to a given value

Finds the last element less than or equal to a given value. For example, such a set of data is stored in the array: 3, 5, 6, 8, 9, 10. The last element less than or equal to 7 is 6. similar to the above

public int bsearch7(int[] a, int n, int value) {
    
    
  int low = 0;
  int high = n - 1;
  while (low <= high) {
    
    
    int mid =  low + ((high - low) >> 1);
    if (a[mid] > value) {
    
    
      high = mid - 1;
    } else {
    
    
      if ((mid == n - 1) || (a[mid + 1] > value)) return mid;
      else low = mid + 1;
    }
  }
  return -1;
}

How to quickly locate the attribution of an IP address?

If the corresponding relationship between the IP range and the attribution is not updated frequently, we can preprocess the 120,000 pieces of data and sort them according to the starting IP from small to large. How to sort it? We know that an IP address can be converted into a 32-bit integer. Therefore, we can sort the starting addresses from small to large according to the size relationship of the corresponding integer values

Then, this problem can be transformed into the fourth variant problem "in an ordered array, find the last element less than or equal to a given value"

When we want to query the location of an IP, we can first find the last IP range whose starting IP is less than or equal to this IP through binary search, and then check whether the IP is in this IP range. If it is, we will take out the corresponding location and display it;