Why time complexity analysis is needed?
Through statistics and monitoring, the execution time of the algorithm and the memory size occupied can be obtained. However, this statistical method
has many shortcomings, such as:
- The test results depend on the test environment. For example, if the chip of the test PC is changed from i7 to i5, the running time will increase.
- Test results depend on the size of the test data, such as small-scale data sorting, insertion sorting is faster than quick sorting
Representation of Time Complexity
Big O notation (important)
Definition: If and only if there are two parameters c > 0, n0 > 0, for all n >= n0, f(n) <= cg(n), then f(n) = O(g(
n )), as shown in the figure:
Example of time complexity in Big O notation:
int cal(int n) {
int sum = 0;
int i = 1;
int j = 1;
for (; i <= n; ++i) {
j = 1;
for (; j <= n; ++j) {
sum = sum + i * j;
}
}
return sum;
}
Assuming the above code, the time to execute a line of code is t, and the total time spent is (2*n^2+2*n+4)*t. When n is very large, the time spent by the above code
depends only on n^2, that is, T(n) = O(n 2), the time complexity of the above code is O(n 2)
To put it simply: the big O notation is to ignore the constants, low-orders, and coefficients in the formula, and only need to record the magnitude of the largest order
Big Ω notation (just understand)
Definition: If there are positive numbers c and n0, so that for all n >= n0,
f(n) >= cg(n), then f(n) = Ω(g(n)), as shown in the figure:
Big θ notation (just understand)
Definition: A function is called θ(g(n)) if it is in both the set O(g(n)) and the set Ω(g(n
)). That is, when the upper and lower limits are the same, the big θ representation can be used, as shown in the figure:
Commonly used time complexity scale
- Constant order O(1)
O(1) is just a representation of constant-level time complexity, not that only one line of code is executed. For example, this code, even if there are 3 lines, its time complexity is O(1),
not O(3).
int i = 8;
int j = 6;
int sum = i + j;
As long as the execution time of the code does not increase with the increase of n, the time complexity of the code is recorded as O(1). Or, in general, as long as there are no loop statements or recursive statements in the algorithm
, even if there are tens of thousands of lines of code, its time complexity is Ο(1).
- Logarithmic order O(logn)
- Linear order O(n)
- Linear-logarithmic order O(n * logn)
- Square order O(n 2), legislative order O(n 3), k square order O(n^k), etc.
- Exponential order O(2^n)
- Factorial order O(n!)
Time Complexity Analysis
Calculate the number of runs to determine
The time complexity of some algorithms can be determined simply by counting the number of times they are run.
Example 1, an algorithm to calculate the sum of two arrays:
int count(int[] array1,int m,int[] array2,int n){
int sum = 0;
for(int i = 0;i < m;i++){
//运行 m 次
sum += array1[i];
}
for(int i = 0;i < n;i++){
//运行 n 次
sum += array2[i];
}
return sum;
}
Since m and n are not equal, the number of executions is m+n, so the time complexity is O(m+n)
Example 2, bubble sort algorithm:
private static void sort(int[] data){
if(data.length <= 1)return;
int n = data.length;
for (int i = 0; i < n; i++) {
for (int j = 0; j+1 < n - i; j++) {
if(data[j] > data[j+1]){
int cache = data[j];
data[j] = data[j+1];
data[j+1] = cache;
}
}
}
}
The number of times the above code runs is n*(n+1)/2. Due to the big O notation, constants, low-orders, and coefficients need to be removed, so the time complexity is O(n^2)
Example 3, some math is needed to get the time complexity:
i=1;
while (i <= n) {
i = i * 2;
}
As shown in the code, the execution time x of this program satisfies n = 2^0 * 2^1 * 2^2 * ... *2^x, that is, x = logn, so the time complexity is O(logn)
execution tree
For recursive functions, the number of recursive calls rarely scales linearly with the size of the input. In this case, it is better to use an execution tree, which is a tree used to represent the execution flow of a recursive function. Each node in the tree represents a call to a recursive function. Thus, the total number of nodes in the tree corresponds to the number of recursive calls during execution.
The following is the Fibonacci number programming problem on leetcode .
斐波那契数，通常用 F(n) 表示，形成的序列称为斐波那契数列。
该数列由 0 和 1 开始，后面的每一项数字都是前面两项数字的和。也就是：
F(0) = 0, F(1) = 1
F(N) = F(N - 1) + F(N - 2), 其中 N > 1.
给定 N，计算 F(N)。
The answer code is as follows (note that the following code can be optimized by memoization)
class Solution {
public int fib(int N) {
if(N == 0) return 1;
return fib(N-1)+fib(N-2);
}
}
The diagram below (image source leetcode) shows the execution tree used to calculate the Fibonacci number f(4).
In a complete binary tree with n levels, the total number of nodes is 2^n -1. So the upper bound (though not strict) on the number of recursions in f(n) is also 2^n -1. Then we can estimate the time complexity of the algorithm as O(2^n)
other
Best, worst, average case time complexity
Example, to find the position of x in the array:
// n表示数组array的长度
int find(int[] array, int n, int x) {
int i = 0;
int pos = -1;
for (; i < n; ++i) {
if (array[i] == x) {
pos = i;
break;
}
}
return pos;
}
When x is in the i=0 position of the array, the program only needs to be executed once, and the time complexity is O(1);
When x is not in the array, the program needs to traverse the array, and the time complexity is O(n);
In order to express the different time complexity of the code in different situations, we need to introduce three concepts: best case time complexity , worst case time complexity and average case time complexity . But the probability of the best and worst time complexity is small. In order to better represent the complexity of the average case, we need the average case time complexity.
Taking the above code as an example, the probability that x exists or not is 1/2, then the probability that x exists and appears in the 0 ~ n-1 position is (1+2+...+n)/2n, and the probability that x does not exist is n/2, namely (3*n+1)/4, using big O notation, the time complexity is O(n)
Amortized time complexity
Most of the complexity cases of code execution are low-level complexity. When individual cases are high-level complexity and have a timing relationship, the individual high-level complexity can be evenly distributed to the low-level complexity
. Basically amortized result equals low level complexity
为了更全面，更准确的描述代码的时间复杂度，才要引入最好、最坏、平均情况时间复杂度以及均摊时间复杂度。但是只有代码复杂度在不同情况下出现量级差别时才需要区别这四种复杂度。大多数情况下，是不需要区别分析它们的。
Time Complexity of Common Algorithms
During the interview, you are often asked to write the code by hand. By the way, what is the time complexity of the algorithm, so it is very necessary to remember the time complexity of common algorithms.
Sorting Algorithm
| Sorting Algorithm | time complexity |
|---|---|
| Bubble Sort | O(n^2) |
| insertion sort | O(n^2) |
| selection sort | O(n^2) |
| quick sort | O(nlogn) |
| merge sort | O(nlogn) |
| bucket sort | O(n) |
| counting sort | O(n) |
| radix sort | O(n) |
Binary tree traversal algorithm
| Binary tree traversal algorithm | time complexity |
|---|---|
| Preorder traversal (recursive implementation) | O(n) |
| Preorder traversal (iteration implementation) | O(n) |
| Inorder traversal (recursive implementation) | O(n) |
| Inorder traversal (iterative implementation) | O(n) |
| Post-order traversal (recursive implementation) | O(n) |
| Post-order traversal (iteration implementation) | O(n) |
reference
- Time complexity analysis of leetcode recursion
- Chinese University MOOC - Data Structure and Algorithm Course
- Geek Time's "The Beauty of Data Structure and Algorithm" column