Usually, for a given algorithm, we do two analyses. The first is to prove the correctness of the algorithm mathematically. This step mainly uses formal proof methods and related reasoning modes, such as loop invariants and mathematical induction. On the basis of proving that the algorithm is correct, the second part is to analyze the time complexity of the algorithm . The time complexity of the algorithm reflects the magnitude of the increase in the execution time of the program with the increase of the input scale, and to a large extent can well reflect the pros and cons of the algorithm. Therefore, as a programmer, it is necessary to master the basic algorithm time complexity analysis method.
The execution time of the algorithm needs to be measured by the time consumed when the program compiled according to the algorithm runs on the computer. There are usually two ways to measure the execution time of a program.
1. The method of post-event statistics
This method is feasible, but it is not a good method. This method has two shortcomings: one is that if you want to evaluate the performance of the designed algorithm, you must first compile the corresponding program based on the algorithm and actually run it; the other is that the time statistics obtained depend on environmental factors such as computer hardware and software. , Sometimes it is easy to conceal the advantages of the algorithm itself.
2. Methods of pre-analysis and estimation
Because post-event statistical methods rely more on computer hardware, software and other environmental factors, it is sometimes easy to conceal the advantages and disadvantages of the algorithm itself. Therefore, people often use the method of pre-analysis and estimation.
Before writing the program, estimate the algorithm based on statistical methods. The time consumed by a program written in a high-level language to run on a computer depends on the following factors:
(1). 算法采用的策略、方法；(2). 编译产生的代码质量；(3). 问题的输入规模；(4). 机器执行指令的速度。
An algorithm is composed of the control structure (sequence, branch and loop) and the original operation (referring to the operation of the inherent data type), and the algorithm time depends on the combined effect of the two. In order to facilitate the comparison of different algorithms for the same problem, the usual approach is to select an original operation from the algorithm that is a basic operation for the problem (or algorithm type) being studied, and take the number of repeated executions of the basic operation as The time measurement of the algorithm.
1. Time complexity
(1) Time frequency The time taken for an algorithm to execute cannot be calculated theoretically, and it can only be known by running a test on the computer. But it is impossible and unnecessary for us to test each algorithm on the computer. We only need to know which algorithm takes more time and which algorithm takes less time. And the time spent by an algorithm is directly proportional to the number of executions of the statement in the algorithm. The more the number of executions of the statement in the algorithm, the more time it takes. The number of executions of statements in an algorithm is called statement frequency or time frequency. Denoted as T(n).
(2) Time complexity In the time frequency just mentioned, n is called the scale of the problem. When n changes constantly, the time frequency T(n) will also change constantly. But sometimes we want to know what pattern it presents when it changes. To this end, we introduce the concept of time complexity. In general, the number of repeated executions of the basic operation in the algorithm is a certain function of the problem scale n, represented by T(n), if there is a certain auxiliary function f(n), so that when n approaches infinity, T( The limit value of n)/f(n) is a constant that is not equal to zero, and f(n) is said to be a function of the same order of magnitude of T(n). Denoted as T(n)=O(f(n)), call O(f(n)) the progressive time complexity of the algorithm, referred to as time complexity.
In addition, the Landau symbol used in the above formula was actually first introduced by the German number theorist Paul Bachmann in his 1892 book "Analytic Number Theory", by another German number theorist Edmund Promotion by Edmund Landau. The role of Landau notation is to use simple functions to describe the behavior of complex functions, giving an upper or lower (exact) bound. Generally, only the big O symbol is used when calculating the complexity of the algorithm, and the small o symbol, Θ symbol, etc. in the Landau symbology are less commonly used. The O here was originally the uppercase Greek letter, but now all use the uppercase English letter O; the small o symbol also uses the lowercase English letter o, and the Θ symbol maintains the uppercase Greek letter Θ.
T (n) = Ο(f (n)) means that there is a constant C such that when n approaches positive infinity, there is always T (n) ≤ C * f(n). To put it simply, T(n) is as large as f(n) when n approaches positive infinity. That is to say, when n approaches positive infinity, the upper bound of T (n) is C * f(n). Although there is no regulation on f(n), it is generally the simplest function possible. For example, O(2n2+n +1) = O (3n2+n+3) = O (7n2 + n) = O (n2), generally only use O(n2) to represent it. Note that there is a constant C hidden in the big O symbol, so generally no coefficient is added to f(n). If T(n) is regarded as a tree, then what O(f(n)) expresses is the trunk, and only cares about the main trunk, and discards all other minutiae.
In various algorithms, if the number of sentence executions in the algorithm is a constant, the time complexity is O(1). In addition, when the time frequency is not the same, the time complexity may be the same, such as T(n)= n2+3n+4 and T(n)=4n2+2n+1 have different frequencies, but the time complexity is the same, both are O(n2). Arranged in increasing order of magnitude, common time complexity are: constant order O(1), logarithmic order O(log2n), linear order O(n), linear logarithmic order O(nlog2n), square order O(n2), Cubic order O(n3),..., k-th order O(nk), exponential order O(2n). As the problem scale n continues to increase, the above-mentioned time complexity continues to increase, and the execution efficiency of the algorithm becomes lower.
It can be seen from the figure that we should choose polynomial order O(nk) algorithms as much as possible, and do not want to use exponential order algorithms.
常见的算法时间复杂度由小到大依次为：Ο(1)＜Ο(log2n)＜Ο(n)＜Ο(nlog2n)＜Ο(n2)＜Ο(n3)＜…＜Ο(2n)＜Ο(n!)
一般情况下，对一个问题（或一类算法）只需选择一种基本操作来讨论算法的时间复杂度即可，有时也需要同时考虑几种基本操作，甚至可以对不同的操作赋予不同的权值，以反映执行不同操作所需的相对时间，这种做法便于综合比较解决同一问题的两种完全不同的算法。
(3) The specific steps to solve the time complexity of the algorithm are :
⑴ Find out the basic sentences in the algorithm;
The most frequently executed statement in the algorithm is the basic statement, which is usually the loop body of the innermost loop.
⑵ Calculate the order of magnitude of the execution times of the basic statement;
Only the order of magnitude of the execution times of the basic statement is calculated, which means that as long as the highest power in the function of the execution times of the basic statement is correct, all the coefficients of the low power and the highest power can be ignored. This simplifies the analysis of the algorithm and focuses attention on the most important point: the growth rate.
⑶ The time performance of the algorithm is represented by a big Ο symbol.
Put the order of magnitude of the number of executions of the basic sentence into the big Ο symbol.
If the algorithm contains nested loops, the basic statement is usually the innermost loop body. If the algorithm contains parallel loops, the time complexity of the parallel loops is added. E.g:
for (i=1; i<=n; i++)
x++;
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
x++;
The time complexity of the first for loop is Ο(n), and the time complexity of the second for loop is Ο(n2), so the time complexity of the entire algorithm is Ο(n+n2)=Ο(n2).
Ο(1) means that the number of executions of the basic statement is a constant. Generally speaking, as long as there is no loop statement in the algorithm, the time complexity is Ο(1). Among them, Ο(log2n), Ο(n), Ο(nlog2n), Ο(n2) and Ο(n3) are called polynomial times, and Ο(2n) and Ο(n!) are called exponential times. Computer scientists generally believe that the former (that is, the algorithm of polynomial time complexity) is an effective algorithm, and these problems are called P (Polynomial, polynomial) problems, and the latter (that is, the algorithm of exponential time complexity) is called NP ( Non-Deterministic Polynomial, non-deterministic polynomial) problem.
Generally speaking, the complexity of the polynomial level is acceptable, and many problems have polynomial-level solutions-that is to say, for such a problem, for an input of size n, the result is obtained in n^k time, said For P question. Some problems are more complicated and do not have a polynomial time solution, but you can verify whether a guess is correct in polynomial time. For example, is 4294967297 a prime number? If you want to start directly, then take out all the prime numbers that are less than the square root of 4294967297 and see if they can be divisible. Fortunately, Euler told us that this number is equal to the product of 641 and 6700417. It is not a prime number and is very easy to verify. By the way, please tell Fermat that his conjecture does not hold. Problems such as decomposition of large numbers and Hamilton cycles can verify whether a "solution" is correct in polynomial time. Such problems are called NP problems.
(4) There are several simple program analysis rules when calculating the time complexity of the algorithm:
(1). For some simple input and output statements or assignment statements, it is approximately considered that it takes O(1) time
(2). For sequential structures, the time required to execute a series of statements in sequence can use the "summation rule" under the big O
The law of summation: means that if the time complexity of the two parts of the algorithm is T1(n)=O(f(n)) and T2(n)=O(g(n)), then T1(n)+T2 (n)=O(max(f(n), g(n)))
In particular, if T1(m)=O(f(m)), T2(n)=O(g(n)), then T1(m)+T2(n)=O(f(m) + g( n))
(3). For the selection structure, such as the if statement, its main time consumption is the time used to execute the then sentence or the else sentence. It should be noted that the test conditions also require O(1) time
(4). For the loop structure, the running time of the loop statement is mainly reflected in the time consumption of executing the loop body and checking the loop conditions in multiple iterations. Generally, the "rule of multiplication" under the big O can be used.
Multiplication rule: If the time complexity of the two parts of the algorithm is T1(n)=O(f(n)) and T2(n)=O(g(n)), then T1*T2=O(f (n)*g(n))
(5). For a complex algorithm, it can be divided into several easy-to-estimate parts, and then the time complexity of the entire algorithm using the summation rule and multiplication rule technology
In addition, there are the following two algorithms: (1) If g(n)=O(f(n)), then O(f(n))+ O(g(n))= O(f(n)) ; (2) O(Cf(n)) = O(f(n)), where C is a normal number
(5) The following are examples of several common time complexity:
(1) 、 O (1)
Temp=i;
i=j;
j=temp;
The frequency of the above three single statements is 1, and the execution time of this program segment is a constant that has nothing to do with the problem scale n. The time complexity of the algorithm is a constant order, denoted as T(n)=O(1). Note: If the execution time of the algorithm does not increase with the increase of the problem size n, even if there are thousands of statements in the algorithm, the execution time is just a relatively large constant. The time complexity of this type of algorithm is O(1).
(2), O(n2)
2.1. Exchange the contents of i and j
sum=0； （一次）
for(i=1;i<=n;i++) （n+1次）
for(j=1;j<=n;j++) （n2次）
sum++； （n2次）
Solution: Because Θ(2n2+n+1)=n2 (Θ is: remove the low-order terms, remove the constant terms, and remove the common parameters of the high-order terms to get), so T(n)==O(n2);
2.2.
for (i=1;i<n;i++)
{
y=y+1; ①
for (j=0;j<=(2*n);j++)
x++; ②
}
Solution: The frequency of sentence 1 is n-1
The frequency of sentence 2 is (n-1)*(2n+1)=2n2-n-1
f(n)=2n2-n-1+(n-1) =2n2-2;
又Θ(2n2-2)=n2
该程序的时间复杂度T(n)=O(n2).
Under normal circumstances, only need to consider the number of executions of the statement in the loop body for the step loop statement, ignore the step length plus 1, final value judgment, control transfer and other components in the statement. When there are several loop statements, the algorithm time is complicated. The degree is determined by the frequency f(n) of the innermost sentence in the loop sentence with the most nesting levels.
(3) 、 O (n)
a=0;
b=1; ①
for (i=1;i<=n;i++) ②
{
s=a+b; ③
b=a; ④
a=s; ⑤
}
Solution: The frequency of sentence 1: 2,
the frequency of sentence 2: n,
the frequency of sentence 3: n-1,
the frequency of sentence 4: n-1,
the frequency of sentence 5: n-1,
T( n)=2+n+3(n-1)=4n-1=O(n).
(4), O(log2n)
i=1; ①
while (i<=n)
i=i*2; ②
Solution: The frequency of sentence 1 is 1, and
the frequency of sentence 2 is f(n), then: 2^f(n)<=n; f(n)<=log2n
take the maximum value f(n)=log2n ,
T(n)=O(log2n)
(5) 、 O (n3)
for(i=0;i<n;i++)
{
for(j=0;j<i;j++)
{
for(k=0;k<j;k++)
x=x+2;
}
}
Solution: When i=m, j=k, the number of inner loops is k. When i=m, j can take 0,1,...,m-1, so the innermost loop here has a total of 0+1 +…+m-1=(m-1)m/2 times. Therefore, if i is taken from 0 to n, the cycle is carried out: 0+(1-1)*1/2+…+(n-1) n/2=n(n+1)(n-1)/6 so the time complexity is O(n3).
(5) The time complexity and space complexity of commonly used algorithms are
an empirical rule: where c is a constant. If the complexity of an algorithm is c, log2n, n, n*log2n, then the time efficiency of this algorithm is relatively high, if It is 2n, 3n, n!, then a slightly larger n will make the algorithm unable to move, and the ones in the middle are not satisfactory.
Algorithm time complexity analysis is a very important issue. Any programmer should be proficient in its concepts and basic methods, and must be good at exploring its essence from a mathematical level in order to accurately understand its meaning.
2. The space complexity of the algorithm
Similar to the discussion of time complexity, the space complexity (Space Complexity) S(n) of an algorithm is defined as the storage space consumed by the algorithm, which is also a function of the problem size n. Asymptotic space complexity is often referred to simply as space complexity.
Space Complexity (Space Complexity) is a measure of the amount of storage space that an algorithm temporarily occupies during its operation. The storage space occupied by an algorithm on the computer memory includes the storage space occupied by the storage algorithm itself, the storage space occupied by the input and output data of the algorithm, and the storage space temporarily occupied by the algorithm during operation. The storage space occupied by the input and output data of the algorithm is determined by the problem to be solved. It is passed from the calling function through the parameter table, and it does not change with the algorithm. The storage space occupied by the storage algorithm itself is proportional to the length of the algorithm. To compress the storage space in this area, a shorter algorithm must be written. The storage space temporarily occupied by the algorithm varies with the algorithm. Some algorithms only take up a small amount of temporary work units and do not change with the size of the problem. We call this algorithm "in-place". , Is a storage-saving algorithm, such as the several algorithms introduced in this section; the number of temporary work units that some algorithms need to occupy is related to the scale n of the problem, and it increases with the increase of n. When n is large, more storage units will be occupied. For example, the quick sort and merge sort algorithms that will be introduced in Chapter 9 fall into this situation.
For example, when the space complexity of an algorithm is a constant, that is, it does not change with the amount of processed data n, it can be expressed as O(1); when the space complexity of an algorithm is the logarithm of n based on 2 When it is proportional, it can be expressed as 0 (10g2n); when the complexity of an algorithm is linearly proportional to n, it can be expressed as 0(n). If the formal parameter is an array, you only need to assign one to it. The space for storing an address pointer transferred from the actual parameter, that is, a machine word length space; if the formal parameter is a reference, it only needs to allocate a space for storing an address, and use it to store the address of the corresponding actual parameter variable , So that the actual parameter variable is automatically referenced by the system.
Reference:
http://www.cnblogs.com/songQQ/archive/2009/10/20/1587122.html
http://www.cppblog.com/85940806/archive/2011/03/12/141672.html
https: //blog.csdn.net/zolalad/article/details/11848739