Data structure and algorithm (Chapter 2): Complexity analysis

1. Complexity analysis

First of all, we must make it clear that the essence of data structure and algorithm is to solve the problem of "fast" and "saving". To describe the quality of an algorithm, complexity analysis is needed. The complexity analysis can be divided into the following two types.

• time complexity

• Space complexity

Time complexity is the speed of describing the algorithm, and space complexity is the saving of describing the algorithm. Generally speaking, the complexity is time complexity. After all, the storage space of modern computers is no longer so tight. Time complexity is our key research content.

Second, the great Ocomplexity of the representation

First, look at a piece of code, seek the 1~naccumulated sum.

int demo(int n) {

    int i;
    int sum = 0;

    for(i=1; i<n; i++) {
        sum += i;
    }

    return sum;
}

Now let's estimate the execution time of this code (the following are examples of time complexity, and space complexity will be discussed at the end).

From the CPUpoint of view, every line of code performs a similar operation to read data - operation - write data . Here, for convenience of calculation, it is assumed execution time of each line of code is the same, a trepresents the time required for one line, nrepresents the magnitude of the data size, T(n)it indicates the total execution time code.

So what is the total execution time of this code? Let's count.

First, the body has a function 5statement, the 1、2、5statement is executed a total of 3times, the time required is 3*t; the first 3、4statement is executed each ntime, the time required is 2*n*t. Add the execution time of these two code segments, and the result is the total time required for this code.

$$T(n)=(2n+3)t$$

A rule can be obtained by the above equation, T(n)as the nlarger and larger, it becomes smaller and smaller. So, T(n)with na proportional, mathematical notation can be written.

$$T(n)=O(f(n))$$

Which f(n)is the time required for the execution and code segments, Oindicates T(n)the f(n)relationship between the proportional.

From the formula, the time required for the execution of the code segment can be expressed as $T\left(n\right)=O(2n+3)$ . This is thebigOtime complexity notation. BigOtime complexity does not actually perform the specific code indicating real time, but rathercode execution time with the change of trend data scale growth, therefore, also known asprogressive complexity of timereferred totime complexity.

Actually $O(2n+3)\; It\; is$ not a representation of the final time complexity. In the actual complexity analysis, theconstants,coefficients, andlow-ordersin the formula are generallyignored. Because these three parts do not affect the growth trend (remember that time complexity is actually progressive time complexity!), you only need to record a maximum magnitude. The final representation of time complexity is$O\left(n\right)$。

Third, the analysis method of complexity

1. Maximum order

int demo(int n) {

    int i;
    int sum = 0;

    for(i=1; i<n; i++) {
        sum += i;
    }

    return sum;
}

When analyzing the time complexity of an algorithm or a piece of code, only focus on the piece of code that has the most loop execution times.

2. The Rule of Addition

int demo(int n) {

    int i;
    int sum = 0;

    for(i=1; i<n; i++) {
        sum += i;
    }
    
    for(i=1; i<n; i++) {
        int j;
        for (j=1; j<n; j++)
            sum += i;
    }

    return sum;
}

If there are different levels of time complexity in the code, the total time complexity is equal to the time complexity of the code with the largest magnitude.

3. The Law of Multiplication

int demo(int n) {

    int i;
    int sum = 0;

    for(i=1; i<n; i++) {
        int j;
        for (j=1; j<n; j++)
            sum += i;
    }

    return sum;
}

If it is nesting, function call, recursion, etc., you only need to multiply each part.

Fourth, the magnitude of complexity

• Constant order: $O\left(1\right)$

• Logarithmic order: $O(\log n)$

• Linear order: $O\left(n\right)$

• Linear logarithmic order: $O(n\log n)$

• Square order: $O(n^2)$

• Cubic order: $O(n^3)$

• kPower order: $O(n^k)$

• Exponential order: $O\left(2^n\right)$

• Factorial order: $O(n!)$

The above-mentioned different magnitudes can be divided into two categories: polynomial magnitude and non-polynomial magnitude . Among them, there are only two non-polynomial magnitudes: $O\left(2^n\right)$and$O(the\; n-!)$ , Also known as non-polynomialNPproblems.

Under normal circumstances, our common complexity is only $O(1)$、$O(\log n)$、$O(n)$、$O\left(n\log n\right)$、$O(n^2)\; For$these five, the commonly used analysis methods include the maximum order, the rule of addition, and the rule of multiplication. As long as these are mastered, there is basically no big problem.

Five, time complexity

We have analyzed the time complexity, but still a little small problem, for example, we want to find an element of length nindex of the array. If you traverse in order, the ideal situation is that the first one is what we are looking for, so the time complexity is O(1); if the last one finds the data we want, then its time complexity is O(n).

In order to solve the same order of magnitude difference in time a piece of code complexity appear in different situations, we need to further refine the classification of time complexity, in order to more accurate and more comprehensive description of the time complexity of the code, introduced about 4a concept.

1. Best-case time complexity

The time complexity of code execution in the most ideal case.

2. Worst-case time complexity

The time complexity of code execution in the worst case.

3. Average case time complexity

The above two best and worst cases are small probability events, and the average case time complexity is the most representative of the time complexity of an algorithm. Because the time complexity of the average case requires the introduction of probability for analysis, it is also called the weighted average time complexity .

4. Amortized time complexity

Under normal circumstances, the code is at low-level complexity during execution, and high-level complexity will appear in very rare cases. This is how we can evenly spread the high-level complexity to each low-level complexity. This analysis uses the idea of amortized analysis .

In fact, we only need to know the time complexity. These four methods are all supplements to some special cases of time complexity, and there is no need to spend a lot of effort to study it. You probably know that there is this type of time complexity classification. If you want to learn it yourself or have a brain-disabled interviewer If you want to ask these, then you can find the information and research by yourself. I will not explain it here.

Six, space complexity

As explained earlier, time complexity is progressive time complexity, which represents the growth relationship between the execution time of an algorithm and the scale of data. Then the space complexity is the progressive space complexity, which represents the growth relationship between the storage space of the algorithm and the data scale.

Look at a piece of code, define a new array, and traverse the output after assignment.

void demo(int n) {

    int i;
    int data[n];

    for(i=0; i<n; i++) {
        data[i] = i * i;
    }

    for(i=0; i<n; i++) {
        printf("%d\n", data[i]);
    }
}

With time complexity analysis, the function of the body of 1the statement is a constant order, ignore; first 2statements apply a size nof intan array type, so that the whole space of code complexity is $O\left(n\right)$。