An article allows you to thoroughly learn time complexity and space complexity

Objectives of this section

1. Learn what algorithm efficiency is
2. Learn the representation and calculation method of
time complexity and space complexity 3. In-depth understanding of time complexity and space complexity
4. Can independently calculate the time and space complexity of common algorithms in C language

1. Algorithm Efficiency

There are two types of algorithm efficiency analysis: the first is time efficiency, and the second is space efficiency. Time efficiency is called time complexity, while space efficiency
is called space complexity. Time complexity mainly measures the running speed of an algorithm, while space complexity mainly measures the extra space required by an algorithm . In the early days of computer development, the storage capacity of computers was very small. So it is very concerned about the space complexity. But after
the rapid development of the computer industry, the storage capacity of the computer has reached a very high level. So we no longer need to pay special attention
to the space complexity of an algorithm.

2. Time complexity

1. The concept of time complexity

Definition of time complexity: In computer science, the time complexity of an algorithm is a function that quantitatively describes the running time of the algorithm. The time it takes for an
algorithm to execute, theoretically speaking, cannot be calculated, only when you run your program on a machine can you know it
. But do we need to test every algorithm on the machine?
It is possible to test all on the computer, but it is very troublesome, so there is the analysis method of time complexity . The time spent by an algorithm is proportional to the number of executions of the statements in it, and the number of executions of the basic operations in the algorithm is the time complexity of the algorithm .

2. Representation method of time complexity

When we calculate the time complexity, we do not calculate the specific number of times the algorithm runs, but use the asymptotic representation of big O to calculate it. The specific calculation method is as follows:

• Replace all additive constants in runtime with the constant 1.
• In the modified run times function, only the highest order terms are kept.
• If the highest-order term exists and is not 1, remove the constant multiplied by this term.

3. Simple time complexity calculation

Example 1

void Func(int N)
{
    
    
int count = 0;
for (int k = 0; k < 100; ++ k)
{
    
    
 ++count;
}
printf("%d\n", count);
}

The number of times the above program is executed: 100
The asymptotic representation of big O is used to get the time complexity: O(1) (Replace all the addition constants in the running time with the constant 1.)

Example 2

void Func1(int N)
{
    
    
	int count = 0;
	for (int i = 0; i < N; ++i)
	{
    
    
		for (int j = 0; j < N; ++j)
		{
    
    
			++count;
		}
	}
	for (int k = 0; k < 2 * N; ++k)
	{
    
    
		++count;
	}
	int M = 10;
	while (M--)
	{
    
    
		++count;
	}
	printf("%d\n", count);
}

The number of times the above program is executed: N * N + 2*N + 10
The asymptotic representation of big O is used to obtain the time complexity: O(N^2) (only the highest order item is retained)

Example three

void Func2(int N)
{
    
    
int count = 0;
for (int k = 0; k < 2 * N ; ++ k)
{
    
    
 ++count;
}
int M = 10;
while (M--)
{
    
    
 ++count;
}
printf("%d\n", count);
}

The number of times the above program is executed: 2*N + 10
The asymptotic representation of big O is used to obtain the time complexity: O(N) (if the highest-order item exists and is not 1, remove the constant multiplied by this item)

There are three cases of algorithm complexity :

• Worst case: maximum number of runs for any input size (upper bound)
• Average case: desired number of runs for any input size
• Best case: minimum number of runs for any input size (lower bound)

For example: search for a data x in an array of length N
Best case: 1 find
Average case: N/2 times find
Worst case: N times find
Average case: N/2 times find
what is generally concerned in practice is the worst-case operation of the algorithm , so the time complexity of searching for data in the array is O(N).

4. Calculation of complex time complexity

(1) Time complexity of bubble sort

void  BubbleSort(int arr[], int n)
{
    
    
	int i = 0;
	int j = 0;
	for (i = 0; i < n - 1; i++)
	{
    
    
		for (j = 0; j < n - i - 1; j++)
		{
    
    
			if (arr[j] > arr[j + 1])
			{
    
    
				int temp = arr[j];
				arr[j] = arr[j + 1];
				arr[j + 1] = temp;
			}
		}
	}
}

The specific number of executions: the worst case is the time complexity calculation, assuming that the array is in reverse order at the beginning, then the first sorting is performed n-1 times, the second sorting is performed n-2 times, and the third is performed n-1 times -3 times ... until the order is finally achieved, so the specific execution times of bubble sorting is an arithmetic sequence, specific times = (first item + last item)/2*Number of items = (N^2-N)/ 2
Using the asymptotic representation of big O to get the time complexity: O(N^2)

(2) Time complexity of binary search

int BinarySearch(int arr[], int n, int x) //n元素个数  x:要查找的数
{
    
    
	int left = 0;
	int right = n - 1;
	while (left < right)
	{
    
    
		int mid = (left + right) / 2;
		if (arr[mid] > x)
		{
    
    
			right = mid - 1;     //中间元素比x大就挪动right下标
		}
		else if (arr[mid] < x)
		{
    
    
			left = mid + 1;    //中间元素比x小就挪动left下标
		}
		else
			return mid;     //找到就返回该元素所在的下标
	}
	return 0;               //找不到就返回0
}

Specific execution times: As above, the worst case is considered here, that is, there is no element you want to find in the array, the array will start searching from the middle, and half of the elements will be excluded each time until all the elements are excluded. After the first exclusion 1/2 of the elements are left, 1/4 of the elements are left after the second exclusion, 1/8 of the elements are left after the third exclusion ... , let the number of elements be N and the number of searches be X, then X * ( ½)^N = 1 -> (½)^N = X -> Specific times: X = log ₂ N
Time complexity in big O asymptotic notation: O(logN)
Note: Because it cannot be represented on the keyboard To get the base of log, the base 2 of log is omitted from the time complexity, and logN is used to directly represent the time complexity of log _{2 N.}

(3) Time complexity of factorial recursion

long long Factorial(int N)
{
    
    
	return N < 2 ? N : Factorial(N - 1) * N;
}

Specific times: Here n calls n-1, n-1 calls n-2 ... until n = 1, so a total of n-1 times are executed.
Time complexity in big-O asymptotic notation: O(N)

Example with factorial of five:

(4) Time complexity of Fibonacci recursion

long long Fibonacci(size_t N)
{
    
    
	return N < 2 ? N : Fibonacci(N - 1) + Fibonacci(N - 2);
}

Specific times: Take the above figure as an example, we can see that when the value is greater than 2, the number of calls of each layer increases exponentially by 2, which is a proportional sequence, so it is obtained by the asymptotic representation of big O. The time complexity is: O(2^N)

5. Comparison of efficiency of different time complexity

We can see that the efficiency of O(logN) and O(1) is almost the same when the test data is large, so binary search is a very efficient algorithm, but it also has a flaw, that is, the array it operates on Elements must be ordered.

3. Space complexity

1. The concept of space complexity

Space complexity is a measure of the amount of storage space temporarily occupied by an algorithm during its execution. The space complexity is not how many bytes the program occupies, because this does not make much sense, so the space complexity is the number of variables . The space complexity calculation rules are basically similar to the practical complexity, and also use the big-O asymptotic notation.

2. Calculation method of space complexity

The calculation method of space complexity and time complexity are very similar, and both are expressed in big-O asymptotic notation.
The specific calculation method is as follows:

• Replace the constant variables defined during runtime with the constant 1.
• In the modified run times function, only the highest order terms are kept.
• If the highest-order term exists and is not 1, remove the constant multiplied by this term.

3. Calculation of common space complexity

(1) Space complexity of bubble sort

void  BubbleSort(int arr[], int n)
{
    
    
	int i = 0;
	int j = 0;
	for (i = 0; i < n - 1; i++)
	{
    
    
		for (j = 0; j < n - i - 1; j++)
		{
    
    
			if (arr[j] > arr[j + 1])
			{
    
    
				int temp = arr[j];
				arr[j] = arr[j + 1];
				arr[j + 1] = temp;
			}
		}
	}
}

The space complexity is calculated by the number of variables defined. Three variables are defined here, so the space complexity is O(1) .

(2) Space complexity of binary search

int BinarySearch(int arr[], int n, int x) //n元素个数  x:要查找的数
{
    
    
	int left = 0;
	int right = n - 1;
	while (left < right)
	{
    
    
		int mid = (left + right) / 2;
		if (arr[mid] > x)
		{
    
    
			right = mid - 1;     //中间元素比x大就挪动right下标
		}
		else if (arr[mid] < x)
		{
    
    
			left = mid + 1;    //中间元素比x小就挪动left下标
		}
		else
			return mid;     //找到就返回该元素所在的下标
	}
	return 0;               //找不到就返回0
}

Like the space complexity of bubble sort, only three (constant) variables are defined here, so the space complexity is O(1) .

(3) The space complexity of factorial recursion

long long Factorial(int N)
{
    
    
	return N < 2 ? N : Factorial(N - 1) * N;
}

When calculating the space complexity of recursive classes, we look at the depth of recursion. Here, every time we call Factorial, we will open up a space on the stack area, and the space opened up before n = 1 will not be released, only wait until n = 1, the space opened up when the recursion starts to return step by step will be gradually released, so the depth of the call here is n - 1 (recursion n - 1 times), so the space complexity is O(N) .

(4) Space complexity of Fibonacci recursion

long long Fibonacci(size_t N)
{
    
    
	return N < 2 ? N : Fibonacci(N - 1) + Fibonacci(N - 2);
}

To solve this problem, we must first be clear that Fibonacci recurses branch by branch. Take the above figure as an example, it will first recurse 6-5-4-3-2-1, and then recurse 6-5-4 -3-2, then recurse 6-5-4-2, and so on, until the last branch is recursed, and we also know that the recursive call opens up space on the stack layer by layer according to the recursion depth, and when the recursion returns And release space layer by layer , that is, when we recurse 6-5-4-3-1, the recursion depth (space complexity) is five, and when we recurse 6-5-4-3-1, The space complexity will be calculated from 0 again, and the calculation of the complexity will look at the worst case, where the maximum number of recursion is n - 1, so the space complexity is O(N) .

4. Summary

• Both time complexity and space complexity are expressed in big-O asymptotic notation.
• The time complexity depends on the number of operations performed, and the space complexity depends on the number of variable definitions.
• The time complexity of recursion depends on the number of calls, and the space complexity depends on the depth of the call.
• The time complexity of bubble sort is O(N^2) and the space complexity is O(1).
• The time complexity of binary search is O(logN) and the space complexity is O(1).
• The time complexity of factorial recursion is O(N) and the space complexity is O(N).
• The time complexity of Fibonacci recursion is O(2^N) and the space complexity is O(N).