Dynamic Programming: The Elegant Craftsmanship of "Exchanging the Universe for the Universe"

Those who do not remember the past are condemned to repeat it.

Those who do not remember the past are condemned to repeat it.

InDynamic Programming - Classic Case Analysis we mentioned the solution to the Fibonacci sequence. Know that the main advantage of dynamic programming is the ability to avoid double calculations when solving problems, by utilizing results that have already been calculated to speed up the solution process.

How is the recursive operation of the Fibonacci sequence completed? What repetitive calculations can we save by using dynamic programming?
These issues are what this article will study and discuss.

The Fibonacci sequence is a classic mathematical problem defined as follows:

- F(0) = 0, F(1) = 1
- F(n) = F(n-1) + F(n-2)，其中 n > 1

If recursion is used to solve the problem, the code is as follows

long fib(int n){
    
    
	if (n==0) return 0;
	if (n==1) return 1;
	return fib(n-1) + fib(n-2);
}

The time complexity of the algorithm at this time is $F(n)=\Theta ({\varphi }^n)$，其中 $\varphi =\frac{\sqrt{5}+1}{2}\approx 1.618$

Take fib(7) as an example to analyze its recursive call tree

It can be seen that there are many repeated calculations, among which fib(2) is repeated 8 times; fib(3) is repeated 5 times. But we don’t need to repeat the calculation $F_k$If the value of has been calculated $F_k$The value of may be stored and used directly when needed next time. The storage space of intermediate calculation results can be vividly called memo .

---

#define UNKNOWN -1
std::vector<long> f;
long fib_m(int n) {
    
    
	if (f[n] == UNKNOWN)
		f[n] = fib_m(n - 1) + fib_m(n - 2);
	return f[n];
}

long fib_m_driver(int n) {
    
    
	f = std::vector<long>(n + 1，UNKNOWN);
	f[0] = 0; f[1] = 1;
	return fib_m(n);
}

transfer entry forgetting empty space $f\left[n\right]$ , we can recurse A lot of double counting pivots are eliminated. At this time, the time complexity of the algorithm is $O\left(n\right)$; $O(n)$。

There is still room for optimization after reaching this point. For each fib(n), we actually only need to know fib(n-1) and fib(n-2), so there is no need to save the values ​​from fib(0) to fib(n-3). So the space complexity can be further reduced to $O(1)$

---

long fib_o(int n) {
    
    
	long prev = 1, curr = 0, next;
	for (int i = 1; i <= n; i++) {
    
    
		next = curr + prev;
		prev = curr;
		curr = next;
	}
	return curr;
}

This is bottom-up active fill memo . This step requires deciding how to populate the memo based on the recursive relationship. By filling in the memo from the bottom up, we eliminate the time and space overhead of recursive calls.

---

summary

• Motivation: Repeatedly solving subproblems in a recursive solution
• Strategy: Exchange space for time and store the solution to the sub-problem to avoid repeated calculations
  • Space: number of subproblems
  • The number of subproblems is determined by the parameters of the subproblem
    The subproblem of Fibonacci numbers has only one parameter and O i< n
• The key to dynamic programming: finding correct and efficient recursive relationships
• Solution method
  1. Top-down: Passively fill memos, and recursive calls determine the filling order of memos.
  2. Bottom-up: Actively fill in the memo, you need to decide how to fill the memo based on the recursive relationship
     Bottom-up does not have the time and space overhead of recursive calls

---