[Mathematics] 3. Dynamic programming

1. Principle

Following the topic of text search, let’s talk about the implementation process of query recommendation (Query Suggestion), and the mathematical ideas it uses, Dynamic Programming (Dynamic Programming).

What is dynamic programming? In the recursion section, we can simplify complex tasks into the most basic small problems by continuously decomposing the problem, such as merge sort, permutation and combination based on recursion.

But sometimes, we don't have to deal with all possible situations, just find the optimal solution that satisfies the conditions. In this case, we need to find out those local solutions that may reach the optimum among various possible local solutions, and give up other local solutions. This process of finding the optimal solution is actually dynamic programming.

Dynamic programming needs to derive the optimal solution of the final problem through the optimal solution of the sub-problems, so this method pays special attention to the transfer relationship between the sub-problems. We usually refer to the transitions between these sub-problems as state transitions, and the expressions used to describe these state transitions are called state transition equations. Obviously, finding the appropriate state transition equation is the key to dynamic programming.

1.1 How to think of dp

If you just look at the case, maybe you think dynamic programming is not difficult to understand. However, in practical applications, you may have these questions: When should dynamic programming be used? This problem can be solved with dynamic programming, why didn't I think of it? Here are some personal experiences.

• First of all, if there are many possibilities for a problem, it seems that you need to use the idea of ​​permutation or combination, but the final solution is only some optimal solution (such as minimum value, maximum value, shortest substring, longest substring, etc.), then You might as well try to see if you can use dynamic programming.
• Second, the state transition equation is the key. You can use state transition tables to help you understand the process. If the exact transfer equation can be found, then the final code implementation is not far away. Of course, the best way is to combine the projects at work, practice, try, and then summarize.

Second, the case

The following uses actual cases to explain in detail how to use the dynamic programming method to find the optimal solution, including how to decompose the problem, discover the law of state transition, and define the state transition equation.

2.1 Edit distance

When you enter a word in the search box of a search engine, have you noticed that the search engine will return a series of related keywords, which is convenient for you to click directly. Even, when you enter a word incorrectly, the search engine will still return the correct search results.

Search drop-down tips and keyword error correction, these two functions are actually query recommendations. The core idea of ​​query recommendation is actually to find similar keywords for user input and return them. The most commonly used indicator for measuring Latin text similarity is Edit Distance.

As I just said, the two functions recommended by the query are to return corresponding results for strings with missing or wrong inputs. So, how does the process of converting the wrong string into the correct one to return the query result?

The minimum number of editing operations required to convert a string into another string is called the edit distance. This concept was proposed by Russian scientist Levenshtein, so we also call the edit distance Levenshtein distance. Obviously, the smaller the edit distance, the more similar the two strings are, and they can be recommended as queries to each other. There are three types of editing operations: replace one character with another; insert a character; delete a character.

For example, if we want to convert mouuse to mouse, there are many ways to achieve it, but obviously, it is easiest to delete a "u" directly, so the edit distance between the two is 1.

2.1.1 State transition

For the example of mouse and mouuse, we can quickly observe with the naked eye that the edit distance is 1. But in our real scene, it is often not so simple. Given any two very complex strings, how to efficiently calculate the edit distance between them?

We talked about permutations and combinations earlier. Let's first try to use the idea of ​​arrangement for editing operations.

• For example, to replace one character with another, we can think of replacing a character in A with a character in B. Suppose there are m different characters in B, then there are m possibilities when replacing.
• For inserting a character, we can think of inserting a character from B in A, and assuming that there are m different characters in B, then there are m possibilities.
• As for deleting a character, we can think of deleting any character in A, assuming that A has n different characters, then there are n possibilities.

However, when it is realized, you will find that the actual situation is much more complicated than imagined.

• First, the amount of calculation is very large. We assume that the length of string A is n, and the number of different characters in string B is m, then all possible permutations of A are roughly in the order of m^n, which will lead to a very long processing time. For real-time services such as query and recommendation, the response time of the server is too long, which is definitely unacceptable to users.
• Secondly, if characters need to be added to string A, how many characters should be added and where should they be added? The same goes for deleting characters. Therefore, the possible permutations are far more than m^n.

We now return to the question itself. In fact, the edit distance only requires the minimum number of operations, and does not require listing all possibilities. Moreover, the arrangement process is very error-prone and wastes a lot of computing resources. It seems that the permutation method is not feasible.

Well, let's think about it again here. In fact, we don't need all the possibilities of the arrangement, but only care about the optimal solution, that is, the shortest distance. So, can we choose an optimal solution so far each time, and only keep this optimal solution? If this is the case, although we still use iterative or recursive programming to achieve it, the efficiency can be improved a lot.

• Let's consider the simplest case first. Assuming that strings A and B are both empty strings, it is obvious that the edit distance is 0 at this time. If A adds a character a1 and B remains unchanged, the edit distance increases by 1. Similarly, if B increases by one character b1, A remains unchanged and the edit distance increases by 1.
• However, if A and B have a character, then the problem is a bit more complicated, and we can break it down into the following cases.
  • Let's look at the case of inserting characters first. When the A string is a1, add one character to the B empty string to change it to b1; or when the B string is b1, add one character to the A empty string to change it to a1. Obviously, in this case, the edit distance must be increased by 1.
  • Look at the replacement character situation. When both A and B are empty strings, add a character at the same time. If the characters a1 and b1 to be added are not equal, it means that characters need to be replaced when converting between A and B, then the edit distance is increased by 1; if a1 and b1 are equal, no replacement is required, then the edit distance remains unchanged.
  • There is no need to consider deletions, because deletions are the inverse of insertions. If we start with the complete string A or B instead of an empty string, this is a delete operation.
  • Finally, take the minimum value of the edit distance in the above three cases as the current edit distance. Note that here we [only need to keep this minimum value, and discard other larger values]. Why is this? Because the edit distance increases monotonically as the string grows. Therefore, to require the final minimum value, it must be ensured that the minimum value is obtained for each substring. With this in mind, we can use iterative methods to deduce step by step until the comparison of the two strings ends.
  • It can be seen from the above process that we can indeed divide the complex problem of finding the edit distance into more and smaller sub-problems. Moreover, the more important point is, [We only need to keep one optimal solution in each sub-problem. Subsequent problem solving depends only on this optimal value]. This method of finding the edit distance is dynamic programming, and [these sub-problems are called different states in dynamic programming].

If the text description is not very clear, I will draw another table here to clearly mark the transitions between each state, and you will be clear at a glance.

Still using the example of mouuse and mouse.

• Take the character array of mouuse as the rows of the table, and each row represents one of the letters (the first row in the figure below is the string B)
• The character array of mouse is used as a column, and each column represents one of the letters (the first column in the figure below is the string A)
• This results in the following form:
  • The situation from (0, 0) to (m, m) is as follows:
    - The first two insertion cases are empty string insertion, A and B each insert an m, and a total of two insertions are made, so the edit distance is 0+1 +1=2; because the edit distance 2 means that A adds an m character, and B adds another m character. Although the two are equal from the human point of view, it is necessary for the computer to traverse this situation.
    • For the third replacement situation: both A and B are inserted into m, no replacement is required, so there is no replacement operation in the whole process, so the edit distance is 0+0=0 (PS: how to replace the same letter, then it is 0, otherwise it is 1) .
    • Therefore, the third case is the smallest and can take the minimum value of 0
  • From (0, m) to (0, mo), the possible path is to delete one m, and increase m and o respectively, then the edit distance is 3, but this is not optimal, and the min function will be Take the possibility of directly adding o to m, the distance is 1

The transition between different states in this table is the state transition. The red part indicates the way of string evolution (or state transition) and the corresponding edit distance calculation. For the other blank parts in the table, I will not give them for the time being, you can try to deduce them yourself.

Edit distance is symmetric, that is to say, the edit distance from string A to B and the edit distance from string B to A must be equal. This should be well understood: it is actually determined by the three operations of the edit distance. For example, every operation that evolves from string A to B can be converted into an operation that evolves from string B to A (as shown in the figure below). So, for every change from string A to B, you can find a corresponding way to evolve from string B to A, and the number of operations is the same. Naturally, the edit distance representing the minimum number of operations is the same.

2.1.2 State transition equation and programming implementation

After completing the above table, the complete state transition table is as follows:

There are three parameters in the min function for finding the minimum value in the table, which correspond to the edit distances of the three situations we mentioned in the previous section, namely: replacement, insertion and deletion of characters. In the lower right corner of the table I marked the edit distance 1 between the two strings.

The concept and analysis process are understood. As a programmer, you still have to settle on coding in the end. Here I will take you to do some preparatory work before coding.

Assume that the character arrays A[] and B[] represent strings A and B respectively, A[i] represents the character at position i in string A, and B[i] represents the character at position i in string B. The two-dimensional array d[,] represents the two-dimensional table just used for derivation, and d[i,j] represents the final edit distance obtained from row i and column j in this table. The function r(i, j) represents the edit distance produced when replacing. If A[i] and B[j] are the same, the return value of the function is 0, otherwise the return value is 1.

With these definitions, we use iteration to express the above derivation process below.

• If i is 0 and j is also 0, then d[i, j] is 0.
• If i is 0 and j is greater than 0, then d[i, j] is j.
• If i is greater than 0 and j is 0, then d[i, j] is i.
• If i is greater than 0, and j is greater than 0, then d[i, j]=min(d[i-1, j] + 1, d[i, j-1] + 1, d[i-1, j- 1] + r(i, j)). This expression represents some changes that may exist between the previous state and the next state in dynamic programming, and the final decision results based on these changes. We call such expressions state transition equations.

With the state transition equation, we can clearly describe the state transition and its corresponding decision-making process in a mathematical way. Moreover, with the state transition equation, the specific coding is actually very easy. Based on the state transition equation of edit distance, I have listed an encoded implementation here, you can take a look.

// 首先要定义函数的参数和返回值，需要注意判断一下 a 和 b 为 null 的情况。
 public class Lesson10_1 {
    
    
	/**
    * @Description:	使用状态转移方程，计算两个字符串之间的编辑距离
    * @param a- 第一个字符串，b- 第二个字符串
    * @return int- 两者之间的编辑距离
    */
	public static int getStrDistance(String a, String b) {
    
    
		if (a == null || b == null) return -1; // 防御式编程, 无效的空指针
		


// 然后，初始化状态转移表。我用 int 型的二维数组来表示这个状态转移表，并对 i 为 0 且 j 大于 0 的元素，以及 i 大于 0 且 j 为 0 的元素，赋予相应的初始值。
		int[][] d = new int[a.length() + 1][b.length() + 1]; // 初始用于记录化状态转移的二维表
		for (int j = 0; j <= b.length(); j++) {
    
     // 如果 i 为 0，且 j 大于等于 0，那么 d[i, j] 为 j
			d[0][j] = j;
		}
		for (int i = 0; i <= a.length(); i++) {
    
     // 如果 i 大于等于 0，且 j 为 0，那么 d[i, j] 为 i
			d[i][0] = i;
		}
		
		

// 我这里实现的时候，i 和 j 都是从 0 开始，所以我计算的 d[i+1, j+1]，而不是 d[i, j]。而 d[i+1, j+1] = min(d[i, j+1] + 1, d[i+1, j] + 1, d[i, j] + r(i, j)。		
		// 实现状态转移方程
		// 请注意由于 Java 语言实现的关系，代码里的状态转移是从 d[i, j] 到 d[i+1, j+1]，而不是从 d[i-1, j-1] 到 d[i, j]。本质上是一样的。
		for (int i = 0; i < a.length(); i++) {
    
    
			for (int j = 0; j < b.length(); j++) {
    
    
				
				int r = 0;
				if (a.charAt(i) != b.charAt(j)) {
    
    
					r = 1;
				} 
				
				int first_append = d[i][j + 1] + 1;
				int second_append = d[i + 1][j] + 1;
				int replace = d[i][j] + r;
				
				int min = Math.min(first_append, second_append);
				min = Math.min(min, replace);
				d[i + 1][j + 1] = min;
			}
		}
		return d[a.length()][b.length()];	
	}
}



// 最后，我们用测试代码测试不同字符串之间的编辑距离。
public static void main(String[] args) {
    
    
  System.out.println(getStrDistance("mouse", "mouuse"));
}


// 从推导的表格和最终的代码可以看出，我们相互比较长度为 m 和 n 的两个字符串，一共需要求 m * n 个子问题，因此计算量是 m * n 这个数量级。和排列法的 m^n 相比，这已经降低太多太多了。
// 我们现在可以快速计算出编辑距离，所以就能使用这个距离作为衡量字符串之间相似度的一个标准，然后就可以进行查询推荐了。
// 到这里，使用动态规划来实现的编辑距离其实就讲完了。我把两个字符串比较的问题，分解成很多子串进行比较的子问题，然后使用状态转移方程来描述状态（也就是子问题）之间的关系，并根据问题的定义，保留最小的值作为当前的编辑距离，直到过程结束。
// 如果我们使用动态规划法来实现编辑距离的测算，那就能确保查询推荐的效率和效果。不过，基于编辑距离的算法也有局限性，它只适用于拉丁语系的相似度衡量，所以通常只用于英文或者拼音相关的查询。如果是在中文这种亚洲语系中，差一个汉字（或字符）语义就会差很远，所以并不适合使用基于编辑距离的算法。

2.2 Coin Combinations

Compared with exhaustive methods such as permutations and combinations, dynamic programming methods focus on finding some optimal solution. If a problem does not require all possible solutions, but to find the optimal solution that satisfies certain conditions, then you can think about whether dynamic programming can be used to reduce the workload of the solution.

Did we mention the story of the new version of King Shekhan's reward? The king needs to pay a certain amount of bounty, and the prime minister has to list all possible combinations of coins, which uses the idea of ​​permutation and combination. If this problem is changed to "given the total amount and possible coin denominations, can we find the reward method with the least number of coins?", then can we use dynamic programming?

The idea is similar to before. We first decompose this problem into many smaller sub-problems, and then try to find the state transition equation. If a coin c is added, then the current total number of coins is the total number of coins before adding c plus the current one. For example, suppose here we have three denominations of coins, 2 yuan, 3 yuan and 7 yuan. To round up the $100 total, we have three options.

• The first type, the sum of 98 yuan coins, plus a 2 yuan coin. If the minimum number of coins to make up 98 yuan is x1, then adding one 2 yuan will be (x1 + 1) coins.
• The second type, the sum of 97 yuan coins, plus a 3 yuan coin. If the minimum number of coins to get to 97 yuan is x2, then after adding 3 yuan, it will be (x2 + 1) coins.
• The third type, the sum of 93 yuan coins, plus a 7 yuan coin. If the minimum number of coins to get to 93 yuan is x3, then after adding one 7 yuan, it will be (x3 + 1) coins.

Compare the total number of coins in the above three situations, and the smallest one is the smallest number of coins when the total amount is 100 yuan. In other words, since the total amount of rewards is fixed, the denomination of the coin selected last will determine the amount up to the previous step, and also determine the minimum number of coins up to the previous step. According to this, we can derive the following state transition equation:

Among them, c[i] represents the minimum number of coins required when the total amount is i, where j=1,2,3,...,n represents coins of n denominations, and value[j] represents the value of the jth coin denomination. c[i - values(j)] indicates the minimum number of coins up to the previous step when selecting the jth coin. It should be noted that i - value(j) needs to be greater than or equal to 0, and c[0] = 0.

Here we use this state transition equation to do some derivation. The specific data can be seen in the table below. Each row of the table represents the total amount of rewards, the first 3 columns represent the denominations of 3 coins, and the last column records the minimum number of coins. The "/" in the table means impossible, or no solution.

This state transition table can also help to understand the correctness of the state transition equation. Once the state transition equation is determined, it is not difficult to write code to realize it. Refer to LeetCode 322. Change Exchange .