Introduction to Algorithms (DP III: Parenthesization, Edit distance, Knapsack)

* Subproblems for strings/sequences

• suffixes x[i:] i
• prefixes x[:i] i
• substrings x[i:j] ​​​​​​​i < j

Parenthesization

Optimal evaluation of associative expression A[0] · A[1] · · · A[n − 1] — e.g., multiplying rectangular matrices

• 2. guessing = outermost multiplication ( · · · )( · · · ) 
• 1. subproblems = prefixes & suffixes? NO = cost of substring A[i : j] 
• 3. recurrence:
  • DP[i, j] = min(DP [i, k] + DP[k, j]+ cost of multiplying (A[i] · · · A[k − 1]) by (A[k] · · · A[j − 1]) for k in range(i + 1, j)) i j i k j DAG
  • DP[i, i + 1] = 0
• 4. topological order: increasing substring size. Total time = O(n^3)
• 5. original problem = DP[0, n]

Edit distance

given two strings x&y, what's the cheapest possible sequence of character edit to (insert c, delete c, replace c → c’) to transform x into y?

• If insert & delete cost 1, replace costs 0, minimum edit distance equivalent to finding longest common subsequence. Note that a subsequence is sequential but not necessarily contiguous.

1. subproblems: c(i, j) = edit-distance(x[i :], y[j :]) for 0 ≤ i < |x|, 0 ≤ j < |y| =⇒ Θ(|x| · |y|) subproblems
2. guess whether, to turn x into y, (3 choices):
   1. x[i] deleted
   2. y[j] inserted
   3. x[i] replaced by y[j]
3. recurrence: c(i, j) = minimum of:
   1. cost(delete x[i]) + c(i + 1, j) if i < |x|,
   2. cost(insert y[j]) + c(i, j + 1) if j < |y|,
   3. cost(replace x[i] → y[j]) + c(i + 1, j + 1) if i < |x|&j < |y|
   4. base case: c(|x|, |y|) = 0 ⇒ Θ(1) time per subproblem
4. topological order: DAG in 2D table:
   1. bottom-up OR right to left
   2. only need to keep last 2 rows/columns ⇒ linear space
   3. total time = Θ(|x| · |y|)
5. original problem: c(0, 0)

Knapsack

Knapsack of size S you want to pack

• item i has integer size si & real value vi
• goal: choose subset of items of maximum total value subject to total size ≤ S

1. subproblem = value for suffix i: given knapsack of size X ⇒ # subproblems = O(nS)
2. recurrence:
   1. DP[i, X] = max(DP[i + 1, X], vi + DP[i + 1, X − si ] if si ≤ X)
   2. DP[n, X] = 0 ⇒ time per subproblem = O(1)
3. topological order: for i in n, . . . , 0: for X in 0, . . . S total time = O(nS)
4. original problem = DP[0, S] (& use parent pointers to recover subset)

Polynomial time

Polynomial time = polynomial in input size​​​​​​​

• here Θ(n) if number S fits in a word
• O(n lg S) in general
• S is exponential in lg S (not polynomial)

Pseudopolynomial Time

Pseudopolynomial time = polynomial in the problem size AND the numbers (here: S, si ’s, vi ’s) in input. Θ(nS) is pseudopolynomial.