The knapsack problem is a very good introductory topic to dynamic programming and is the most common problem form in dynamic programming. It has many variants, such as 0-1 knapsack problem, complete knapsack problem, multiple knapsack problem, etc.
In this section, we first solve the most common 0-1 knapsack problem.
!!! question
给定 $n$ 个物品，第 $i$ 个物品的重量为 $wgt[i-1]$、价值为 $val[i-1]$ ，和一个容量为 $cap$ 的背包。每个物品只能选择一次，问在不超过背包容量下能放入物品的最大价值。
Observe the picture below, since item number iii from1 1Counting starts at 1 , and the array index starts from 0 00 starts counting, so itemiii corresponds to weightwgt [ i − 1 ] wgt[i-1]wgt[i−1 ] and the valueval [ i − 1 ] val[i-1]val[i−1] 。
We can think of the 0-1 knapsack problem as a problem consisting of nIt is a process consisting of n rounds of decision-making. Each object has two decisions: not to put it in or to put it in, so the problem satisfies the decision tree model.
The goal of this problem is to solve the "maximum value under limited backpack capacity", so it is most likely a dynamic programming problem.
Step 1: Think about the decision-making in each round, define the state, and get dp dpd p table
For each item, if it is not placed in the backpack, the backpack capacity remains unchanged; if it is placed in the backpack, the backpack capacity is reduced. From this we can get the status definition: current item number iii and remaining backpack capacityccc , recorded as[i, c] [i, c][i,c] 。
state [i,c][i,c][i,c ] The corresponding sub-problem is:former iii items have a remaining capacity ofccThe maximum value in c ’s backpack is denoted asdp [i, c] dp[i, c]dp[i,c] 。
What is to be solved is dp [ n , cap ] dp[n, cap]dp[n,c a p ] , so a dimension of( n + 1 ) × ( cap + 1 ) (n+1) \times (cap+1) is(n+1)×(cap+1 ) Two-dimensionaldp dpd p table.
Step 2: Find the optimal substructure and derive the state transition equation
When we make item iiAfter the decision of i , the remainder isi − 1 i-1i−The decision-making of an item can be divided into the following two situations.
- No items placed iii : The backpack capacity remains unchanged, and the state changes to[i − 1, c] [i-1, c][i−1,c] 。
- Place itemsiii : The backpack capacity decreaseswgt [i − 1] wgt[i-1]wgt[i−1 ] , the value increasesval [ i − 1 ] val[i-1]val[i−1 ] , the state change is[ i − 1 , c − wgt [ i − 1 ] ] [i-1, c-wgt[i-1]][i−1,c−wgt[i−1]] 。
The above analysis reveals to us the optimal substructure of this question: maximum value dp [i, c] dp[i, c]dp[i,c ] is equivalent to not placing itemsiii and put itemsiiWhich of the two options is more valuable. From this, the state transition equation can be derived:
d p [ i , c ] = max ( d p [ i − 1 , c ] , d p [ i − 1 , c − w g t [ i − 1 ] ] + v a l [ i − 1 ] ) dp[i, c] = \max(dp[i-1, c], dp[i-1, c - wgt[i-1]] + val[i-1]) dp[i,c]=max(dp[i−1,c],dp[i−1,c−wgt[i−1]]+val[i−1])
It should be noted that if the current item weight wgt [ i − 1 ] wgt[i − 1]wgt[i−1 ] Exceeded remaining backpack capacityccc , you can only choose not to put it in the backpack.
Step 3: Determine boundary conditions and state transition sequence
The maximum value when there are no items or remaining backpack capacity is 0 00 , that is, the first columndp [i, 0] dp[i, 0]dp[i,0 ] and the first rowdp [ 0 , c ] dp[0, c]dp[0,c ] are equal to0 00 。
Current state [i, c] [i, c][i,c ] from the upper state[ i − 1 , c ] [i-1, c][i−1,c ] and the upper left state[ i − 1 , c − wgt [ i − 1 ] ] [i-1, c-wgt[i-1]][i−1,c−wgt[i−1 ]] is transferred, so the entiredp dpd p table is enough.
Based on the above analysis, we will implement brute force search, memory search, and dynamic programming solutions in order.
Method 1: Brutal search
The search code contains the following elements.
- Recursion parameters : state [i, c] [i, c][i,c] 。
- Return value : solution to the subproblem dp [i, c] dp[i, c]dp[i,c] 。
- Termination condition : when the item number crosses the boundary i = 0 i = 0i=0 or the remaining capacity of the backpack is0 0At 0 , terminate the recursion and return the value0 00 。
- Pruning : If the current weight of the item exceeds the remaining capacity of the backpack, it will not be placed in the backpack.
=== “Python”
```python title="knapsack.py"
[class]{}-[func]{knapsack_dfs}
```
=== “C++”
```cpp title="knapsack.cpp"
[class]{}-[func]{knapsackDFS}
```
=== “Java”
```java title="knapsack.java"
[class]{knapsack}-[func]{knapsackDFS}
```
=== “C#”
```csharp title="knapsack.cs"
[class]{knapsack}-[func]{knapsackDFS}
```
=== “Go”
```go title="knapsack.go"
[class]{}-[func]{knapsackDFS}
```
=== “Swift”
```swift title="knapsack.swift"
[class]{}-[func]{knapsackDFS}
```
=== “JS”
```javascript title="knapsack.js"
[class]{}-[func]{knapsackDFS}
```
=== “TS”
```typescript title="knapsack.ts"
[class]{}-[func]{knapsackDFS}
```
=== “Dart”
```dart title="knapsack.dart"
[class]{}-[func]{knapsackDFS}
```
=== “Rust”
```rust title="knapsack.rs"
[class]{}-[func]{knapsack_dfs}
```
=== “C”
```c title="knapsack.c"
[class]{}-[func]{knapsackDFS}
```
=== “Zig”
```zig title="knapsack.zig"
[class]{}-[func]{knapsackDFS}
```
As shown in the figure below, since each item will generate two search branches: no selection and selection, the time complexity is O ( 2 n ) O(2^n)O(2n) 。
Observing the recursion tree, it is easy to find that there are overlapping sub-problems, such as dp [1, 10] dp[1, 10]dp[1,10 ] etc. When there are many items and the backpack capacity is large, especially when there are many items of the same weight, the number of overlapping sub-problems will increase significantly.
Method 2: Memorized search
In order to ensure that the overlapping sub-problems are only calculated once, we use the memory list memto record the solutions to the sub-problems, where mem[i][c]corresponds to dp [i, c] dp[i, c]dp[i,c] 。
After memoization is introduced, the time complexity depends on the number of sub-problems , which is O ( n × cap ) O(n \times cap)O ( n×cap) 。
=== “Python”
```python title="knapsack.py"
[class]{}-[func]{knapsack_dfs_mem}
```
=== “C++”
```cpp title="knapsack.cpp"
[class]{}-[func]{knapsackDFSMem}
```
=== “Java”
```java title="knapsack.java"
[class]{knapsack}-[func]{knapsackDFSMem}
```
=== “C#”
```csharp title="knapsack.cs"
[class]{knapsack}-[func]{knapsackDFSMem}
```
=== “Go”
```go title="knapsack.go"
[class]{}-[func]{knapsackDFSMem}
```
=== “Swift”
```swift title="knapsack.swift"
[class]{}-[func]{knapsackDFSMem}
```
=== “JS”
```javascript title="knapsack.js"
[class]{}-[func]{knapsackDFSMem}
```
=== “TS”
```typescript title="knapsack.ts"
[class]{}-[func]{knapsackDFSMem}
```
=== “Dart”
```dart title="knapsack.dart"
[class]{}-[func]{knapsackDFSMem}
```
=== “Rust”
```rust title="knapsack.rs"
[class]{}-[func]{knapsack_dfs_mem}
```
=== “C”
```c title="knapsack.c"
[class]{}-[func]{knapsackDFSMem}
```
=== “Zig”
```zig title="knapsack.zig"
[class]{}-[func]{knapsackDFSMem}
```
The figure below shows the search branch being pruned in memoized recursion.
Method three: dynamic programming
Dynamic programming is essentially filling in dp dp in the state transitionThe process of d p table, the code is as follows.
=== “Python”
```python title="knapsack.py"
[class]{}-[func]{knapsack_dp}
```
=== “C++”
```cpp title="knapsack.cpp"
[class]{}-[func]{knapsackDP}
```
=== “Java”
```java title="knapsack.java"
[class]{knapsack}-[func]{knapsackDP}
```
=== “C#”
```csharp title="knapsack.cs"
[class]{knapsack}-[func]{knapsackDP}
```
=== “Go”
```go title="knapsack.go"
[class]{}-[func]{knapsackDP}
```
=== “Swift”
```swift title="knapsack.swift"
[class]{}-[func]{knapsackDP}
```
=== “JS”
```javascript title="knapsack.js"
[class]{}-[func]{knapsackDP}
```
=== “TS”
```typescript title="knapsack.ts"
[class]{}-[func]{knapsackDP}
```
=== “Dart”
```dart title="knapsack.dart"
[class]{}-[func]{knapsackDP}
```
=== “Rust”
```rust title="knapsack.rs"
[class]{}-[func]{knapsack_dp}
```
=== “C”
```c title="knapsack.c"
[class]{}-[func]{knapsackDP}
```
=== “Zig”
```zig title="knapsack.zig"
[class]{}-[func]{knapsackDP}
```
As shown in the figure below, both time complexity and space complexity are dpdetermined by the size of the array, that is, O ( n × cap ) O(n \times cap)O ( n×cap) 。
=== “<1>”
=== “<2>”
=== “<3>”
=== “<4>”
=== “<5>”
=== “<6>”
=== “<7>”
=== “<8>”
=== “<9>”
=== “<10>”
=== “<11>”
=== “<12>”
=== “<13>”
=== “<14>”
space optimization
Since each state is only related to the state of the row above it, we can use two arrays to scroll forward, reducing the space complexity from O ( n 2 ) O (n^2)O ( n2 )will be as low asO(n) O(n)O ( n )。
Thinking further, can we achieve space optimization with only one array? Observation shows that each state is transferred from the grid directly above or to the upper left. Assuming there is only one array, when starting to traverse the iithWhen row i is reached, the array still stores the i-thi − 1 i-1i−1 row status.
- If positive order traversal is adopted, then traverse to dp [i, j] dp[i, j]dp[i,j ] , upper leftdp [ i − 1 , 1 ] dp[i-1, 1]dp[i−1,1] ~ d p [ i − 1 , j − 1 ] dp[i-1, j-1] dp[i−1,j−1 ] value may have been overwritten, and the correct state transfer result cannot be obtained at this time.
- If reverse order traversal is adopted, the coverage problem will not occur and the state transfer can proceed correctly.
The figure below shows the sequence from i = 1 i = 1 in a single arrayi=Row 1 converts to i = 2 i = 2i=2 lines of process. Please think about the difference between forward traversal and reverse order traversal.
=== “<1>”
=== “<2>”
=== “<3>”
=== “<4>”
=== “<5>”
=== “<6>”
In the code implementation, we only need to change dpthe first dimension of the array iiJust delete i directly and change the inner loop to traverse in reverse order.
=== “Python”
```python title="knapsack.py"
[class]{}-[func]{knapsack_dp_comp}
```
=== “C++”
```cpp title="knapsack.cpp"
[class]{}-[func]{knapsackDPComp}
```
=== “Java”
```java title="knapsack.java"
[class]{knapsack}-[func]{knapsackDPComp}
```
=== “C#”
```csharp title="knapsack.cs"
[class]{knapsack}-[func]{knapsackDPComp}
```
=== “Go”
```go title="knapsack.go"
[class]{}-[func]{knapsackDPComp}
```
=== “Swift”
```swift title="knapsack.swift"
[class]{}-[func]{knapsackDPComp}
```
=== “JS”
```javascript title="knapsack.js"
[class]{}-[func]{knapsackDPComp}
```
=== “TS”
```typescript title="knapsack.ts"
[class]{}-[func]{knapsackDPComp}
```
=== “Dart”
```dart title="knapsack.dart"
[class]{}-[func]{knapsackDPComp}
```
=== “Rust”
```rust title="knapsack.rs"
[class]{}-[func]{knapsack_dp_comp}
```
=== “C”
```c title="knapsack.c"
[class]{}-[func]{knapsackDPComp}
```
=== “Zig”
```zig title="knapsack.zig"
[class]{}-[func]{knapsackDPComp}
```