Experiment 4: Dynamic programming algorithm and backtracking method
Use the dynamic programming algorithm to design and realize the 0-1 knapsack problem, use the backtracking method to design and realize the 0-1 knapsack problem, and use different data volumes for experimental comparison and analysis. It is required to analyze the time complexity of the algorithm and submit relevant code and other documents;
- Problem Description
(1) Knapsack problem
Given a set of items, each item has its own weight and price, within the limited total weight, how do we choose to make the total price of the items the highest. The knapsack problem can also be described as a decisive problem, that is, can the total value reach V under the premise that the total weight does not exceed W? It was proposed by Merkle and Hellman in 1978.
(2) Dynamic programming algorithm
Dynamic programming (Dynamic Programming, DP) is a branch of operations research, which is the process of solving the optimization of the decision-making process. In the early 1950s, American mathematician Bellman (R. Bellman) and others proposed the famous optimization principle when studying the optimization problem of multi-stage decision-making process, thus creating dynamic programming. The application of dynamic programming is extremely wide, including engineering technology, economy, industrial production, military and automation control, etc. significant effect has been achieved in
(3) Backtracking method
The backtracking method (exploration and backtracking method) is an optimal search method, also known as the heuristic method, which searches forward according to the optimal conditions to achieve the goal. But when a certain step is explored, if the original choice is not optimal or the goal cannot be reached, then go back one step and make another choice. Called the "backtrack point".
- Purpose
Master dynamic programming and backtracking design ideas and program design.
- Experimental principle
(1) 0-1 knapsack problem based on dynamic programming algorithm:
Given n items and a knapsack, item i has weight wi, value vi, and knapsack capacity c. How to choose items to maximize the total value of the items in the backpack.
The solution is to use dynamic programming and recursion to decompose a large problem into several small problems.
① Optimal substructure properties:
The 0-1 knapsack problem has optimal substructure properties. One reason is that the total number of items corresponding to several optimal solutions is the total number of optimal solution items remaining after subtracting the weight of an optimal solution.
②Recursive relationship:
Repeatedly seek the optimal solution. The next optimal solution is the previous one:
If the item is not loaded, the optimal solution is skipped directly.
If the item is planned to be loaded, take the maximum value of the value of loading and non-loading.
My function design is consistent with the one in the book, but it is implemented in python, the reason has been explained in the previous experiment. The design is as follows:
The time complexity of Bag function is O(nc)
The time complexity of the Show function is O(n)
(2) The 0-1 knapsack problem designed based on the backtracking method:
Backtracking core:
If you can advance, you will advance; if you cannot enter, you will change; if you cannot change, you will retreat. (According to the conditional depth-first search, when a certain step is found, if it is not optimal or the goal cannot be reached, take a step back and choose again). The currently considered backtracking method does not perform secondary optimization of the algorithm, but only writes the original code.
The solution adopts object-oriented thinking design (in fact, it is because other design schemes are difficult to solve the problem)
Ideas to solve this problem:
Use the 0/1 sequence to represent the placement of items. Think of the search as a binary tree. The i-th layer of the binary tree represents the i-th item. If the remaining space allows item i to be put into the backpack, expand the left subtree. If it cannot be put into the backpack, judge the boundary conditions, and if the subsequent expansion may obtain the optimal value, then expand the right subtree (that is, the i item is not put in, but the subsequent items are considered). When the number of layers reaches the number of items, stop expanding and start backtracking.
Restrictions:
The total mass of items put in the backpack is less than or equal to the backpack capacity
Limit conditions:
The total value of the items currently in the backpack (i and before) + the total value of the items after i < the known optimal value In this case, there is no need to search
- experimental design
4.1 0-1 Knapsack Problem (Dynamic Programming)
The input is the number of items and the weight that the schoolbag can bear, both of which are integers
Also weight and value arrays (python lists)
The output is the optimal value, the items added (in the form of a numbered list)
and the execution time of the program
Execute multiple recording data and graphical analysis.
4.2 0-1 knapsack problem (solution by backtracking method)
Solved using an object-oriented approach.
Because the efficiency of vs code running code is relatively low, pycharm is used this time.
- Experimental results and analysis
5.1 0-1 Knapsack Problem (Dynamic Programming)
First try to execute the program as follows:
No problem, great, start the random number test:
First of all, there is a logical relationship between n and c. If c is too large, most of the items will be selected. If c is too small, the increase of n will be meaningless. As for the value and weight of the item, a random number in the range of 1-n is taken for convenience.
By default, the time of Bag and show takes the average value of 3 times, which is reflected in the table
Ignore any impact of print on runtime
(1) First, do not change n, and let c increase in arithmetic difference:
| c | n | c*n | bag time (ns) | show time (ns) | bag1 | behind2 | bag3 | show1 | show2 | show3 |
| 1000 | 50 | 50000 | 23246633.33 | 1364933.333 | 22898700 | 22937500 | 23903700 | 1102600 | 996800 | 1995400 |
| 1200 | 50 | 60000 | 28243233.33 | 984700 | 29920300 | 25927500 | 28881900 | 999100 | 957800 | 997200 |
| 1400 | 50 | 70000 | 33128433.33 | 979533.3333 | 35489100 | 30984900 | 32911300 | 995900 | 928300 | 1014400 |
| 1600 | 50 | 80000 | 38128733.33 | 959900 | 37474800 | 39948500 | 36962900 | 1002900 | 942000 | 934800 |
| 1800 | 50 | 90000 | 47240066.67 | 964266.6667 | 47907200 | 45941000 | 47872000 | 960000 | 933800 | 999000 |
This is a graph of c*n vs. bag, show time
It can be seen that the increase of c*n value makes the bag time increase linearly, but the value of show is almost unchanged, because the time complexity of bag is O(cn), and the time complexity of show is O(n), while The value of n is controlled unchanged, only the value of c is changed.
(2) Change the value of n to make it grow arithmetically, but do not change the value of c:
| c | n | c*n | bag time (ns) | show time (ns) | bag1 | behind2 | bag3 | show1 | show2 | show3 |
| 2000 | 50 | 100000 | 49222800 | 961433.3 | 47931200 | 47829100 | 51908100 | 936100 | 998600 | 949600 |
| 2000 | 55 | 110000 | 56543767 | 955700 | 50886100 | 58842000 | 59903200 | 937000 | 997300 | 932800 |
| 2000 | 60 | 120000 | 61846667 | 985600 | 64827900 | 63827400 | 56884700 | 999300 | 996900 | 960600 |
| 2000 | 65 | 130000 | 65841567 | 979866.7 | 66872700 | 64827600 | 65824400 | 944300 | 997400 | 997900 |
| 2000 | 70 | 140000 | 67170133 | 985833.3 | 66822400 | 65791100 | 68896900 | 956000 | 1003600 | 997900 |
Show time basically has a linear relationship with the increase of n. Since the data difference is too small, this linear relationship may not be very obvious.
Screenshot of a run of N=50, c=1400:
Screenshot of a run with N=70, c=2000:
5.2 0-1 knapsack problem (backtracking method)
Try running first:
Take a data randomization measure:
(Note: After theoretical proof and multiple trial runs, the efficiency of this algorithm is much lower than that of the dynamic programming algorithm, so do not take too large a number)
Take the backpack weight 200, the value and the value of a single weight randomly from 1 to 50:
| Number of Products |
execution time (ns) |
| 15 |
16806966 |
| 20 |
376673867 |
| 25 |
2410045300 |
| 30 |
14938310733 |
| 35 |
38633429100 |
- in conclusion
Using dynamic programming algorithm and backtracking method to solve the 0-1 knapsack problem, the conclusion has been well verified. In the process of writing the program, I consulted some information on the Internet and books, and I would like to express my gratitude.
- program source code
7.1 Dynamic programming algorithm:
import random
import time
def bag(n, c, w, v):
# 置零，表示初始状态
value = [[0 for j in range(c + 1)] for i in range(n + 1)]
for i in range(1, n + 1):
for j in range(1, c + 1):
value[i][j] = value[i - 1][j]
# 背包总容量够放当前物体，遍历前一个状态考虑是否置换
if j >= w[i - 1] and value[i][j] < value[i - 1][j - w[i - 1]] + v[i - 1]:
value[i][j] = value[i - 1][j - w[i - 1]] + v[i - 1]
print("weight: ")
print(w)
print("value: ")
print(v)
# for x in value:
# print(x)
return value
def show(n, c, w, value):
print('the best value is ', value[n][c])
x = [False for i in range(n)]
j = c
for i in range(n, 0, -1):
if value[i][j] > value[i - 1][j]:
x[i - 1] = True
j -= w[i - 1]
print('the things in the bag: ')
for i in range(n):
if x[i]:
print('NO. ', i+1, ' thing,', end='')
n = 70 # 物品的数量，
c = 2000 # 书包能承受的重量，
w=[0 for _ in range(n)] # the weight of goods
v=[0 for _ in range(n)] # the values of goods
for k in range(n):
w[k]=random.randint(1,n)
v[k]=random.randint(1,n)
# 乱码严重，改为英文输出（utf-8 也会乱码我也是醉了）
t1=time.time_ns()
total = bag(n,c,w,v)
e1=time.time_ns()
t2=time.time_ns()
show(n,c,w,total)
e2=time.time_ns()
print("\n===============================")
print("time of function bag :"+str(e1-t1)+" ns")
print("time of function show :"+str(e2-t2)+" ns")
7.2 Backtracking method
import random
import time
import numpy
class BackSack(): # 定义背包类
def __init__(self, capacity): # 类的初始化
self.capacity = capacity # 背包最大容量（重量）
self.currentWeight = 0 # 背包当前重量
self.bestValue = 0 # 背包可容纳货物的最大价值，最优值
self.currentValue = 0 # 背包内当前已装货物的价值
def Backtrack(self, i): # 遍历解空间寻找最优值，I:当前搜索的深度
global length, weight, value, goods # 全局变量
if (i > length):
if self.currentValue > self.bestValue: # 更新最优值
self.bestValue = self.currentValue
self.currentCapacity = self.currentWeight # 当前最优解下的背包重量
self.bestgoods = goods[0:10]
print('best:', self.bestgoods) # 输出当前的最优解，最后一次输出即是最终的最优解
return
if self.currentWeight + weight[i] <= self.capacity: # 进入左子树，即选取goods[i]放入背包
goods[i] = 1
self.currentWeight = self.currentWeight + weight[i]
self.currentValue = self.currentValue + value[i]
self.Backtrack(i + 1)
self.currentValue = self.currentValue - value[i] # 进入右子树，即舍弃goods[i]，不放入背包
self.currentWeight = self.currentWeight - weight[i]
goods[i] = 0
self.Backtrack(i + 1)
def main():
global length, weight, value, goods # 全局变量，分别表示货物数目，货物的重量数组，价值数组，货物的选取即0-1值
# currentWeight = 0
# bestValue = 0
# currentValue = 0
capacity = 200
number_goods = 35
value_max = 50
weight_max = 50
weight = [0 for _ in range(number_goods)]
for index1 in range(number_goods):
weight[index1] = random.randint(1, weight_max)
print(weight)
# weight = [2, 2, 6, 5, 4]
value = [0 for _ in range(number_goods)]
for index2 in range(number_goods):
value[index2] = random.randint(1, value_max)
print(value)
# value = [6, 3, 5, 4, 6]
goods = [0 for _ in range(number_goods)]
length = len(weight) - 1
backsack = BackSack(capacity)
start_time = time.time_ns()
backsack.Backtrack(0)
end_time = time.time_ns()
# backsack.Backtrack=Backtrack
print("===============================================")
print("Bag weight: " + str(capacity))
print("Number of goods: " + str(capacity // 2))
print("Best value: " + str(backsack.bestValue))
print("Current weight: " + str(backsack.currentCapacity)) # 输出最优值和背包内物品的总重量
print("Total time: " + str(end_time - start_time) + " ns.")
print(backsack.bestgoods) # 输出最优解
return end_time - start_time
times = [0, 0, 0]
for ix in range(3):
ctime = main()
times[ix] = ctime
print("################################################################")
print(numpy.mean(times))
Project source code: github address