A Variable Neighborhood Search Algorithm to Solve the 0-1 Knapsack Problem
This article is about [ Data Magician]: Dry goods | Variable Neighborhood Search Algorithm to Solve the 0-1 Knapsack Problem (Knapsack Problem) code example article.
]: Dry goods | Code example of variable neighborhood search algorithm to solve 0-1 knapsack problem (Knapsack Problem)
1.0-1 knapsack problem
1.1 What is the 0-1 knapsack problem?
0-1 Knapsack Problem: Given n items and a knapsack of capacity C, item i has weight w_i and value v_i .
Question: How should the items in the backpack be selected so that the total value of the items in the backpack is maximized?
1.2 Solution Design
Suppose we have n items in front of us, then we can set the solution as a one-dimensional array selection[n]. The array weights[n] represents the weight, and the array values[n] represents the value.
- selection[i] = 1 means to load the i-th item.
- selection[i] = 0 means do not load the i-th item.
- 总价值total_value = selection[i] * values[i]。 (i=1,2,3,4……n)
- 总重量total_weight = selection[i] * weights[i]。(i=1,2,3,4……n)
2. Variable Neighborhood Search
2.1 How to change the domain?
The main idea of the variable neighborhood search algorithm is: use multiple different neighborhoods to search systematically. The smallest neighborhood is searched first, switching to a slightly larger neighborhood when the solution cannot be improved. If you can continue to improve the solution, return to the smallest neighborhood, otherwise continue to switch to a larger neighborhood.
So obviously, multiple different field actions are required!
Therefore, the following three neighborhood actions are designed:
2.2 Domain actions
Neighborhood Action 1
Invert the i-th bit of the solution selection[n] (i=1,2,3,4...n).
Neighborhood Action 2
For the solution selection[n], in the case of inverting the i (i=1,2,3,4...n) bit, the j (j=i+1,2,3,4... The n) bit is also inverted .
Neighborhood Action 3
Swap the i-th and i-3th digits. for example:
- selection[0] and selection[n-3] are swapped.
- selection[1] and selection[n-2] are swapped.
- selection[2] and selection[n-1] are exchanged.
3. Key codes and explanations
Calculate a plan
//对该方案selection[N]进行评价，计算total_values和total_weights
void Evaluate_Solution(KP_Solution & x) {
//初始化total_values和total_weights
x.total_values = 0;
x.total_weights = 0;
//计算total_values和total_weights
for (int i = 0; i < n; i++) {
x.total_values += x.selection[i] * values[i];
x.total_weights += x.selection[i] * weights[i];
}
//超过背包最大容纳重量，说明该方案不满足最大容纳重量约束，价值设置为负数
if (x.total_weights > maxWeight)
x.total_values = maxWeight - x.total_weights; //这样赋值的原因：对于同样不满足最大容纳重量约束的解。超出的重量越少，解越优，权值也就越大。
}
Generating Neighborhood Solutions Using Neighborhood Actions
//对x采取邻域动作flip生成邻居解
void neighborhood(KP_Solution &x, int flip) {
//邻域动作1
if (flip == 1) {
nbrhood.clear();
for (int i = 0; i < n; i++) {
nbrhood.push_back(x);
nbrhood[i].selection[i]^=1;
}
}
//邻域动作2
if (flip == 2) {
nbrhood.clear();
int a = 0;
for (int i = 0; i < n; i++)
for (int j = i+1; j < n; j++) {
nbrhood.push_back(x);
nbrhood[a].selection[i]^=1;
nbrhood[a].selection[j]^=1;
a++;
}
}
//邻域动作3
if (flip == 3) {
nbrhood.clear();
for (int i = 0; i < n; i++) {
nbrhood.push_back(x);
//其实下面这段可以直写成
//swap(nbrhood[i].selection[i], nbrhood[i].selection[(n + i - 3)%n]);
if ( i < 3)
swap(nbrhood[i].selection[i], nbrhood[i].selection[n + i - 3]);
else
swap(nbrhood[i].selection[i], nbrhood[i].selection[i - 3]);
}
}
}
Randomly generate a solution
//随机生成解决方案 （不一定满足最大背包容量的约束条件）
void Random_Solution(KP_Solution &x) {
x.total_values = 0;
x.total_weights = 0;
for (int i = 0; i < n; i++) {
double rate = (rand() % 100) / 100.0;
//不选该物品的概率为0.8，选该物品的概率为0.2
if ( rate < 0.8 )
x.selection[i] = 0;
else
x.selection[i] = 1;
}
}
Change field search
The main idea of the variable neighborhood search algorithm is: use multiple different neighborhoods to search systematically. The smallest neighborhood is searched first, switching to a slightly larger neighborhood when the solution cannot be improved. If you can continue to improve the solution, return to the smallest neighborhood, otherwise continue to switch to a larger neighborhood.
//通过变邻域搜索产生邻域解，并在邻域解中的选最优解作为新一代的解
void Variable_Neighborhood_Descent(KP_Solution &x) {
int flip = 1;//初始化flip为1，表示一开始采取邻域动作1（也就是最小的邻域搜索）
KP_Solution x_curr;
while ( flip < MaxFlip + 1) {
neighborhood(x, flip);//对x采取flip这个邻域动作进行搜索，得到邻域解
//找到 flip这个邻域动作得到邻域解中的最优解x_curr
x_curr = nbrhood[0];
Evaluate_Solution(x_curr);
for(unsigned int i = 1; i < nbrhood.size(); i++) {
solutionsChecked ++;
Evaluate_Solution(nbrhood[i]);
if (nbrhood[i].total_values > x_curr.total_values)
x_curr = nbrhood[i];
}
//邻域复位
if (x_curr.total_values > x.total_values) {
// 当前flip这个邻域动作能改进解，则退回到最小的邻域
x = x_curr;
flip = 1;
} else// 当前flip这个邻域动作不能改进解，则切换到更大的邻域
flip ++;
}
}
random modification
//随机取反 selection中的一些数
void Shaking_Procdure(KP_Solution &x) {
int num = (rand() %n) /10 + 3; // num为取反的个数：3 ~ 3+(n/10)个
for (int i = 0; i < num; i++) {
int pos = rand() % n;//得到随机取反的位置pos
x.selection[pos]^=1;
}
Evaluate_Solution(x);//计算x的total_values和total_weights
}
Iterative multiple searches to find the optimal solution
//迭代iteration次变邻域搜索，取最优解
void Variable_Neighborhood_Search(KP_Solution &x, int iteration) {
KP_Solution best = x;//best为当前最优解，一开始best为初始解
Variable_Neighborhood_Descent(best);
for (int i = 0; i < iteration; i++) {
Shaking_Procdure(x);//对第i代的最优解X进行随机修改（我个人觉得这个优化能否更快找到最优解还有待考察）
Variable_Neighborhood_Descent(x);//对X进行变邻域搜索，生成新一代的邻域解
if (best.total_values < x.total_values)//如果新一代邻域解当中的最优解x比当前最优解best更优，则更新best。
best = x;
}
x = best;
}
4. Complete code
#include <bits/stdc++.h>
using namespace std;
#define N 210//n的上限
int n;// 物品的数量 每一个物品有0和1两种选择 0代表选择当前物品 1代表不选择当前物品
int Max_Iteration;//算法最大迭代次数
const int MaxFlip = 3;//邻域数量
int maxWeight;//背包最大容量
int solutionsChecked = 0;//记录已经检查的背包数量
int values[N],weights[N];//物品对应价值&&重量
struct KP_Solution {
int selection[N]; //当前方案的物品选择情况 selection[i] == 0 or 1 <==> 不选择 or 选择 第i个物品
int total_values; //当前方案下物品总价值
int total_weights; //当前方案下物品总重量
} kpx; //kpx为当前解
//对该方案selection[N]进行评价，计算total_values和total_weights
void Evaluate_Solution(KP_Solution & x) {
//初始化total_values和total_weights
x.total_values = 0;
x.total_weights = 0;
//计算total_values和total_weights
for (int i = 0; i < n; i++) {
x.total_values += x.selection[i] * values[i];
x.total_weights += x.selection[i] * weights[i];
}
//超过背包最大容纳重量，说明该方案不满足最大容纳重量约束，价值设置为负数
if (x.total_weights > maxWeight)
x.total_values = maxWeight - x.total_weights; //这样赋值的原因：对于同样不满足最大容纳重量约束的解。超出的重量越少，解越优，权值也就越大。
}
vector<KP_Solution> nbrhood;//用vector记录邻居解集合
//对x采取邻域动作flip生成邻居解
void neighborhood(KP_Solution &x, int flip) {
//邻域动作1
if (flip == 1) {
nbrhood.clear();
for (int i = 0; i < n; i++) {
nbrhood.push_back(x);
nbrhood[i].selection[i]^=1;
}
}
//邻域动作2
if (flip == 2) {
nbrhood.clear();
int a = 0;
for (int i = 0; i < n; i++)
for (int j = i+1; j < n; j++) {
nbrhood.push_back(x);
nbrhood[a].selection[i]^=1;
nbrhood[a].selection[j]^=1;
a++;
}
}
//邻域动作3
if (flip == 3) {
nbrhood.clear();
for (int i = 0; i < n; i++) {
nbrhood.push_back(x);
//其实下面这段可以直写成
//swap(nbrhood[i].selection[i], nbrhood[i].selection[(n + i - 3)%n]);
if ( i < 3)
swap(nbrhood[i].selection[i], nbrhood[i].selection[n + i - 3]);
else
swap(nbrhood[i].selection[i], nbrhood[i].selection[i - 3]);
}
}
}
//随机生成解决方案 （不一定满足最大背包容量的约束条件）
void Random_Solution(KP_Solution &x) {
x.total_values = 0;
x.total_weights = 0;
for (int i = 0; i < n; i++) {
double rate = (rand() % 100) / 100.0;
//不选该物品的概率为0.8，选该物品的概率为0.2
if ( rate < 0.8 )
x.selection[i] = 0;
else
x.selection[i] = 1;
}
}
/*变邻域搜索算法的主要思想是：采用多个不同的邻域进行系统搜索。
首先采用最小的邻域搜索，当无法改进解时，则切换到稍大一点的邻域。
如果能继续改进解，则退回到最小的邻域，否则继续切换到更大的邻域。*/
//通过变邻域搜索产生邻域解，并在邻域解中的选最优解作为新一代的解
void Variable_Neighborhood_Descent(KP_Solution &x) {
int flip = 1;//初始化flip为1，表示一开始采取邻域动作1（也就是最小的邻域搜索）
KP_Solution x_curr;
while ( flip < MaxFlip + 1) {
neighborhood(x, flip);//对x采取flip这个邻域动作进行搜索，得到邻域解
//找到 flip这个邻域动作得到邻域解中的最优解x_curr
x_curr = nbrhood[0];
Evaluate_Solution(x_curr);
for(unsigned int i = 1; i < nbrhood.size(); i++) {
solutionsChecked ++;
Evaluate_Solution(nbrhood[i]);
if (nbrhood[i].total_values > x_curr.total_values)
x_curr = nbrhood[i];
}
//邻域复位
if (x_curr.total_values > x.total_values) {
// 当前flip这个邻域动作能改进解，则退回到最小的邻域
x = x_curr;
flip = 1;
} else// 当前flip这个邻域动作不能改进解，则切换到更大的邻域
flip ++;
}
}
//随机取反 selection中的一些数
void Shaking_Procdure(KP_Solution &x) {
int num = (rand() %n) /10 + 3; // num为取反的个数：3 ~ 3+(n/10)个
for (int i = 0; i < num; i++) {
int pos = rand() % n;//得到随机取反的位置pos
x.selection[pos]^=1;
}
Evaluate_Solution(x);//计算x的total_values和total_weights
}
//迭代iteration次变邻域搜索，取最优解
void Variable_Neighborhood_Search(KP_Solution &x, int iteration) {
KP_Solution best = x;//best为当前最优解，一开始best为初始解
Variable_Neighborhood_Descent(best);
for (int i = 0; i < iteration; i++) {
Shaking_Procdure(x);//对第i代的最优解X进行随机修改（我个人觉得这个优化能否更快找到最优解还有待考察）
Variable_Neighborhood_Descent(x);//对X进行变邻域搜索，生成新一代的邻域解
if (best.total_values < x.total_values)//如果新一代邻域解当中的最优解x比当前最优解best更优，则更新best。
best = x;
}
x = best;
}
//数据输入
void read_in() {
cout << "请输入背包最大容量为:" ;
cin>> maxWeight;
cout << "请输入石头个数为:" ;
cin>> n;
Max_Iteration=n*10;
for(int i=0; i<n; i++) {
cout << "请分别输入第"<<i+1<<"个石头的重量和价值为:" ;
cin>> weights[i] >>values[i];
}
}
int main() {
read_in();//读入数据，并可视化数据
srand(time(0));//初始化随机数种子
Random_Solution(kpx);//先随机生成一个解 kpx
Variable_Neighborhood_Search(kpx, Max_Iteration);//执行变领域搜索更新kpx，迭代次数为Max_Iteration
cout << endl << "最终结果: 已检查的总方案数 = " << solutionsChecked << endl;
cout << "找到最大价值为: " << kpx.total_values << endl;
cout << "背包当前重量为: " << kpx.total_weights << endl;
for (int i = 0; i < n; i++) {
cout << kpx.selection[i] << " ";
if ((i+1) % 25 == 0)
cout << endl;
}
return 0;
}
5. Running results
The running time is relatively large because I was thinking about how to construct the input data when I entered it, and the actual running speed is very fast!
6. Personal gains and experiences
Learned a new intelligent optimization algorithm: variable neighborhood search
For the dynamic programming method of the 01 backpack: the complexity is O(NV). If the maximum capacity of the backpack is very, very large, the running time of the dynamic programming method may not be acceptable.
Therefore, for problems that do not require an optimal solution, you can use this variable domain search to get a pretty good solution.