Really good adaptive large neighborhood search algorithm ALNS

Article directory

introduction
Evolution route
Code
- TSP in 34 domestic cities
- Test set XQF131
Related Reading

introduction

The previously introducedDifferential Evolution AlgorithmandAnt Colony AlgorithmApplicable to solving continuous optimization problems and combinatorial optimization problems respectively. They are both intelligent optimization algorithms based on population evolution.

In addition, there is a large category of intelligent optimization algorithms, namely intelligent optimization algorithms based on a single starting point, such as simulated annealing algorithms, tabu search algorithms, and neighborhood search algorithms. This article will focus on one of the neighborhood search algorithms: Adaptive large neighborhood search (ALNS).

There are three main reasons for choosing ALNS: The logic of many parameter design in ALNS is adaptive and more in line with the needs of long-term development; from the perspective of industry practice,Rookie’s The vehicle path planning engine has been shortlisted for the 2021 Franz Edelman Outstanding Achievement Award (known as the "Oscar" in the operations research community). Its core algorithm is ALNS, so ALNS looks more promising; before, I myself I have done project practice on the variable neighborhood search algorithm, and the results are pretty good. As the name suggests, ALNS is more advanced than variable neighborhood search, so I am motivated to learn ALNS.

See text below.

Evolution route

When searching for neighborhood search algorithms on the Internet, you can see many related terms, such as neighborhood, variable neighborhood, large neighborhood, etc. It is really confusing for a person like me who knows little about it. In order to trace the source, I studied in detail the review article by the great Pisinger (actually a chapter of the book): Large Neighborhood Search. There are many concepts. I personally think that it is enough to understand the following four: Neighborhood Search (NS), Variable Neighborhood Search (VDNS), Large Neighborhood Search (LNS) and Adaptive Large Neighborhood Search (ALNS).

Neighborhood Search, NS

The word neighborhood should not be difficult to understand. The essence is that in the current solution $x0 \pmb x_0$ some areas nearby. As for how to define the area, you need to look at the characteristics of the actual problem.

Result $\pmb x$ is continuous, and the neighborhood can be defined as distance $x0 \pmb x_0$ does not exceed the spatial range of 1, when $x0 \pmb x_0$ When is one dimension, for example $x_0=2$ ，This is the area at this time $[1, 3]$ ；当 $x0 \pmb x_0$ When is two-dimensional, for example $\pmb x_0=[2,2]$ ，At this time the area is the heart of the world $[2, 2]$ , radius 1 area.

Result $\pmb x$ is discrete, for example $\pmb x_0=[2,2]$ , the neighborhood can be defined as a component moving 1 unit to the left or right, and the neighborhood includes : $[1, 2] 、 [3, 2] 、 [2, 1]$ sum $[2, 3]$ 。

After you have the neighborhood, you can do neighborhood search, which means to find the best solution in the neighborhood $\pmb x ^{\star} < /span>$ , then update $x0 \pmb x_0$ 。

The core of neighborhood search is to design a reasonable neighborhood. The figure below shows a schematic diagram of different solutions finally obtained in different neighborhoods. The basic conclusion is: the larger the neighborhood, the greater the chance of finding the global optimal solution, but this also requires more computing time.

Let’s take another example of a neighborhood in TSP: 2-opt. Assume that the current solution is
$A - B - C - D - AND - F - G - H - A$
Randomly select two points, assuming they are 4 and 7, corresponding to D and G. At this time, the current point can be disassembled into the following 3 segments
$A-B-C,\quad D-E-F-G,\quad H-A$
The definition of 2-opt is that the front and rear sections remain unchanged, the middle section is flipped, and then spliced back together, that is,
$A - B - C - G - F - AND - D - H - A$

Variable neighborhood search, VDNS

If the neighborhood space grows exponentially as the problem size increases, or the neighborhood space itself is relatively large, it is usually defined as VLSN (very large-scale neighborhood search). For VLSN, it is almost unrealistic to achieve complete neighborhood traversal, so some new solutions are needed.

Variable-Depth Neighborhood Search (VDNS) is one of them. The basic idea is: first $x0 \pmb x_0$ Search within the small neighborhood of . If there is a better solution, update $x0 \pmb x_0$ 为 $x1 \pmb x_1$ ，Next time $x1 \pmb x_1$ Search again within the small neighborhood of ; if there is no better solution, expand $x0 \pmb x_0$ 's neighborhood, continue searching, and after finding a better solution, update $x0 \pmb x_0$ And go back to small neighborhood search.

The figure below is an example of VDNS. Through three neighborhood expansions at 2, a better solution 3 was found.

Compared with NS, VDNS can find better solutions in more neighborhoods, and the calculation time is also more controllable.

Large Neighborhood Search, LNS

Large Neighborhood Search (LNS) can be understood as another solution to solve VLSN, and is mainly used to solve combinatorial optimization problems. Here we directly use the VRP example in the original article to describe the implementation process of LNS.

VRP, vehicle routing problem, is a very classic type of combinatorial optimization problem. It can be simply described as: a center point and multiple customers. The main goal is to use as few vehicles as possible and drive as short a distance as possible, or spend Minimize the cost as much as possible. The basic constraint is to visit all customers and the vehicle finally returns to the center point. If other constraints are added, such as each vehicle has loading restrictions, it is called a CVRP problem. If constraints are continued to be added, it can be extended to VRP-TW etc.

The figure below is an example of a VRP solution. The square in the picture is the center point. There are a total of 21 customers, using 4 vehicles, and visiting 6 (blue and green), 3 (red), 5 (blue) and 7 (purple) customers respectively. From the picture, it can be seen that the paths between many vehicles are intersecting, which is not beautiful. It can be roughly judged that the total driving distance has room for optimization. So how to optimize it specifically?

LNS provides an optimization solution that combines "destroy" + "repair". First, perform a destroy operation on the above-mentioned current solution to destroy the current solution to a certain extent. The figure below shows a destruction scheme, that is, for each vehicle, delete 2, 1, 1 and 2 customers respectively, then skip the destroyed customer points and reconnect them into a closed curve.

Then, perform a repair operation on the destroyed solution, reinsert the deleted customers into the path, ensure that the constraints are satisfied, and obtain a test solution. The scheme in the figure below is a greedy strategy: insert the deleted customers into the position with the shortest total path.

Then, compare the index sizes of the current solution and the test solution. If the index is better, update the current solution to the test solution, otherwise it will not be updated.

After that, perform destroy+repair again until the termination conditions are met and the optimization exits.

The following figure is the pseudo code of LNS. The objective function is to minimize $c (x)$ ；第4行的 $d (x)$ 指的是destroy， $r (d (x))$ Indicative is repair; It is important to note that it is necessary to pay attention to this. $x$ ： $x$ refers to the current solution, $x^t$ refers to based on the current solution $Test solution generated by x$ + destroy + repair, $x^b$ refers to the historical optimal solution up to now; in addition, there is $\text{accept}(x^t,x)$ Function, assertion validity update obvious solution $x$ , a judgment is still needed. The judgment index we just mentioned is the initial version. Subsequent versions are more likely to draw on the design ideas in the simulated annealing algorithm: generate A random number if its value is lower than $e^{-(c(x^t)-c(x)) /T}$ ( $T$ is an external parameter), even if the indicator is not better, the current solution will be updated to the test solution, so that the current solution has the opportunity to jump out of the local optimal solution and find a better solution. Excellent solution.

Adaptive Large Neighborhood Search, ALNS

Judging from the design logic of LNS, the core of the algorithm is to design destroy and repair operators that match specific problems. For repair, there are not many commonly used operators, mainly random and greedy strategies; but for destroy, the number of operators is a bit large: the first thing that needs to be determined is the proportion of destroy, or take the VRP problem in the previous section as an example. , a total of 21 customers. If the destroy ratio is 10%, you need to delete 2 customers; then you need to determine which two to delete, which means there are 420 situations. If the destroy ratio is 20%, there are 143,640 situations. Therefore, how to find the appropriate destroy and repair operators in each iteration is particularly important.

The goal of ALNS is to adaptively find the most appropriate operator based on the characteristics of the specific problem. The design idea can be understood as follows: at the beginning, all operators that can be selected are listed, and the selection probabilities of different operators are exactly the same; thereafter, if a certain operator can bring greater index improvement, the operator will be selected The probability will increase, thereby having a greater probability of being used in subsequent iterations.

The following figure is the pseudo code of ALNS. Compared with LNS, the main changes are: lines 2, 4 and 12. $\rho^- in line 2$ refers to the probability of choosing different destroy operators, $\rho^+$ refers to the probability of selecting different repair operators. At the beginning of the iteration, they are all set to 1, that is, the probability of being selected remains the same; after each round of iteration, they will be updated, that is, line 12. The update logic is slightly complicated, which will be discussed in detail later. $\Omega^-$ refers to the set of destroy operators, $\Omega^+$ refers to the set of repair operators. Which destroy and repair is selected in the current round is determined by roulette, that is, $\rho^-$ 和 $\rho^+$ , the greater the probability of being selected.

with $\rho^-$ 和 $\rho^+$ . In order to hook up their updates and index improvement, an additional parameter needs to be introduced first
$\Psi=\left\{ \begin{aligned} w_1, & the test solution is the historical optimal Solution\\ w_2, & The test solution is not the historical optimal solution, but it is better than the current solution\\ w_3, & The current solution is updated to the test solution\\ w_4, & The test solution is worse and the current solution is not updated\end{ aligned} \right.$

Then it can be updated according to the following formula $\rho^-$ 和 $\rho^+$
$\rho^- = \lambda \rho^- + (1-\lambda )\Psi$
$\rho ^+ = \lambda \rho^+ + (1-\lambda )\Psi$
此处， $\lambda$ can be compared to the volatility coefficient of pheromone in the ant colony algorithm.

It should be noted that in order to ensure a larger $\Psi$ 能裹 $\rho^-$ 和 $\rho^+$ is also larger, generally required $w_1>w_2>w_3> w_4$ 。

Code

TSP in 34 domestic cities

In the previous article introducingant colony algorithm, the calculation example is the TSP problem of 34 major cities in China. Among them, the optimal solution obtained by the ant colony algorithm is 15944.43, and the global optimal solution obtained by using ortools is 15614.84.

This example is also used in this section to initially evaluate the capabilities of ALNS. The following is the code implemented in python. Among them, there are three destroy operators: randomly screening N cities, deleting N cities with the largest distance, and randomly deleting consecutive N cities; there are two repair operators: random insertion and greedy insertion, but considering the random insertion The effect is likely to be poor, so only greedy insertion is actually used in the code. For the setting method, see line 138 of the code. The scalability of the code is relatively good. If you want to adjust the destroy and repair operators, you only need to adjust the destroy function on line 66 and the repair function on line 109.

import copy
import math
import time
import random
import numpy as np
import pandas as pd


# 计算TSP总距离
def dis_cal(path, dist_mat):
    distance = 0
    for i in range(len(path) - 1):
        distance += dist_mat[path[i]][path[i + 1]]
    distance += dist_mat[path[-1]][path[0]]
    return distance


# 随机删除N个城市
def random_destroy(x, destroy_city_cnt):
    new_x = copy.deepcopy(x)
    removed_cities = []

    # 随机选择N个城市，并删除
    removed_index = random.sample(range(0, len(x)), destroy_city_cnt)
    for i in removed_index:
        removed_cities.append(new_x[i])
        x.remove(new_x[i])
    return removed_cities


# 删除距离最大的N个城市
def max_n_destroy(x, destroy_city_cnt):
    new_x = copy.deepcopy(x)
    removed_cities = []

    # 计算顺序距离并排序
    dis = []
    for i in range(len(new_x) - 1):
        dis.append(dist_mat[new_x[i]][new_x[i + 1]])
    dis.append(dist_mat[new_x[-1]][new_x[0]])
    sorted_index = np.argsort(np.array(dis))

    # 删除最大的N个城市
    for i in range(destroy_city_cnt):
        removed_cities.append(new_x[sorted_index[-1 - i]])
        x.remove(new_x[sorted_index[-1 - i]])

    return removed_cities


# 随机删除连续的N个城市
def continue_n_destroy(x, destroy_city_cnt):

    new_x = copy.deepcopy(x)
    removed_cities = []

    # 随机选择N个城市，并删除
    removed_index = random.sample(range(0, len(x)-destroy_city_cnt), 1)[0]
    for i in range(removed_index + destroy_city_cnt, removed_index, -1):
        removed_cities.append(new_x[i])
        x.remove(new_x[i])
    return removed_cities


# destroy操作
def destroy(flag, x, destroy_city_cnt):
    # 三个destroy算子，第一个是随机删除N个城市，第二个是删除距离最大的N个城市，第三个是随机删除连续的N个城市
    removed_cities = []
    if flag == 0:
        # 随机删除N个城市
        removed_cities = random_destroy(x, destroy_city_cnt)
    elif flag == 1:
        # 删除距离最大的N个城市
        removed_cities = max_n_destroy(x, destroy_city_cnt)
    elif flag == 2:
        # 随机删除连续的N个城市
        removed_cities = continue_n_destroy(x, destroy_city_cnt)

    return removed_cities


# 随机插入
def random_insert(x, removed_cities):
    insert_index = random.sample(range(0, len(x)), len(removed_cities))
    for i in range(len(insert_index)):
        x.insert(insert_index[i], removed_cities[i])


# 贪心插入
def greedy_insert(x, removed_cities):
    dis = float('inf')
    insert_index = -1

    for i in range(len(removed_cities)):
        # 寻找插入后的最小总距离
        for j in range(len(x) + 1):
            new_x = copy.deepcopy(x)
            new_x.insert(j, removed_cities[i])
            if dis_cal(new_x, dist_mat) < dis:
                dis = dis_cal(new_x, dist_mat)
                insert_index = j

        # 最小位置处插入
        x.insert(insert_index, removed_cities[i])
        dis = float('inf')


# repair操作
def repair(flag, x, removed_cities):
    # 两个repair算子，第一个是随机插入，第二个贪心插入
    if flag == 0:
        random_insert(x, removed_cities)
    elif flag == 1:
        greedy_insert(x, removed_cities)


# 选择destroy算子
def select_and_destroy(destroy_w, x, destroy_city_cnt):
    # 轮盘赌逻辑选择算子
    prob = destroy_w / np.array(destroy_w).sum()
    seq = [i for i in range(len(destroy_w))]
    destroy_operator = np.random.choice(seq, size=1, p=prob)[0]

    # destroy操作
    removed_cities = destroy(destroy_operator, x, destroy_city_cnt)

    return removed_cities, destroy_operator


# 选择repair算子
def select_and_repair(repair_w, x, removed_cities):
    # # 轮盘赌逻辑选择算子
    prob = repair_w / np.array(repair_w).sum()
    seq = [i for i in range(len(repair_w))]
    repair_operator = np.random.choice(seq, size=1, p=prob)[0]

    # repair操作：此处设定repair_operator=1，即只使用贪心策略
    repair(1, x, removed_cities)

    return repair_operator


# ALNS主程序
def calc_by_alns(dist_mat):
    # 模拟退火温度
    T = 100
    # 降温速度
    a = 0.97

    # destroy的城市数量
    destroy_city_cnt = int(len(dist_mat) * 0.1)
    # destroy算子的初始权重
    destroy_w = [1, 1, 1]
    # repair算子的初始权重
    repair_w = [1, 1]
    # destroy算子的使用次数
    destroy_cnt = [0, 0, 0]
    # repair算子的使用次数
    repair_cnt = [0, 0]
    # destroy算子的初始得分
    destroy_score = [1, 1, 1]
    # repair算子的初始得分
    repair_score = [1, 1]
    # destroy和repair的挥发系数
    lambda_rate = 0.5

    # 当前解，第一代，贪心策略生成
    removed_cities = [i for i in range(dist_mat.shape[0])]
    x = []
    repair(1, x, removed_cities)

    # 历史最优解，第一代和当前解相同，注意是深拷贝，此后有变化不影响x，也不会因x的变化而被影响
    history_best_x = copy.deepcopy(x)

    # 迭代
    cur_iter = 0
    max_iter = 1000
    print(
        'cur_iter: {}, best_f: {}, best_x: {}'.format(cur_iter, dis_cal(history_best_x, dist_mat), history_best_x))

    while cur_iter < max_iter:

        # 生成测试解，即伪代码中的x^t
        test_x = copy.deepcopy(x)

        # destroy算子
        remove, destroy_operator_index = select_and_destroy(destroy_w, test_x, destroy_city_cnt)
        destroy_cnt[destroy_operator_index] += 1

        # repair算子
        repair_operator_index = select_and_repair(repair_w, test_x, remove)
        repair_cnt[repair_operator_index] += 1

        if dis_cal(test_x, dist_mat) <= dis_cal(x, dist_mat):
            # 测试解更优，更新当前解
            x = copy.deepcopy(test_x)
            if dis_cal(test_x, dist_mat) <= dis_cal(history_best_x, dist_mat):
                # 测试解为历史最优解，更新历史最优解，并设置最高的算子得分
                history_best_x = copy.deepcopy(test_x)
                destroy_score[destroy_operator_index] = 1.5
                repair_score[repair_operator_index] = 1.5
            else:
                # 测试解不是历史最优解，但优于当前解，设置第二高的算子得分
                destroy_score[destroy_operator_index] = 1.2
                repair_score[repair_operator_index] = 1.2
        else:
            if np.random.random() < np.exp((dis_cal(x, dist_mat) - dis_cal(test_x, dist_mat))) / T:
                # 当前解优于测试解，但满足模拟退火逻辑，依然更新当前解，设置第三高的算子得分
                x = copy.deepcopy(test_x)
                destroy_score[destroy_operator_index] = 0.8
                repair_score[repair_operator_index] = 0.8
            else:
                # 当前解优于测试解，也不满足模拟退火逻辑，不更新当前解，设置最低的算子得分
                destroy_score[destroy_operator_index] = 0.5
                repair_score[repair_operator_index] = 0.5

        # 更新destroy算子的权重
        destroy_w[destroy_operator_index] = \
            destroy_w[destroy_operator_index] * lambda_rate + \
            (1 - lambda_rate) * destroy_score[destroy_operator_index] / destroy_cnt[destroy_operator_index]
        # 更新repair算子的权重
        repair_w[repair_operator_index] = \
            repair_w[repair_operator_index] * lambda_rate + \
            (1 - lambda_rate) * repair_score[repair_operator_index] / repair_cnt[repair_operator_index]
        # 降低温度
        T = a * T

        # 结束一轮迭代，重置模拟退火初始温度
        cur_iter += 1
        print(
            'cur_iter: {}, best_f: {}, best_x: {}'.format(cur_iter, dis_cal(history_best_x, dist_mat), history_best_x))

    # 打印ALNS得到的最优解
    print(history_best_x)
    print(dis_cal(history_best_x, dist_mat))


if __name__ == '__main__':
    original_cities = [['西宁', 101.74, 36.56],
                       ['兰州', 103.73, 36.03],
                       ['银川', 106.27, 38.47],
                       ['西安', 108.95, 34.27],
                       ['郑州', 113.65, 34.76],
                       ['济南', 117, 36.65],
                       ['石家庄', 114.48, 38.03],
                       ['太原', 112.53, 37.87],
                       ['呼和浩特', 111.65, 40.82],
                       ['北京', 116.407526, 39.90403],
                       ['天津', 117.200983, 39.084158],
                       ['沈阳', 123.38, 41.8],
                       ['长春', 125.35, 43.88],
                       ['哈尔滨', 126.63, 45.75],
                       ['上海', 121.473701, 31.230416],
                       ['杭州', 120.19, 30.26],
                       ['南京', 118.78, 32.04],
                       ['合肥', 117.27, 31.86],
                       ['武汉', 114.31, 30.52],
                       ['长沙', 113, 28.21],
                       ['南昌', 115.89, 28.68],
                       ['福州', 119.3, 26.08],
                       ['台北', 121.3, 25.03],
                       ['香港', 114.173355, 22.320048],
                       ['澳门', 113.54909, 22.198951],
                       ['广州', 113.23, 23.16],
                       ['海口', 110.35, 20.02],
                       ['南宁', 108.33, 22.84],
                       ['贵阳', 106.71, 26.57],
                       ['重庆', 106.551556, 29.563009],
                       ['成都', 104.06, 30.67],
                       ['昆明', 102.73, 25.04],
                       ['拉萨', 91.11, 29.97],
                       ['乌鲁木齐', 87.68, 43.77]]
    original_cities = pd.DataFrame(original_cities, columns=['城市', '经度', '纬度'])
    D = original_cities[['经度', '纬度']].values * math.pi / 180
    city_cnt = len(original_cities)
    dist_mat = np.zeros((city_cnt, city_cnt))
    for i in range(city_cnt):
        for j in range(city_cnt):
            if i == j:
                # 相同城市不允许访问
                dist_mat[i][j] = 1000000
            else:
                # 单位：km
                dist_mat[i][j] = 6378.14 * math.acos(
                    math.cos(D[i][1]) * math.cos(D[j][1]) * math.cos(D[i][0] - D[j][0]) +
                    math.sin(D[i][1]) * math.sin(D[j][1]))

    # ALNS求解TSP
    time0 = time.time()
    calc_by_alns(dist_mat)
    print('使用ALNS求解TSP，耗时: {} s'.format(time.time() - time0))

After running the code, it was found that after less than 4 seconds of calculation time, ALNS can obtain the solution of 15662.59, and the gap between it and the global optimal solution of 15614.84 is only about 0.3%.

It seems that the effect of ALNS is quite good.

Test set XQF131

In order to further evaluate the effect of ALNS, increase the problem size. This section uses one of the common TSP test sets: XQF131. This TSP contains 131 city points, and the global optimal solution is 564.

I first tried to call ortools to solve the problem, but found that no optimization results were returned after 6 hours of calculation, so I gave up and continued to work hard.

Then try ALNS, the following is the code implemented in python. The logic of ALNS remains the same as the previous section, except that the input data is replaced.

import copy
import math
import time
import random
import numpy as np
import pandas as pd


# 计算TSP总距离
def dis_cal(path, dist_mat):
    distance = 0
    for i in range(len(path) - 1):
        distance += dist_mat[path[i]][path[i + 1]]
    distance += dist_mat[path[-1]][path[0]]
    return distance


# 随机删除N个城市
def random_destroy(x, destroy_city_cnt):
    new_x = copy.deepcopy(x)
    removed_cities = []

    # 随机选择N个城市，并删除
    removed_index = random.sample(range(0, len(x)), destroy_city_cnt)
    for i in removed_index:
        removed_cities.append(new_x[i])
        x.remove(new_x[i])
    return removed_cities


# 删除距离最大的N个城市
def max_n_destroy(x, destroy_city_cnt):
    new_x = copy.deepcopy(x)
    removed_cities = []

    # 计算顺序距离并排序
    dis = []
    for i in range(len(new_x) - 1):
        dis.append(dist_mat[new_x[i]][new_x[i + 1]])
    dis.append(dist_mat[new_x[-1]][new_x[0]])
    sorted_index = np.argsort(np.array(dis))

    # 删除最大的N个城市
    for i in range(destroy_city_cnt):
        removed_cities.append(new_x[sorted_index[-1 - i]])
        x.remove(new_x[sorted_index[-1 - i]])

    return removed_cities


# 随机删除连续的N个城市
def continue_n_destroy(x, destroy_city_cnt):

    new_x = copy.deepcopy(x)
    removed_cities = []

    # 随机选择N个城市，并删除
    removed_index = random.sample(range(0, len(x)-destroy_city_cnt), 1)[0]
    for i in range(removed_index + destroy_city_cnt, removed_index, -1):
        removed_cities.append(new_x[i])
        x.remove(new_x[i])
    return removed_cities


# destroy操作
def destroy(flag, x, destroy_city_cnt):
    # 三个destroy算子，第一个是随机删除N个城市，第二个是删除距离最大的N个城市，第三个是随机删除连续的N个城市
    removed_cities = []
    if flag == 0:
        # 随机删除N个城市
        removed_cities = random_destroy(x, destroy_city_cnt)
    elif flag == 1:
        # 删除距离最大的N个城市
        removed_cities = max_n_destroy(x, destroy_city_cnt)
    elif flag == 2:
        # 随机删除连续的N个城市
        removed_cities = continue_n_destroy(x, destroy_city_cnt)

    return removed_cities


# 随机插入
def random_insert(x, removed_cities):
    insert_index = random.sample(range(0, len(x)), len(removed_cities))
    for i in range(len(insert_index)):
        x.insert(insert_index[i], removed_cities[i])


# 贪心插入
def greedy_insert(x, removed_cities):
    dis = float('inf')
    insert_index = -1

    for i in range(len(removed_cities)):
        # 寻找插入后的最小总距离
        for j in range(len(x) + 1):
            new_x = copy.deepcopy(x)
            new_x.insert(j, removed_cities[i])
            if dis_cal(new_x, dist_mat) < dis:
                dis = dis_cal(new_x, dist_mat)
                insert_index = j

        # 最小位置处插入
        x.insert(insert_index, removed_cities[i])
        dis = float('inf')


# repair操作
def repair(flag, x, removed_cities):
    # 两个repair算子，第一个是随机插入，第二个贪心插入
    if flag == 0:
        random_insert(x, removed_cities)
    elif flag == 1:
        greedy_insert(x, removed_cities)


# 选择destroy算子
def select_and_destroy(destroy_w, x, destroy_city_cnt):
    # 轮盘赌逻辑选择算子
    prob = destroy_w / np.array(destroy_w).sum()
    seq = [i for i in range(len(destroy_w))]
    destroy_operator = np.random.choice(seq, size=1, p=prob)[0]

    # destroy操作
    removed_cities = destroy(destroy_operator, x, destroy_city_cnt)

    return removed_cities, destroy_operator


# 选择repair算子
def select_and_repair(repair_w, x, removed_cities):
    # # 轮盘赌逻辑选择算子
    prob = repair_w / np.array(repair_w).sum()
    seq = [i for i in range(len(repair_w))]
    repair_operator = np.random.choice(seq, size=1, p=prob)[0]

    # repair操作：此处设定repair_operator=1，即只使用贪心策略
    repair(1, x, removed_cities)

    return repair_operator


# ALNS主程序
def calc_by_alns(dist_mat):
    # 模拟退火温度
    T = 100
    # 降温速度
    a = 0.97

    # destroy的城市数量
    destroy_city_cnt = int(len(dist_mat) * 0.1)
    # destroy算子的初始权重
    destroy_w = [1, 1, 1]
    # repair算子的初始权重
    repair_w = [1, 1]
    # destroy算子的使用次数
    destroy_cnt = [0, 0, 0]
    # repair算子的使用次数
    repair_cnt = [0, 0]
    # destroy算子的初始得分
    destroy_score = [1, 1, 1]
    # repair算子的初始得分
    repair_score = [1, 1]
    # destroy和repair的挥发系数
    lambda_rate = 0.5

    # 当前解，第一代，贪心策略生成
    removed_cities = [i for i in range(dist_mat.shape[0])]
    x = []
    repair(1, x, removed_cities)

    # 历史最优解，第一代和当前解相同，注意是深拷贝，此后有变化不影响x，也不会因x的变化而被影响
    history_best_x = copy.deepcopy(x)

    # 迭代
    cur_iter = 0
    max_iter = 1000
    print(
        'cur_iter: {}, best_f: {}, best_x: {}'.format(cur_iter, dis_cal(history_best_x, dist_mat), history_best_x))

    while cur_iter < max_iter:

        # 生成测试解，即伪代码中的x^t
        test_x = copy.deepcopy(x)

        # destroy算子
        remove, destroy_operator_index = select_and_destroy(destroy_w, test_x, destroy_city_cnt)
        destroy_cnt[destroy_operator_index] += 1

        # repair算子
        repair_operator_index = select_and_repair(repair_w, test_x, remove)
        repair_cnt[repair_operator_index] += 1

        if dis_cal(test_x, dist_mat) <= dis_cal(x, dist_mat):
            # 测试解更优，更新当前解
            x = copy.deepcopy(test_x)
            if dis_cal(test_x, dist_mat) <= dis_cal(history_best_x, dist_mat):
                # 测试解为历史最优解，更新历史最优解，并设置最高的算子得分
                history_best_x = copy.deepcopy(test_x)
                destroy_score[destroy_operator_index] = 1.5
                repair_score[repair_operator_index] = 1.5
            else:
                # 测试解不是历史最优解，但优于当前解，设置第二高的算子得分
                destroy_score[destroy_operator_index] = 1.2
                repair_score[repair_operator_index] = 1.2
        else:
            if np.random.random() < np.exp((dis_cal(x, dist_mat) - dis_cal(test_x, dist_mat))) / T:
                # 当前解优于测试解，但满足模拟退火逻辑，依然更新当前解，设置第三高的算子得分
                x = copy.deepcopy(test_x)
                destroy_score[destroy_operator_index] = 0.8
                repair_score[repair_operator_index] = 0.8
            else:
                # 当前解优于测试解，也不满足模拟退火逻辑，不更新当前解，设置最低的算子得分
                destroy_score[destroy_operator_index] = 0.5
                repair_score[repair_operator_index] = 0.5

        # 更新destroy算子的权重
        destroy_w[destroy_operator_index] = \
            destroy_w[destroy_operator_index] * lambda_rate + \
            (1 - lambda_rate) * destroy_score[destroy_operator_index] / destroy_cnt[destroy_operator_index]
        # 更新repair算子的权重
        repair_w[repair_operator_index] = \
            repair_w[repair_operator_index] * lambda_rate + \
            (1 - lambda_rate) * repair_score[repair_operator_index] / repair_cnt[repair_operator_index]
        # 降低温度
        T = a * T

        # 结束一轮迭代，重置模拟退火初始温度
        cur_iter += 1
        print(
            'cur_iter: {}, best_f: {}, best_x: {}'.format(cur_iter, dis_cal(history_best_x, dist_mat), history_best_x))

    # 打印ALNS得到的最优解
    print(history_best_x)
    print(dis_cal(history_best_x, dist_mat))


if __name__ == '__main__':
    original_cities = [[0, 13],
                       [0, 26],
                       [0, 27],
                       [0, 39],
                       [2, 0],
                       [5, 13],
                       [5, 19],
                       [5, 25],
                       [5, 31],
                       [5, 37],
                       [5, 43],
                       [5, 8],
                       [8, 0],
                       [9, 10],
                       [10, 10],
                       [11, 10],
                       [12, 10],
                       [12, 5],
                       [15, 13],
                       [15, 19],
                       [15, 25],
                       [15, 31],
                       [15, 37],
                       [15, 43],
                       [15, 8],
                       [18, 11],
                       [18, 13],
                       [18, 15],
                       [18, 17],
                       [18, 19],
                       [18, 21],
                       [18, 23],
                       [18, 25],
                       [18, 27],
                       [18, 29],
                       [18, 31],
                       [18, 33],
                       [18, 35],
                       [18, 37],
                       [18, 39],
                       [18, 41],
                       [18, 42],
                       [18, 44],
                       [18, 45],
                       [25, 11],
                       [25, 15],
                       [25, 22],
                       [25, 23],
                       [25, 24],
                       [25, 26],
                       [25, 28],
                       [25, 29],
                       [25, 9],
                       [28, 16],
                       [28, 20],
                       [28, 28],
                       [28, 30],
                       [28, 34],
                       [28, 40],
                       [28, 43],
                       [28, 47],
                       [32, 26],
                       [32, 31],
                       [33, 15],
                       [33, 26],
                       [33, 29],
                       [33, 31],
                       [34, 15],
                       [34, 26],
                       [34, 29],
                       [34, 31],
                       [34, 38],
                       [34, 41],
                       [34, 5],
                       [35, 17],
                       [35, 31],
                       [38, 16],
                       [38, 20],
                       [38, 30],
                       [38, 34],
                       [40, 22],
                       [41, 23],
                       [41, 32],
                       [41, 34],
                       [41, 35],
                       [41, 36],
                       [48, 22],
                       [48, 27],
                       [48, 6],
                       [51, 45],
                       [51, 47],
                       [56, 25],
                       [57, 12],
                       [57, 25],
                       [57, 44],
                       [61, 45],
                       [61, 47],
                       [63, 6],
                       [64, 22],
                       [71, 11],
                       [71, 13],
                       [71, 16],
                       [71, 45],
                       [71, 47],
                       [74, 12],
                       [74, 16],
                       [74, 20],
                       [74, 24],
                       [74, 29],
                       [74, 35],
                       [74, 39],
                       [74, 6],
                       [77, 21],
                       [78, 10],
                       [78, 32],
                       [78, 35],
                       [78, 39],
                       [79, 10],
                       [79, 33],
                       [79, 37],
                       [80, 10],
                       [80, 41],
                       [80, 5],
                       [81, 17],
                       [84, 20],
                       [84, 24],
                       [84, 29],
                       [84, 34],
                       [84, 38],
                       [84, 6],
                       [107, 27]]
    original_cities = np.array(original_cities)
    dist_mat = np.zeros((len(original_cities), len(original_cities)))
    for i in range(len(original_cities)):
        for j in range(len(original_cities)):
            if i == j:
                dist_mat[i][j] = 100000
            else:
                dist_mat[i][j] = math.sqrt((original_cities[i][0] - original_cities[j][0]) ** 2 +
                                           (original_cities[i][1] - original_cities[j][1]) ** 2)

    # ALNS求解TSP
    time0 = time.time()
    calc_by_alns(dist_mat)
    print('使用ALNS求解TSP，耗时: {} s'.format(time.time() - time0))

After running the code, the solution of 589 can be obtained in 3 minutes, and the gap between it and the optimal solution of 564 is 4%. Comparing the calculation time and the optimal solution, the overall results are satisfactory.

In addition, a big guy on github also wrote ALNS, which is said to be able to get 574 solutions in 1 minute, and the final The gap between optimal solutions is only 2%. The cost of understanding his code is a bit high, so I probably studied it a bit. Some of the ideas that I think are good have been used in the above code. I suggest that children who are new to ALNS can first understand my article and then learn his code.