Common algorithm ideas - simulated annealing algorithm

a brief introdction

The simulated annealing algorithm (Simulated Annealing) is a probability-based global optimization algorithm, which is inspired by the behavior of atoms in the solid annealing process reaching a low-energy state during the cooling process. This algorithm gradually reduces the system temperature by simulating the atomic annealing process, and accepts inferior solutions with a certain probability to avoid falling into the local optimal solution, thereby finding the global optimal solution.

General steps

1. Initialize temperature T and initial solution x.
2. Repeat the following process until the stopping condition is met:
   • Generate the neighborhood solution x’ of the current solution.
   • Calculate the change in the objective function ΔE = f(x’) - f(x).
   • If ΔE < 0, accept the neighborhood solution x' and update x to x'.
   • If ΔE > 0, accept the neighborhood solution x’ with probability exp(-ΔE / T), and update x to x’ if accepted.
   • Lower the temperature T.
3. Return the optimal solution x.

Application examples

The following is a simple example of simulated annealing algorithm implemented in C++, which is used to solve the minimum value of function f(x) = x^2.

#include <iostream>
#include <cmath>
#include <random>

// 目标函数
double f(double x) {
    
    
    return x * x;
}

// 模拟退火算法
double simulatedAnnealing() {
    
    
    // 初始温度
    double temperature = 1000.0;
    // 终止温度
    double minTemperature = 0.01;
    // 温度衰减率
    double coolingRate = 0.95;
    // 当前解
    double currentSolution = 10.0;

    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_real_distribution<double> distribution(-1.0, 1.0);

    while (temperature > minTemperature) {
    
    
        // 生成邻域解
        double nextSolution = currentSolution + distribution(gen);
        // 计算目标函数变化
        double deltaE = f(nextSolution) - f(currentSolution);
        // 接受更差解或根据概率接受
        if (deltaE < 0 || std::exp(-deltaE / temperature) > distribution(gen)) {
    
    
            currentSolution = nextSolution;
        }
        // 降低温度
        temperature *= coolingRate;
    }

    return currentSolution;
}

int main() {
    
    
    double solution = simulatedAnnealing();
    std::cout << "Optimal solution: " << solution << std::endl;
    std::cout << "Minimum value: " << f(solution) << std::endl;
    return 0;
}

In this example, we use random number generators (std::random_device and std::mt19937) to generate neighborhood solutions and acceptance probabilities. The simulatedAnnealing function continuously reduces the temperature through iteration and updates the current solution according to the objective function change and acceptance probability until the stopping condition is met (the temperature reaches the termination temperature). Finally, we output the found optimal solution and its corresponding objective function value.

Article summary

The basic idea of ​​the simulated annealing algorithm is to randomly search in the solution space and accept worse solutions with a certain probability. As the temperature gradually decreases, the probability gradually decreases, and finally converges to the global optimal solution. In actual applications, appropriate adjustments and improvements may be required based on specific problems.