Simulated annealing (SA) is a stochastic method developed in the early 1980s to solve large-scale combinatorial optimization problems. Based on the similarity between the solution of the optimization problem and the annealing process of the physical system, it uses the Metropolis algorithm and properly controls the temperature drop process to achieve simulated annealing, so as to achieve the purpose of solving the global optimization problem.
It has the advantages of wide application range, high reliability in obtaining the global optimal solution, simple algorithm, and easy implementation. The search strategy of simulated annealing algorithm is different from the traditional random search method. It not only introduces appropriate random factors, but also introduces the natural mechanism of the annealing process of physical systems. The introduction of this natural mechanism makes the simulated annealing algorithm not only accept the trial points that make the value of the objective function "good" in the iterative process, but also accept the trial points that make the value of the objective function "bad" with a certain probability. The degree gradually decreases as the temperature decreases. This search strategy of simulated annealing algorithm is beneficial to avoid the drawbacks of the search process being trapped in the local optimal solution and can not dial itself, and it is beneficial to improve the reliability of finding the global optimal solution.
This article will explain the principle of simulated annealing algorithm and give its code implementation.
00 Article list
1 Principle of simulated annealing algorithm
2 Problem introduction
3 MATLAB program implementation
4 Outlook
01 Principle of Simulated Annealing Algorithm
Simulated annealing algorithm was first applied in the field of combinatorial optimization by Kirkpatrick et al. It is a stochastic optimization algorithm based on Monte Carlo iterative solution strategy. To statistics, the starting point is based on the similarity between the annealing process of solid substances in physics and general combinatorial optimization problems.
The basic idea of the SA algorithm is to start from the selected initial solution, in a series of Markov chains generated when the control parameter t is decreasing, using a new solution generation device and acceptance criteria, repeat the process of "generating a new solution - calculating the objective function "The difference is one to determine whether to accept the new solution - one to accept or discard the new solution" and continuously iterate on the current solution to achieve the optimal execution process of the objective function. Since solid annealing must be slowly cooled, the solid can reach thermal equilibrium at each temperature and eventually reach an equilibrium state. Therefore, the value of the control parameter t must decay slowly to ensure that the simulated annealing algorithm eventually tends to the overall optimal solution to the optimization problem.
The simulated annealing algorithm combines the probabilistic jump feature to randomly search for the global optimal solution of the objective function in the solution space, that is, the local optimal solution can jump out of the probability and eventually tend to the global optimal solution. The simulated annealing algorithm is a general optimization algorithm. In theory, the algorithm has probabilistic global optimization performance.
The solution steps are as follows:
(1) choose an initial state x0 from the feasible solution space, calculate its objective function value f(x0), and select the initial control temperature T0 and the length of the Markov chain; (2) in the feasible solution
space Generate a random disturbance in the space, use the state generation function to generate a new state x1, and calculate its objective function value f(x1); (3) Judge
whether to accept according to the state acceptance function: if f(x1)<f(x0), then Accept the new state x1 as the current state, otherwise judge whether to accept x1 according to the Metropolis criterion, if accepted, make the current state equal to x1, if not, make the current state equal to x0; (4) According to a certain convergence criterion, judge the sampling
process If yes, go to 5, otherwise go to 2
(5) Reduce the control temperature T according to a certain temperature cooling scheme;
(6) According to a certain convergence criterion, judge whether the annealing is terminated, if yes, go to 7, otherwise go to 2;
( 7) The current solution is output as the optimal solution;
02 Problem introduction
: A multi-peak nonlinear function is introduced to verify the performance of the SA algorithm. The function is as follows:
Its image is as follows:
Its extreme position is to obtain a maximum value near (0,0), and the maximum value is 1.0054
03 MATLAB program implementation
According to the solution steps of the algorithm, part of the main program is as follows:
The complete program can be found in the comment area or privately message me your email address. I will send it to you when I see it.
After executing the program, the following results are obtained
Since the iterative mechanism of simulated annealing is different from the algorithms introduced above, the number of iterations of the simulated annealing algorithm is usually very large, but because its comparison method is pairwise comparison, the iteration speed is very fast.
04 Outlook
4.1 Limitations of the traditional simulated annealing algorithm
Although the simulated annealing algorithm has a limited acceptance of inferior solutions, can jump out of local optimal solutions, has a simple principle, is flexible to use, and is suitable for solving global optimal or approximate global optimal solutions of optimization problems, etc. Advantages, but it obviously has the following disadvantages:
(1) The solution time is too long. When there are many variables and the objective function is complex, in order to obtain a good approximate solution, the control parameter T needs to start from a larger value, and execute the Metropolis algorithm multiple times at each temperature value T, so the iterative operation speed is slow.
(2) The initial value and reduction step size of temperature T are difficult to determine. If the initial value of T is large and the step size is too small, although a better solution can be obtained in the end, the convergence speed of the algorithm is too slow; if the initial value of T is small and the step size is too large, it is likely that The global optimal solution cannot be obtained.
(3) During the search process, the optimal solution currently encountered is lost due to the execution of the probability acceptance link.
4.2 Improvement of Simulated Annealing Algorithm [1]
Simulated annealing algorithm theoretically uses a Markov chain to describe the change process of simulated annealing algorithm, so it has global optimality. The simulated annealing algorithm in practical applications is a heuristic algorithm. It has many parameters that need to be adjusted, such as the initial temperature, the temperature drop scheme, the iteration length of the fixed temperature type and the termination rule, etc., so it needs to be adjusted artificially.
On the basis of ensuring a certain optimization quality, improving the search efficiency (time performance) of the simulated annealing algorithm is the main content of improving the SA algorithm. Possible solutions include:
(1) Increase the heating or reheating process. At an appropriate time in the algorithm process, the temperature is appropriately increased to activate the acceptance probability of each state to adjust the current state in the search process and avoid the algorithm from stagnating at the local minimum solution.
(2) Increase memory function. In order to avoid losing the optimal solution currently encountered due to the implementation of the probability acceptance link in the search process, the state of "Best So Far" can be memorized by adding a storage link.
(3) Add supplementary search process. That is, after the annealing process is completed, the simulated annealing process or local chemotaxis search is performed again with the searched optimal solution as the initial state.
(4) For each current state, multiple search strategies are used to accept the optimal state in the area with probability, instead of the single comparison method of standard SA.
(5) Algorithms combined with other search mechanisms, such as genetic algorithms, chaotic search, etc.
References
[1] Zhu Haodong, Zhong Yong. An improved simulated annealing algorithm [J]. Computer Technology and Development, 2009, 19(06): 32-35.
Another note: If any partners have optimization problems to be solved (in any field), you can send them to me, and I will selectively update articles that use optimization algorithms to solve these problems.
If this article is helpful or inspiring to you, you can click the like (ง •̀_•́)ง in the lower right corner (you don’t have to click). If you have any customization needs, you can send a private message to the author.