[Operational Research Optimization] Meta-heuristic Algorithm Detailed Explanation: Simulated Annealing Algorithm (Simulated Annealing, SA) + Case Explanation & Code Practice

---

1. Introduction

Simulated Annealing (SA) is the simplest and most famous meta-heuristic algorithm, which is often used to solve complex black-box global optimization problems, and it is widely used in real life.

The main advantage of SA is simplicity. SA is based on an analogy to physical annealing of materials, and SA avoids the disadvantages of Monte-Carlo methods (possible trapping in local minima) due to the application of an efficient Metropolis acceptance criterion.

When the evaluation of the objective function comes from a complex simulation process that manipulates a large-dimensional state space involving a large amount of memory, population-based algorithms are not applicable, and simulated annealing is an effective algorithm for solving these problems.

---

2. Basic knowledge

In the early 1980s, three IBM researchers, Kirkpatrick et al., introduced the concept of annealing in combinatorial optimization.

These concepts are based on strong analogies to physical annealing of materials. This process involves bringing the solid into a low-energy state after raising its temperature. It can be summarized in the following two steps ( see Figure 1.1 ):

• bring solids to very high temperatures until the structure "melts"
• Solids are cooled according to very specific cooling schemes to achieve a solid state with minimal energy

In the liquid state, the particles are randomly distributed. The results show that the minimum energy state can be reached as long as the initial temperature is high enough and the cooling time is long enough. If this were not the case, the solid would be in a metastable state with a non-minimum energy; this is called hardening, the sudden cooling of the solid.

Before describing the simulated annealing algorithm for optimization, we need to understand the principle of the local search optimization algorithm, of which simulated annealing is an extension.

2.1 Local search (or Monte Carlo) algorithm

These algorithms improve the objective function by exploring the neighborhood of the current point in the solution space.

In the next definition we consider $(S,f)$ Instantiation of combinatorial optimization problems ($S$ : set of feasible solutions,$f$ : the objective function to be minimized).

Definition 1 : Let $N$ is an application that for each solution$i\in S$ defines an AND solutioniiThe subsetS i ⊂ S S_i \sub S$of\; solutions\; that\; are\; "close"\; to\; i$$S_i\subset S.$ _ child collection$S_i$solution $i$ 's neighborhood.

In the following definitions, we consider $N$ is with$(S,f)$ Neighborhood structure of associations.

Definition 2 : The generative mechanism is used in a given solution iiAny neighborhoodS i S_i$of\; i$$S_i$choose the solution jjmeans of$j$

The local search algorithm is an iterative algorithm that starts the search from a randomly drawn feasible point in the state space.

The generative mechanism is then continuously applied by exploring the neighborhood of the current solution in order to find a better solution.

If a better solution is found, it becomes the current solution. When an improved solution cannot be found, the algorithm ends and the current solution is considered an approximate solution to the optimization problem.

For minimization problems, the algorithm can be summarized by the following pseudocode:

Translated into Chinese is:

step	Concrete operation
1	generate initial solution $i$
2	From the current solution iiRandomly select a solution jjin the neighborhood of$i$$j$ (generate a new solution)
3	If the new solution jjThe objective function value of$j$$The\; objective\; function\; value\; of\; i$ , then let the new solution$j$ as the current solution
4	If the current solution iiThere is no solutionjj$in\; the\; neighborhood\; of\; i$The objective function value of$j$$The\; objective\; function\; value\; of\; i$ , then the program ends
5	return to step 2

Definition 3 : If for all $j\in S_i^{\ast }$All satisfy $f(i^{\ast })\le f(j)$，则解 $i^{\ast }\in S$ is called relative toN \mathscr{N}Local optimal solution of$N$

Definition 4 : If for about N \mathscr{N}Each local optimal solution of$N$$i^{\ast }\in S$，$i^{\ast }$ is also$(S,The\; global\; optimal\; value\; of\; f)$ is called the neighborhood structure$N$ is exact.

Therefore, by definition, local search algorithms are guaranteed to converge to local optima only when there is an exact neighborhood structure.

This notion of an exact neighborhood is theoretical because in practice it usually leads to full enumeration by means of the search space.

Thus, if the current solution "falls" in a subdomain where the objective function is convex, the algorithm remains stuck in that subdomain unless the neighborhood structure associated with the new solution generation mechanism can reach points outside the subdomain.

To avoid getting stuck in local minima, it is necessary to define a procedure that may accept a temporary reduction in the quality of the current solution; this is the main principle of simulated annealing.

Before describing this algorithm, it is necessary to introduce the Metropolis algorithm, which is a basic component of SA.

2.2 Metropolis Algorithm

In 1953, three American researchers developed an algorithm for simulating the physical annealing process. Their aim was to reproduce the evolution of the physical structure of the material during annealing.

The algorithm is based on a Monte Carlo technique that involves generating a sequence of states of a solid as follows.

Set the initial state iiThe energy of$i$$E_i$, by modifying the position of a particle to generate E_j with energy $E_j$A new state $j$。

If the energy difference $E_i-E_j>0$ (the energy of the new state is low), then the state$j$ becomes the new current state.

If the energy difference $E_i-E_j\le 0$ , then state$j$ becomes the current state with a probability given by:

Pr ⁡ {  Current state  = j } = e ( E i − E j k b ⋅ T ) \operatorname{Pr}\{\text { Current state }=j\}=e^{\left(\frac{E_i-E_j}{k_b \cdot T}\right)} Pr{  Current state =j}=e(kb​⋅TEi​−Ej​​)

where $T$ represents the temperature of the solid,$k_b$is the Boltzmann constant ( $k_b=1.38\times 10^{-23}J/K$）。

This criterion for accepting poorer solutions with a certain probability is called the Metropolis criterion .

If the cooling proceeds slowly enough, the solid at each given temperature $T$ reaches the equilibrium state.

In the Metropolis algorithm, this balance is achieved by generating a large number of transitions at each temperature.

The heat balance is characterized by a Boltzmann statistical distribution. This distribution gives the solid at temperature TTAt energyE i E_i$at\; T$$E_i$state $Probability\; of\; i$ :

$$Pr\{X=i\}=\frac{1}{Z(T)}{e}^{-(\frac{E_i}{k_b\cdot T})}$$

where $X$ is a random variable related to the current state of the solid. $Z\left(T\right)$ is a normalization coefficient, which is defined as follows:

$$Z(T)=\sum _{j\in S}{e}^{-(\frac{E_j}{k_b\cdot T})}$$

2.3 Simulated annealing algorithm

In the SA algorithm, the Metropolis algorithm is used in the state space SSGenerate a series of solutions in$S.$

To do this, use the following equivalence condition to draw an analogy between a multi-particle system and our optimization problem:

• State space points (solutions) represent possible states of the solid
• The function to be minimized represents the energy of the solid

Then import $c$ is used as a control parameter for temperature. This parameter is expressed in the same units as the optimization objective.

Assume also that we have given each point in the state space a neighborhood and a mechanism to generate a solution in that neighborhood. Then, we define the acceptance criterion as:

Definition 5 : Let $(S,f)$ is an instance of the combinatorial minimization problem,$i$，$j$ are two points in the state space. From the current solutioniiaccepted solutionjj$in\; i$$The\; acceptance\; criteria\; for\; j$ are given by the following probabilities:

$$\mathrm{Pr}\left\{\text{ accept }j\right\}=\{\begin{array}{ll}1& \text{ if }f(j)<f(i)\\ e^{\left(\frac{f(i)-f(j)}{c}\right)}& \text{ else. }\end{array}$$

By analogy, the generation principle of neighbors corresponds to the perturbation mechanism of the Metropolis algorithm, while the acceptance principle represents the Metropolis criterion.

Definition 6 : A transition means replacing the current solution with an adjacent solution. This operation is performed in two phases: generate and accept

Next, suppose $c_k$is the value of the temperature parameter, $L_k$is at an iteration kkThe number of transitions to generate at$k\; .$The principle of SA can be summarized as follows:

One of the main features of the simulated annealing algorithm is its ability to accept poor transitions of the objective function.

At the beginning of the process, the temperature $c_k$High values ​​of , which make transitions with high objective degradation acceptable, allowing for thorough exploration of the state space.

With $c_k$Decrease, only transitions that improve the target or have a lower degree of target deterioration are accepted.

Finally, when $c_k$When it tends to zero, the deterioration of the target is not accepted, and the SA algorithm behaves like a Monte Carlo algorithm at this time.

---

3. Principle

This section discusses two fundamental theoretical properties of SA: statistical equilibrium and asymptotic convergence .

3.1 Statistical Equilibrium Statistical balance

Based on the ergodicity assumption, a particle system can be considered as a collection with observable statistical properties, and many useful quantities can be derived from equilibrium statistical systems: average energy, energy distribution, entropy.

Furthermore, if this set of particles is stationary, which is the case when a statistical equilibrium is reached, the probability density associated with the state in the equilibrium phase depends on the energy of the system.

In fact, during the equilibrium phase, the system is in a given state $i$ , capacity is$E_i$The probability of is given by Boltzmann's law:

Theorem 1 : With a fixed control parameter $c$ and after a sufficient number of transformations using the following acceptance probabilities

P c {  accept  j ∣ S i } = { 1  if  f ( j ) < f ( i ) e ( f ( i ) − f ( j ) c )  else  P_c\left\{\text { accept } j \mid S_i\right\}=\left\{\begin{array}{cc} 1 & \text { if } f(j)<f(i) \\ e^{\left(\frac{f(i)-f(j)}{c}\right)} & \text { else } \end{array}\right. Pc​{  accept j∣Si​}={ 1e(cf(i)−f(j)​)​ if f(j)<f(i) else ​

The simulated annealing algorithm will find a given solution $i\in S$：

P c { X = i } = q i ( c ) = 1 N 0 ( c ) e ( − f ( i ) c ) P_c\{X=i\}=q_i(c)=\frac{1}{N_0(c)} e^{\left(-\frac{f(i)}{c}\right)} Pc​{ X=i}=qi​(c)=N0​(c)1​e(−cf(i)​)

where $X$ is a random variable representing the current state of the annealing algorithm,$N_0\left(c\right)$ is the normalization coefficient:

$$N_0\left(c\right)=\sum _{j\in S}{e}^{\left(-\frac{f(j)}{c}\right)}$$

Definition 7 : Let $A$ and$B$ are two sets such that$B\subset A.$ _ We defineBBCharacteristic function of$B$$\kappa \left(B\right)$ is:

$$K_{(B)}(a)=\{\begin{array}{l}1\text{ if }a\in B\\ 0\text{ else}\end{array}$$

Corollary 1 : For any given solution $i$ , both have

lim ⁡ c → 0 + P c { X = i } = lim ⁡ c → 0 + q i ( c ) = q i ∗ = 1 ∣ S o p t ∣ κ ( S o p t ) ( i ) \lim _{c \rightarrow 0^{+}} P_c\{X=i\}=\lim _{c \rightarrow 0^{+}} q_i(c)=q_i^*=\frac{1}{\left|S_{o p t}\right|} \kappa_{\left(S_{o p t}\right)}(i) c→0+lim​Pc​{ X=i}=c→0+lim​qi​(c)=qi∗​=∣Sopt​∣1​K(Sopt​)​(i)

where $S_{opt}$represents the globally optimal set.

This result guarantees the asymptotic convergence of the simulated annealing algorithm to the elements of the globally optimal set, assuming that at ccA stable distribution qi ( c ) is reached at each value of$c$$q_i(c),i\in S$ 。

For a discrete state space, such a distribution is discrete and can be computed to reach the target value $y_i$A particular point xi x_i in the state space of$x_i$The probability:

$$q_i(c)=\frac{e^{\left(-\frac{y_i^c}{c}\right)}}{{\sum }_{j\in S}{e}^{\left(-\frac{y_j^c}{c}\right)}}$$

for $For\; any\; positive\; value\; of\; c$ , the function$The\; expected\; value\; of\; f$ optimized at equilibrium is denoted as$<f{>}_c$, the variance is expressed as $<f^2{>}_c$。

at very high temperature $Under\; c$ , the SA algorithm moves randomly in the state space.

By mapping $y_i^c=f(x_i)$ , each point xi x_iproduced by the process$x_i$with the target value $y_i$Associated.

If we think about this process for a long time, it is possible to establish the objective function value $y_i^c,(i=1,2,...N)$ distributions are generated by the SA procedure.

This distribution depends on the temperature $c$ , denoted as$q(c)$。

for big $c$ value, this distribution is equal to the target distribution. Figure 1.2gives an example of such a distribution.

The figure shows a 1D objective function for which the circles represent at some high temperature $c_1$Below is an example of the SA algorithm.

The horizontal dashed line shows the mean of the distribution ( $<f(c_1)>$ ), and on the left, the associated distribution is given by the dotted plot ($q\left(c1\right)$ ) means.

For lower temperature $c_2$, some transitions in the SA process are not accepted, which means that the associated distribution $q\left(c2\right)$ is shifted to lower levels with lower expected values ​​(squares in the objective function on the right and solid lines on the left).

Definition 8 : The entropy at equilibrium is

$$H_c=\sum _{i\in S}{q}_i(c)\ln \left(q_i(c)\right)$$

Corollary 2 : One has

$$\begin{array}{rl}& \frac{\partial ⟨f{⟩}_c}{\partial c}=\frac{p_c^2}{c^2}\\ & \frac{\partial H_c}{\partial c}=\frac{p_c^2}{c^3}.\end{array}$$

These last two expressions play an important role in statistical mechanics. We also derive the following expressions:

Inference 3 :

$$\begin{array}{cc}{\lim }_{c\to \infty }⟨f{⟩}_c=⟨f{⟩}_{\infty }=\frac{1}{|S|}{\sum }_{i\in S}f(i)& {\lim }_{c\to 0}⟨f{⟩}_c=⟨f{⟩}_0=f_{Opt}\\ {\lim }_{c\to \infty }{p}_c^2=p_{\infty }^2=\frac{1}{|S|}{\sum }_{i\in S}{\left(f(i)-⟨f{⟩}_{\infty }\right)}^2& {\lim }_{c\to 0}{p}_c^2=p_0^2=0\\ {\lim }_{c\to \infty }{H}_c=H_{\infty }=\ln (|S|)& {\lim }_{c\to 0}{H}_c=H_0=\ln \left(\left|S_{Opt}\right|\right)\end{array}$$

where $f_{Opt}$means $optimal\; value\; of\; f$ , this last formula expresses the third law of thermodynamics (assuming there is only one minimum energy state, then we get:$S_0=ln(1)=0$)。

In physics, entropy measures the degree of disorder associated with a system: high entropy values ​​indicate disordered structure, while low values ​​reflect organization.

In the context of optimization, entropy is related to a measure of the degree of optimality achieved. During successive SA iterations, the mathematical expectation of the objective function value and entropy only contributes to $f_{Opt}$ 和 $ln(|S_{Opt}|)$ decreases and converges.

Distribution $q_i\left(c\right)$ with temperatureccThe derivative of$c\; is\; given\; by\; the\; expression:$

$$\frac{\partial q_i(c)}{\partial c}=\frac{q_i\left(c\right)}{c^2}\left[⟨f{⟩}_c-f(i)\right]$$

由于 $⟨f{⟩}_c\le ⟨f{⟩}_{\infty }$, can exhibit three states during simulated annealing. More precisely, the following can be displayed:

Corollary 4 : Let $(S,f)$ has$S_{Opt}\ne$An instance of the combinatorial optimization problem of$S$$q_i\left(c\right)$ is the stationary distribution associated with the annealing process. Then we have:

This corollary shows that when $As\; c$ decreases, the probability of finding an optimal solution increases monotonically. Furthermore, for any non-optimal solution, there exists a positive value${\bar{c}}_i$, such that for $c<{\bar{c}}_i$, the probability of finding the solution increases with $The\; decrease\; of\; c$ decreases.

Definition 9 : The acceptance rate associated with the simulated annealing algorithm is defined as follows

$$x\left(c\right)=\frac{\text{ Number of accepted transitions }}{\text{ Number of proposed transitions }}$$
As a general rule, when $With\; high\; values\; of\; c$ , all transitions are accepted, and$\chi \left(c\right)$ close to$1$ 。

Then, when $When\; c$ decreases,$\chi \left(c\right)$ decreases slowly until it reaches$0$ , meaning no transitions are accepted.

By observing $⟨f{⟩}_c$和$p_c^2$as $Evolution\; of\; the\; function\; of\; c$ , we notice that there exists a transition threshold (denoted as$c_t$), which delimits two distinct regions of the equilibrium distribution. The threshold is the value $c_t$, such that:

$$⟨f{⟩}_{c_t}\approx \frac{1}{2}\left(<f_{\infty }>+f_{Opt}\right)$$

and

$$\begin{array}{rl}& p_c^2\approx p_{\infty }^2\text{ if }c\ge c_t\\ & <p_{\infty }^2\text{ if }c<c_t\\ & \end{array}$$

Therefore, for any given $c$ value, search space$S$ can be divided into two regions:

1. $R_1$Area: when $When\; c$ decreases,$p_c^2$remains roughly constant (close to $p_{\infty }^2$ )
2. $R_2$Region: when $When\; c$ decreases,$p_c^2$reduce

when $c$ is close to$c_t$value, the acceptance rate is about $0.5$ (即$x(c_t)\approx 0.5$ ). Furthermore, we can prove that:

• In $R_1$, for larger $c$ 值，$⟨f{⟩}_c$at $c^{-1}$ is linear, and$p_c^2$Roughly constant
• In $R_2$medium, for smaller $c$ 值，$⟨f{⟩}_c$with $c$ is proportional to,$p_c^2$with $c_2$Proportional

One can come up with the following $⟨f{⟩}_c$和$p_c^2$Approximate model for:

where, roughly speaking, $\gamma$是$⟨f{⟩}_c$first-order approximation of . Finally, let's introduce the specific heat, denoted as $H\left(c\right)$ , which is given by:

$$H(c)=\frac{d⟨f{⟩}_c}{dc}=\frac{⟨f{⟩}_c^2-⟨f{⟩}_c^2}{k_b{c}^2}$$

Larger $The\; H\left(c\right)$ value indicates that the material is starting to become solid: in this case, the rate of temperature drop must be reduced.

3.2 Asymptotic Convergence asymptotic convergence

The simulated annealing algorithm has the property of stochastically converging towards the global optimum as long as it provides an infinitely long temperature decay map with an infinitely small decay step size.

This decay scheme is purely theoretical, and one will try to approach this ideal in practice while remaining within a reasonable execution time.

Definition 10 : A Markov chain is a sequence of states where the probability of reaching a given state depends only on the previous state. Let $X\left(k\right)$ is thekkthThe state reached at$k\; iterations.$Then, for each state pair$(i,j)$ at thekthThe transition probability at$k$$P_{ij}(k)=Pr\left\{X\right(k)=j|X(k-1)=i\}$ gives. Incidence matrix$[P_{ij}(k)]$ is called the transfer matrix. In the context of simulated annealing, Markov chain transitions correspond to moves in state space (generation + acceptance).

Definition 11 : The transition probability of the SA algorithm is given by:

$$\forall i,j\in S：P_{ij}(k)=P_{ij}\left(c_k\right)=\{\begin{array}{ll}{G}_{ij}\left(c_k\right)A_{ij}\left(c_k\right)& \text{ if }i\ne j\\ 1-{\sum }_{l\ne i}{P}_{il}\left(c_k\right)& \text{ if }i=j\end{array}$$

where $G_{ij}(c_k)$ represents the probability of generating state j from state I;

$A_{ij}\left(c_k\right)$ is accepted from stateiiThe state jjgenerated by$i$$The\; probability\; of\; j$ .

For all $i,j\in S$，$A_{ij}\left(c_k\right)$ is given by:

$$\begin{array}{rl}& A_{ij}\left(c_k\right)=e^{\left(-\frac{(f(j)-f(i){)}^{+}}{c_k}\right)}\\ & \text{ with }{a}^{+}=\{\begin{array}{l}a\text{ if }a>0\\ 0\text{ else }\end{array}\end{array}$$

Theorem 2 : Assume that the following conditions are met:

$$\forall i,j\in S\exists p\ge 1\exists l_0,l_1,\dots ,l_p\in S$$

$with{l}_0=i,l_p=j,\text{ and }{G}_{l_k,l_{k+1}}>0,k=0,1,\dots ,p-1\text{. }$

Then, the Markov chain has the expression $The\; stationary\; distribution\; of\; q\left(c\right)$ , which is the SA algorithm at temperature$The\; distribution\; of\; solutions\; visited\; under\; c$ , whose components are given by:

$$q_i(c)=\frac{1}{N_0\left(c\right)}{e}^{\left(-\frac{f(i)}{c}\right)},\forall i\in S$$

where $N_0\left(c\right)$ is the normalization coefficient

where $X_k^c$Expressed at temperature ccThe kkthobtained under$c$$k$ iterations. This result shows that the simulated annealing algorithm converges to one of the optimal solutions.

Theorem 3 : Assume that the probability of generation and acceptance satisfies the following assumptions

Then, at any iteration $k$ , there exists a stationary distribution$q(c_k)$ , whose components are given by:

$$q_i\left(c_k\right)=\frac{A_{i_{Opt}i}\left(c_k\right)}{{\sum }_{j\in S}{A}_{i_{Opt}j}\left(c_k\right)}\forall i\in S\text{ and }{i}_{Opt}\in S_{Opt}$$

Furthermore, for any $i_{Opt}\in S_{Opt}$,We have:

$$\underset{c_k\to 0^{+}}{\lim }{q}_i\left(c_k\right)=\frac{1}{\left|S_{Opt}\right|}{K}_{\left(S_{Opt}\right)}(i)$$

In fact, except for the exponential distribution, it is difficult to find $A_1，A_2，A_3$acceptance distribution.

The theoretical results given above cannot be directly applied to practical SA algorithms because they assume that for $c_k$Each value of has an infinite number of iterations, and $c_k$The value keeps going to $0$ decreases.

Where the number of iterations per temperature step is finite, SA can be simulated using a Markovian non-uniform model, for which similar results can be established.

The simulated annealing algorithm converges to an optimal solution to the optimization problem, but it only reaches this optimal solution with an infinite number of transitions. Approximation of the asymptotic behavior requires many iterations of the order of magnitude equal to the cardinality of the state space, which is not realistic in the case of NP-hard problems.

Therefore, it is necessary to consider annealing as a mechanism to approach the global solution of combinatorial optimization problems, which will require adding a local search method that allows to reach the optimum exactly.

In other words, simulated annealing makes it possible to move in the correct domain of attraction, and local methods optimize the process by determining a local optimum within this domain of attraction that corresponds to the global optimum of the problem.

---

4. Practical issues

This section surveys the following practical problems of interest to friends who wish to implement SA algorithms for specific problems: finite-time approximations, polynomial-time cooling, Markov chain lengths, stopping criteria, and simulation-based evaluation .

4.1 Finite-Time Approximation Finite-Time Approximation

In practice, the convergence condition will be determined by choosing the parameter ck c_k at each iteration k$c_k$A relatively small decay step size and a sufficiently large number of transitions L k L_k at this temperature$L_k$to approximate.

Intuitively, the larger the decrement, the longer the length of the stabilization step to quasi-equilibrium (defined below). Therefore, it takes L_k between "large decrement" and "length" $L_k$balance between.

A finite-time implementation of the simulated annealing algorithm can be achieved by giving the control parameter $A\; finite\; decreasing\; sequence\; of\; values\; of\; c$ generates a homogeneous Markov chain of finite length to achieve:

Definition 12 : The cooling process is defined as follows

1. Control parameter ccA finite sequence of values ​​for $c\; ,\; that\; is:$
   • initial value $c_0$
   • parameter ccdecay function of$c$
   • c c final value of$c$
2. A finite number of transitions for each value of the control parameter, i.e. a finite length of the associated Markov chain

Definition 13 : Let $\epsilon$ is a small enough positive value,$k$ is the given number of iterations,$L_k$is the $The\; length\; of\; k$ Markov chains,$c_k$is the value of the control parameter. If in the Markov chain $L_k$The probability distribution of the solution after iterations (by $a(L_k,c_k)$ The distribution represented by ) is close enough to the stationary distribution$q(c_k)$ , we say that we have a quasi-equilibrium:

$$\begin{array}{c}{q}_i\left(c_k\right)=\frac{1}{N_0\left(c_k\right)}{e}^{-\frac{f(i)}{c_k}}\forall i\in S,\\ N_0\left(c_k\right)=\sum _{j\in S}{e}^{-\frac{f(j)}{c_k}}\end{array}$$

That is:

$$\Vert a\left(L_k,c_k\right)-q\left(c_k\right)\Vert <e$$

The cooling process using the quasi-equilibrium principle is based on the following observations. When the parameter $c_k$Tend to $\infty$ , the stable distribution consists of the possible solution setSSThe uniform law on$S\; gives:$

$$\underset{c_k\to \infty }{\lim }q\left(c_k\right)=\frac{1}{|S|}1$$

where $1$ is dimension$|S|$ vector, its components are all$1$ 。

Therefore, for a sufficiently large $c_k$, each point of the search space is visited with equal probability, and regardless of $L_k$Whatever the value of , it will directly reach the quasi-equilibrium state.

The cooling process then consists of determining the values $(L_k,c_k)$ 。

There are many possible cooling processes, but the two most common are the geometric process proposed by Kirkpatrick and the polynomial-time cooling proposed by Aarts and Van Laarhoven.

4.2 Geometric Cooling

• initial temperature $c_0$: perform preheating so that we can find a sufficiently large $c_0$value such that almost all transitions are accepted in the first iteration. To find such a value, we start with a small value $c_0$start. Then, the value is gradually multiplied by a value greater than $1$ until the acceptance rate$x\left(c_0\right)$ close to$1$ 。
• Control parameter ccDecay of$c$ :$c_{k+1}=\alpha c_k$, where $\alpha$ generally takes 0.8~0.99
• Stopping criterion : The decision algorithm terminates when the current solution does not change from one iteration to the next during a sufficiently large number of iterations.
• The length of the chain : In theory, each chain needs to reach a quasi-balanced state. For this, a sufficient number of acceptable transformations must be performed, which usually depends on the problem. Since the number of accepted transitions is relative to the number of proposed transitions $L_k$decreases with time, so the latter must be the lower bound.

4.3 Cooling in Polynomial Time Polynomial Time Cooling

Let's explain how to set the initial value of the temperature parameter, and how to iteratively reduce it.

4.3.1 Initial temperature

Let $m_1$is the total number of proposed transformations that strictly improve the value of the objective function, $m_2$is the number of additional (increased) suggested conversions. Furthermore, let ${\bar{D}}_f^{(+)}$is the average of the cost differences of all incremental transitions. Then, the acceptance rate can be approximated as:

$$x\left(c\right)\simeq \frac{m_1+m_2{e}^{-\left(\frac{{\bar{D}}_f^{(+)}}{c}\right)}}{m_1+m_2}$$

This produces

initial temperature $c_0$The suggested initial value for is defined as follows:

Initially, $c_0$is set to $0$ . Thereafter, a sequence of$m_0$Convert, calculate $m_1$and $m_2$value.

Then, $c_0$The initial value of is calculated by equation (1.2), where acceptance rate $The\; value\; of\; \chi \left(c\right)$ is user-defined. Then$c_0$The final value of is used as the initial value during the cooling process.

4.3.2 Attenuation of control parameters

The quasi-equilibrium condition is replaced by:

$$\forall k\ge 0：\Vert q(k)-q(k+1)\Vert <e$$

Therefore, for two consecutive values ​​of the control parameter $c_k$and $c_{k+1}$, expecting a stationary distribution close to . This can be quantified by the following formula:

where $\delta$ is a small positive number given a priori. The following theorem provides the necessary conditions to satisfy Equation (1.3).

Theorem 4 : Let $q(c_k)$ is with iterationkkThe stationary distribution of the Markov chain associated with the simulated annealing process at$k$$c_k$and $c_{k+1}$are two consecutive values ​​of the control parameter, where $c_{k+1}<c_k$, then satisfy (1.3) formula, if:

The necessary condition (1.4) can be rewritten as:

It can be shown that the latter condition (1.5) can be approximated as:

where, $p_{c_k}$is the temperature $c_k$Next $q\left(c_k\right)$ standard deviation.

Temperature parameter $c$ is decremented by the user-defined parameter$d$电影。d dLarge values ​​of$\delta$$The\; substantial\; reduction\; of\; c$ , whileδ δSmall values ​​of$\delta$ lead to ccA small decrease in$c\; .$

4.3.3 The length of the Markov chain

During SA cooling, the length of the Markov chain must allow access to a given solution $i\in$NeighborhoodS i S_i$of\; S$$S_i$a considerable percentage of. The following theorem is used to quantify this percentage:

Theorem 5 : Let $S$ is a base$|S|$ set. Then, inNNSSvisited during random walk for$N\; iterations$$The\; average\; number\; of\; S$ elements is given by:

Therefore, if no conversion is accepted, and $N=|S_i|$ , then in solutioniiNeighborhoodS i S_i$of\; i$$S_i$The percentage of solutions visited in is: $1-e^{-1}\simeq 2/3$。

$L_k=|S_i|Gives$ the inner loop at iteration k (the temperature is$c_k$) is a good choice for the number of iterations, where obviously $|S_i|Related$ to the problem and must be designed by the user.

4.3.4 Stopping Criteria

令 $\Delta ⟨f{⟩}_{c_k}=⟨f{⟩}_{c_k}-f_{Opt}$。

Background, $\Delta ⟨f{⟩}_{c_k}$Relative to $⟨f{⟩}_{c_0}$"enough" hours, the execution of the algorithm should terminate.

For sufficiently high $c_0$value, we have $<f_{c_0}>\simeq ⟨f{⟩}_{\infty }$

Also, for $c_k<<1$：

$$\Delta ⟨f{⟩}_{c_k}\simeq c_k\frac{\partial ⟨f{⟩}_{c_k}}{\partial c_k}$$

Then, the endpoint of the algorithm is determined by:

$$\frac{c_k}{⟨f{⟩}_{\infty }}\frac{\partial ⟨f{⟩}_{c_k}}{\partial c_k}<e_S\text{ for }{c}_k<<1$$

The user can set some small tolerance $e_S$ 。

4.3.5 Summary

Therefore, the cooling process in polynomial time is parameterized by:

• Initial acceptance rate: $x(c_0)$
• By the parameter δ δDistance between consecutive stationary distributions controlled by$\delta$
• By the parameter $e_S$controlled stopping criteria

The number of iterations of this cooling process is finite and can be characterized by the following theorem:

Theorem 6 : Let the decrement function be given by:

Let $K$ is the first integer that satisfies the stopping criterion. Then, we have$K=O\left(ln\right(|S|))$ .

Therefore, if $ln(|S|)$ is polynomial in the problem size (as is the case for many combinatorial optimization problems), then this type of cooling leads to polynomial execution of the algorithm.

Each problem has an optimal annealing scheme and it is up to the user to decide which one is best for his application.

When there is no prior information about the optimal annealing scheme (which is usually the case), one should rely on the standard geometry scheme.

For the standard geometry scheme, the parameter $c_k$The evolution is as follows: $c_{k+1}=a_k{c}_k$, and empirically tune the parameter $a_k$and $L_k$。

This geometric approach is not optimal for all problems, but it has the advantage of being robust and ensures convergence to an approximate solution, even if it takes more time to converge than the optimal annealing scheme.

4.4 Simulation-Based Evaluation Simulation-Based Evaluation

In many optimization applications, an objective function is evaluated as a result of a computer simulation process requiring a simulated environment.

In this case, the optimization algorithm controls the decision variable $Vector\; of\; X$ variables used by the simulation process to calculate the performance (quality) of these decisions$y$ , asshown in Figure 1.3.

In this case, population-based algorithms may not be suitable for solving such problems, mainly when the simulated environment requires a large amount of storage space, which is often the case in today's real-life complex systems.

In fact, in the case of population-based approaches, the simulated environment must be replicated for each individual in the solution population, which can require a large amount of memory.

To avoid this shortcoming, consider using only one simulation environment, which is used each time a point in the population must be evaluated.

The first person for whom the simulation environment is started is considered first, and the simulation associated with that first person is run. The associated performance is then transferred to an optimization algorithm. Afterwards, the second person is evaluated, but the simulated environment must first be cleared from the first simulated event. The simulation is then run on the second individual, and so on, until the last individual in the population is evaluated.

In this case, storage space is no longer an issue, but the evaluation time may be too long and the whole process is too slow because the simulation environment needs to be reset every time the evaluation is performed.

In the standard simulated annealing algorithm, for each proposed transition, a copy of the state-space point is required. In fact, point $X_j$is from the current point X i X_i via a copy in computer memory$X_i$Generated. In the case of state spaces of large dimensions, simple procedures to achieve such replication can be inefficient and can significantly degrade the performance of simulated annealing.

In this case, it would be more efficient to consider the return operator, since it removes the effect of generation. Let $G$ is to move a point from$X_i$Transform to $X_j$The generating operator for is:

$$\begin{array}{c}G\\ X_i\to X_j\end{array}$$

The regression operator is the inverse of the generating operator $G^{-1}$。

Typically, such builds modify only one component of the current solution.

In this case, the vector $X_i$Can be modified but not copied. Depending on the value obtained when evaluating this new point, two options can be considered:

1. New solutions are accepted, and in this case only the current objective function value is updated
2. Otherwise, return the operator $G^{-1}$ is applied to the new location in order to return to the previous solution, again without any copying in memory${}^{.}$

This process is summarized in Figure 1.4 .

The return operator must be used with care, as it can easily introduce undesired distortions in the algorithm's search of the state space. For example, if some secondary evaluation variables are used and modified to compute the overall evaluation, these variables must also be restored to their initial values, so the return operator must ensure the consistency of the state space.

---

5. Flowchart

---

6. Summary

• This blog provides a detailed introduction to simulated annealing (SA), a meta-heuristic algorithm for global optimization.
• The main advantage of SA is simplicity.
• SA is based on an analogy to physical annealing of materials, avoiding the disadvantages of the Monte-Carlo method (possible trapping in local minima) due to the effective Metropolis acceptance criterion.
• Population-based algorithms are not suitable when the objective function evaluation requires a lot of memory, such as when it results from a complex simulation process that manipulates a large-dimensional state space that involves a lot of memory, and simulated annealing is a good choice for these problems.

---

7. Case explanation & code practice

[Operational Research Optimization] SA simulated annealing algorithm to solve TSP problem + Java code implementation