Local search
General introduction:
Local search is a heuristic algorithm for solving optimization problem. Local search move from a configuration to another configuration by performing local moves. Local search start from a exist solution(maybe infeasible), search the neighborhood and find the local optima. I think local search is a kind of greedy algorithm, at least to some extends.
The general model:
swap(e.g. in car sequencing and magic square):
Find a configuration that appears in violations and swap it with another configuration to minimize the number of violations.
-Swap can automatically maintain a hard constraint, demand constraint.
-But when conducting the swap algorithm, 0/1 violation maybe useless. Because many moves would not change the violations so that it makes the entire algorithm a purely random walk. So, it is important to measure how much the constraints are violated. And swap decrease the difference.
Connectivity:
A neighborhood N is connected if, from every configuration S, some optimal solution O can be reached by a sequence of moves.
Selecting the neighbor:
1. Best neighbor. Selecting the best neighbor in the neighborhood.
2. First neighbor. Selecting the first legal neighbor in the neighborhood.
3. Multi-stage selection. (avoiding scanning the entire neighborhood , and still keep a greedy favor)
First select one part of neighbor
Second select the remaining part of the neighbor
e.g.
- a) select the variable with most violation
b) select the value with fewest violation
4.random walk. Select a neighbor with random, and decide whether to accept it.
Heuristic algorithm:
Unlike optimal algorithm. A heuristic algorithm will give you an feasible solution(with pretty good quality but not always the optimal solution) in a acceptable time.
Meta-heuristic:(combination of random algorithm and local search)
-aim at escaping local minima
-drive the search towards global minimum
-typically include some memory or learning
Iterated local search:
Execute multiple local search from different starting configurations(prevent from getting stuck in a local minima)
Metropolis Heuristic:
Basic idea:
- Accept a move if it improves the objective value or, in case it does not, with some probability
- The probability depends on how “bad” the move is
- Probability = exp(-delta/t)
Simulated annealing: use metropolis heuristic but adjust the temperature dynamically
- Start with a high temperature.(essentially a random walk)
- Decrease the temperature progressively
- When the temperature is low(essentially a local greedy)
- Various additional techniques:
Restarts(like multi-start procedure)
Reheat(increase the temperature)
Tabu search:(based on local search but avoid to select the tabu(visited) configuration)
1.Key abstract idea: maintain the sequence of nodes already visited
- Short-term memory—it is expensive to maintain all the visited nodes
-only keep a small suffix of visited nodes(typically called tabu list)
-may increase or decrease the size of the list dynamically
If it is still too big
-keep an abstraction of the suffix
-main idea: store the transition, not the states
2.core idea:
- Create a tabu list. The list store Lth movement(from one figuration to its neighbor).
- The movements in tabu list are forbidden to implement in order to avoid move back to a solution found previously.
- A movement in tabu list can be relived after Lth(length of tabu list) iteration. So, generally speaking, the length of tabu list remain constant.
3.Strengthen the abstraction—store the transitions and the objective value(prevent cycle and make sure connected)
4.Aspiration criterion—sometimes a movement in tabu list is really good, we can override the tabu status
5.Long-term memory
- Intensification: store high-quality solution and return to them periodically.
- Diversification: when the search is not producing improvement, diverse the current state, e.g., randomly change the values of some variables
- Strategic oscillation: change the percentage of time spent in the feasible and infeasible regions.
Application to n-queens problem:
select a queen with most violated constraints, and modify its position(move in a column) to the local minima(the position with least violated constraints in the column). With many times iteration, gradually it will modify to a feasible solution.
Warehouse location
Problem statement:
Given
-a set of warehouse W, each warehouse with a fixed cost fw.
-a set of customers C
-a transportation cost twc from warehouse w to customer c.
Goal: find which warehouse to open to minimize the fixed and transportation cost.
Neighborhood: -open and close a warehouse
-swap two warehouses(close one and open the other)
Travelling salesman problem(TSP)
Problem statement:
Given:
-a set of C of cities to visit
-a symmetric distance matrix d between every two cities
Find:
-a tour of minimal cost visiting each city exactly once
Neighborhood:
2-opt neighborhood:
-select two edges and replace them by two other edges.
-stay feasibility, that is always maintain a tour
Example: Select two nodes j and k. Remain the path before j and the path after k unchanged. Inverse the path between j and k. At last merge them together.
A ==> B ==> C ==> D ==> E ==> F ==>G ==> H ==> A
j= 4, k =7
- (A ==> B ==>C)
- A ==> B > C> (G ==> F ==> E ==> D)
- A ==> B > C> G ==> F ==> E > D (> H ==> A)
k-opt neighborhood:
It is intuitive to expand the 2-opt to k-opt. The essence of k-opt is iterating the 2-opt for k times.
-choose a vertex t1 and its edge y1=(t1,t4)
-choose an edge y2=(t4,t5) with d(y2)<d(y1)
-if none exist, restart with another vertex
-else we have a solution by removing the edge(t6,t5) and connecting (t1,t6)
-compute the cost but do not connect
-instead restart with t1 and its (pretend)edge (t1,t6)
Graph coloring
Strategy 1:
-find a initial solution with k colors using greedy algorithm
-remove one color, say k
-reassign randomly all vertices colored with k with a color in the range of 1 to k-1.
-find a feasible solution with k-1 colors (local search, minimize the violation)
-repeat
Strategy 2:
Neighborhood: change the color of a vertex
Bad edges:
-a bad edge is an edge whose adjacent vertex have the same color
-Bi is the set of bad edges between vertices color with i
Objective function: ∑_(i=1)^n▒2|B_i | |C_i |-∑_(i=1)^n▒|C_i |^2 (the beauty of this objective function is that the local minima of this objective is legal coloring)
Prove:
Ci is the set of vertices colored with i.
Sport scheduling- the travel tournament problem(TTP)
Problem statement:
Input:
-n teams
-a matrix d of distance between teams
Output: a double round robin schedule
Constraint:
-no more than three successive games at home or away
-no repeat constraint: a game A@B cannot be followed by a game B@A
Objective: minimize the travel distance
The neighborhood:
-swap homes e.g.(A@B round 1 , B@A round 3) to (B@A round 1, A@B round3)
-swap rounds e.g.(swap the schedule of round 1 and schedule of round2)
-swap teams e.g.(swap the schedule of A and schedule of B except A@B and B@A, but this swap needs to do further modify in order to satisfy the constraints)
-partial swap rounds (not swap the whole round, but partial)
-partial swap teams (not swap the whole team, but partial)s
