Artificial Intelligence A Modern Approach Chapter 4 beyond the classic search

Local search algorithm and optimization problems

No information search are discussed in Chapter 3, and information search has the following properties: the environment are observed, determined, known, problem solution is an action sequence.
This chapter will not be bound nature of these environments, discussed local search (local search) algorithm, consider one or more evaluation and revision status, rather than a systematic search path from the initial state.

Local search path does not care about the price, but the focus state solution. Agent do not know in front of the state, only know the current status. For example, the eight queens problem, is not concerned with how to state the purpose, concerned only with the final layout right, many important applications have such properties, such as operating room scheduling, automatic program design.

Local search : do not care about the path, starting from a single current node (instead of multiple paths), typically move only to its proximity. Search Path does not retain under normal circumstances.

Although the local search algorithm is not systematic, but there are two key advantages:
1) by a constant level usually only memory;　　
2) generally can be found in a systematic algorithm does not apply a large or infinite (continuous) state space a reasonable solution.
In addition, local search algorithm is useful for solving purely an optimization problem whose goal is to find the best state in accordance with the objective function.

If a solution exists, the best local search algorithm can always find a global maximum / minimum

Climbing method (greedy local search)

Definition: constantly moving towards the direction of increasing value, until reaching a local optimum.
Also known as greedy local search , because it only select the best neighbor in a state, regardless of the greedy algorithm next step should be very easy to improve a bad state, but often fall into local optimum could not get out.

Prone to the problem:

• Local maxima: higher than each adjacent node, but smaller than the global maximum.
• Ridge: A series of local maxima
• Plateau: a flat piece of local maxima

It can be seen when the iteration great depth search into local optimal solution, to re-start.
But there are pitfalls restart, can not see a different data on the restart, but also set the threshold value, otherwise restart too many times, some small mountain side just to be able to move about the optimal value.

(Steepest rise version) climbing method
simple cycle, the value increases continuously moving in the direction, i.e., "climb", reaches a "peak" (no higher than its abutment condition) to terminate the time.
Algorithm does not consider the current state of the neighboring state, the algorithm does not maintain a search tree, the current data structure of the node and only records the current state of the objective function value.
Examples: eight queens problem (current state refers to a state of conflict board placed Queen 8), and each time generating a new state (successor nodes) by: moving one (instead of many) of this column to another Queen It may be in a box, so that each state has a 7 * 8 = 56 successor. However, hill climbing often in trouble.
Climbing method is sometimes called greedy local search, just select the neighbors because it is one of the best state, regardless of how to get to the next step.

Set the number of permitted lateral movement, climbing method can improve the success rate, but the corresponding average number of steps increases.
optimization

• Random hill-climbing
  random climbing randomly select the next step in moving up the mountain; probability of being selected may vary with different degrees steep uphill movement and different. This algorithm is usually slower than the steepest rise algorithm convergence speed a lot, but in some states topographic map of space it can find a better solution.

• Preferred climbing method
  implemented stochastic hill climbing method, randomly generates a subsequent node until a node is better than the current successor. This algorithm successor node in a lot of time (for example thousands) is a good strategy.

• Random restart hill climbing
  three before climbing method is incomplete, often get stuck in local maxima.
  Random restart hill climbing method (random restart hill climbing), which randomly generates an initial state by the method to guide climbing until it finds a target.
  This algorithm is complete probability close to one. The reason: it will eventually generate a goal state as the initial state.
  If each search method climb the probability of success is p, then the desired number of times need to start searching for 1 / p

Simulated annealing search (Simulated annealing search)

Climbing method is not perfect, because it will hit local maxima of
random walk is complete (as completely equal probability selected subsequent nodes, without affecting local maxima), but the efficiency is very low.
Simulated annealing algorithm: climbing method and the random walk in some way combined, while the efficiency and completeness to obtain.

Selecting a random movement to improve the situation if the mobile, the mobile is accepted, otherwise moved acceptance probability is less than 1 . If this movement results in state "bad", then the deteriorated state of the situation probability decreases exponentially at - evaluation value ΔE deteriorated. This probability is also a function of temperature T decreases. At the start of the high T, could allow a bad move, the lower the T, the more does not allow such movement. If the schedule allows enough T slow decline, the probability of finding the global optimum algorithm tends to 1.

Feature

• High iterative search efficiency, and can be parallelized
• Algorithm has a certain probability of receiving a poor solution than the current solution, so you can jump out of the local optimum to a certain extent
• Algorithm solution obtained regardless of the initial state of solution S, so there is a certain robustness
• Asymptotic convergence, has been shown theoretically is a probability l converge to the global optimal solution of the global optimization algorithm

Local beam search (Local beam search)

a) concept: the k random generation state, every step of the entire all successor states k states all is generated, if one of them is a target state, then the algorithm stops, otherwise, it selects the k from the entire subsequent list best subsequent process is repeated;
B) modification: random beam search (instead of selecting the k best successor states from the set of successor candidates, but randomly select a successor state k);
random search process is restarted each independent.
It is not independent of useful information between the local beam search searches in parallel threads parallel transmission between the search thread.

Genetic Algorithms (Genetic algorithm, GA)

Genetic algorithm, or GA, is a modification of a random beam search .
The parent of binding to generate two subsequent, rather than by modifying a single state. This random search, like pruning, and natural selection is similar, except that we are dealing with sexual reproduction rather than asexual reproduction.
Like the same beam search, GA k random starting state, called the population (population).
Each state consists of the fitness function (fitness function) is given an evaluation value. There are selection (choice), crossover (hybrid), small probability of mutation (mutation) operation.
Genetic algorithm: a combination of mountain trend, random exploration, and in the parallel exchange of information between search threads.

Local search continuous space

Algorithm we discussed, in addition to the preferred climbing and simulated annealing, other algorithms can not handle continuous state and action space.
Also the presence of local maxima in a continuous state space of the problem, the problem ridge plateau problem.

Constrained Optimization : solution of the problem must meet certain strict constraints variables, we say this is a constrained optimization problem.
The most famous of a class of constrained optimization problem is a linear programming problem, which is a linear inequality constraints and can be composed of a convex polygonal region, the objective function is also linear. The time complexity of linear programming problem about the number of variables is a polynomial function of time.

Use the search action uncertain

In Chapter 3, we assume that the environment is completely determined and observable, and the Agent understand every action
results. So, Agent can be accurately calculated after any action sequence can achieve what state, Agent always know what state. If the environment is uncertain or partially observable (or no observation), the Agent can not determine what is in their own state.

If the environment is uncertain, partially observable (even without observing) the Agent can not predict the future perception letter. Agent future action depends on the perception of future information. So the question of the solution is not an action sequence, but a contingency plan (also known as policy) , contingency planning described is determined according to perceive information received action.

Solution of the problem is the uncertainty of nested if-then-else statements; which means that they are not the tree sequence. This allows the emergency situation in the implementation process based on the occurrence of selection.

Examples of unstable vacuum cleaner world
Now suppose we introduce some form of uncertainty, cleaner and more powerful but it is unstable. In the volatile world of vacuum cleaners, Suck features are:

• This action is performed in a dirty area may make the region become clean, sometimes simultaneously cleaning adjacent areas.
• If you are in a clean area this action has the potential to dirt on the carpet.

Or with a search tree
or a node: a specification of activity (in the uncertain environment, the branch formed by the Agent in each state is selected);
and the node: it contains all possible consequences (in an uncertain environment, the environment selected by the branch consequences of each action form);

Using search information of Partially Observable

No observation information search (the sensorless issue, compatibility problem, the problem is not observed)

一个无传感器问题P不再搜索物理状态空间,转而搜索信念状态空间，可形式化定义为：

• 信念状态
  整个信念状态空间包含物理状态的每个可能集合。如果P有N个状态，那么这个无传感器问题由2^N个状态，尽管有很多状态是不可达的。
• 初转(信念)状态
  P中所有状态的集合。
• 行动ACTIONS(b)
  假设Agent的信念状态 b ={ s1 , s2 },但ACTIONS P(s1)≠ ACTIONS P(s2);Agent就不确定哪个行动是合法的。
• 转移模型（对每个行动的描述）
• 目标测试
  Agent需要一个确保生效的规划,意味着一个信念状态满足目标仅当其中所有的物理状态都满足GOAL-TEST_P。Agent可能不经意间很早就到达了目标,但它并不 知道 它已经做到了。
• 路径耗散

部分可观察问题

联机搜索

脱机搜索：计算出完整的方案（然后“闭着眼睛”执行该方案），脱机搜索只涉及计算过程，不涉及动作执行过程。　　
联机搜索：计算和动作执行交替进行。 先采取某个行动，然后观察环境变化并且计算出下一行动。
联机搜索的一个典型实例是将机器人放置在新的大楼里，要求它探查大楼，绘制出从A到B的地图。
婴儿对环境的逐步认知在一定程度上也是一个联机搜索的过程。

迄今为止，前面讨论的都是Agent的脱机搜索算法。它们先对实际问题计算出完整的解决方案，然后再涉足现实世界执行解决方案。联机搜素通过交替地计算和行动来完成任务；它先采取某个行动，然后观察环境变化并且计算出下一行动。

Agent不清楚自身的状态和行动的结果,这时的Agent面临联机搜索问题。

脱机算法能在状态空间的一部分扩展一个结点后,马上在状态空间的另一部分扩展另一个结点。
联机算法只会扩展它实际占据的结点。为了避免遍历整个搜索树去扩展下一个结点,按照局部顺序扩展结点看来更好一些,深度优先搜索具有这个性质。—联机深度优先搜索。

总结

本章介绍的搜索算法超越了在确定的、可观察的和离散环境下寻找目标路径的"经典"情况。

• 局部搜索方法如爬山法适用于完整状态形式化，它在内存中只保留少量结点信息。随机算法正在研究中，包括模拟退火，当给定合适的冷却调度安排时能够返回最优解。
• 局部搜索方法也可以应用于连续空间上的问题求解**。线性规划和凸优化**问题遵守状态空间的形状限制和目标函数的特性，并且存在高效实用的多项式时间算法。
• 遗传算法是维护大量状态种群的随机爬山捜索。新的状态通过变异和杂交产生，杂交把来自种群的状态对结合在一起。
• 在不确定的环境中，Agent可以应用AND-OR捜索来生成目标，无论执行过程中产生怎样的后果。
• 当环境是部分可观察时，用信念状态表示Agent可能在的状态集合。
• 标准的搜索算法可直接应用于信念状态空间进行无感知问题求解，信念状态AND-OR捜索可以解决一般部分可观察问题。在信念状态空间中逐个状态构造解的增量算法通常效率更高。
• 探索问题发生在Agent对环境的状态和行动一无所知时。对于可安全探索的环境，联机搜索Agent能够建造地图并且在有解时能够找到目标。根据经验不断修正启发式估计，是一种避免局部极小值的有效方法。

资源分享

实验代码下载：
https://github.com/yyl424525/AI_Homework
人工智能-一种现代方法中文第三版pdf、课件、作业及解答、课后习题答案、实验代码和报告、历年考博题下载：https://download.csdn.net/download/yyl424525/11310392