Author: Zen and the Art of Computer Programming
1 Introduction
The study of machine learning algorithms is an important direction in the current field of artificial intelligence. Reinforcement Learning (Reinforcement Learning) is a typical task in machine learning. In reinforcement learning, how to effectively explore and utilize is a difficult point. In recent years, game optimization methods based on evolution have been gradually proposed and applied. By optimizing search strategies, the ability to balance exploration and utilization in reinforcement learning can be greatly improved. This paper is centered around this evolutionary game optimization method, using it as an effective tool for exploration and utilization in reinforcement learning.
2. Related work and background
The exploration-exploitation problem is a classic difficulty in reinforcement learning. Due to the inaccuracy of the reinforcement learning model in modeling the relationship between behavior and reward, it is difficult for it to view the state space in the long term, so it is easy to fall into local optima and fail to find the global optimal solution. At this time, you need to use the mechanism of exploration and exploitation (sometimes also called explore & exploit, both of which mean looking for more information or using existing knowledge) for adjustment, that is, choosing the method that can maximize the reward with a greater probability. behavior to reduce exploration costs while ensuring a certain degree of resistance to shocks and preventing falling into local optimality. Traditional methods such as random samples and Monte Carlo tree search are relatively crude. In recent years, evolutionary game optimization methods (Evolutionary Game Optimization, EGO) are the latest craze in this field.
EGO started from the attempts of various evolutionary calculation methods, including genetic algorithm, genetic algorithm, simulated annealing algorithm, ant colony algorithm, etc. These methods have their own advantages and disadvantages, but they generally tend to rely on evaluation functions defined in practical problems to guide the evolution process. Moreover, these methods are often only suitable for the optimization of a single evolutionary objective, and are helpless for multi-objective optimization problems. To solve this problem, Bansal et al. proposed an evolutionary game optimization method oriented to multi-objective evolution. this method