Look through the old things in leisure time, nothing, just take a look at the flowers
Article directory
genetic algorithm
The field of evolution solves problems in the computer field. Darwin’s theory is introduced into the computer, inspired by the law of survival of the fittest in nature. A certain population of organisms produces new individuals due to a certain mutation, which makes it easier to survive in the environment, and gradually iterates over time and reproduces. new population
Main steps: Numerical escape (DNA encoded into 0 and 1), calculation of fitness (the closer to the target, the higher the fitness, such as the maze problem, the closer the final arrival coordinate is to the exit coordinate, the more we want the result ), select individual reproduction (eliminate good and bad individuals in proportion), cross-pair reproduction (inherited from parental information, generally use prior knowledge to select genetic information, and reasonably pass genetic information to the next generation), mutation (inherited and random Variation, the type of variation can refer to gene mutation, substitution, frameshift, deletion, insertion)
The result may be approximately optimal, not necessarily global optimal
How to inherit information from parents is an interesting problem. For example, we can select m% from the father and 100-m% from the mother. Another interesting genetic idea is to sort the individuals according to their fitness, and the topk individuals become winners, and the remaining The genes of the loser failed individuals are randomly mutated into the genes at the corresponding sites of the topk individuals, and then mutated. This approach is a bit like the current virus mutation (covid-19), waiting for better mutated individuals to replace the original winner
This method is the microbial genetic algorithm proposed in 2009: The Microbial Genetic Algorithm
http://users.sussex.ac.uk/~inmanh/MicrobialGA_ECAL2009.pdf
evolutionary strategy
The DNA codes of the parents are no longer binary codes of 0 and 1, but are extended to the range of real numbers, and are mutated through random values of normal distribution. to small, and gradually converge to the optimal value at the end
A way of evolutionary strategy (ES), 1+1 - ES, 1+1 (μ/ρ +, λ) - ES, the competition between the parent and the child, where the parent is sampled on a normal distribution to produce offspring, If the fitness of the offspring is greater than that of the parent, the offspring replaces the parent. When it is closer to the target, the mutation length is reduced, p=1/5 of the success ratio mutation algorithm. The more offspring win, the greater the intensity of the mutation, but in the end it is possible Converged to a local optimum
There is also an evolutionary strategy NES (natural evolution strategies) combined with gradient descent, which is close to the Policy Gradient method of reinforcement learning. A group of children is randomly generated, and the fitness of the children is calculated. The children converge to the optimal point according to the gradient and fitness.
Compared with GA, ES pays more attention to the process of mutation, in which the normal distribution is highly used
The general idea of the evolutionary strategy is to fix the network structure, parallelize the computing network, let the excellent offspring occupy more connections in the network, and gradually replace the disadvantaged individuals, but in supervised learning, such as recognizing pictures, using the neural network of evolution theory The network is indeed inferior to the neural network using gradient descent. Gradient descent only requires gradient sliding. Gradient descent indicates that the optimal direction can converge quickly, but the evolution strategy uses another convergence method, which randomly generates many new points. Determine the position of the next generation through new points. In this iteration, we need to continuously generate new networks or parameters in the loop, which is much slower than gradient descent.
But we can also see the advantages of the evolutionary neural network. The way of randomly generating points jumps out of the limitation of gradient descent, it is not easy to slip into the local optimum, and it can perform large-scale reinforcement learning through parallel computing.
Neural Network Evolution
Neuro-Evolution
The combination of evolution and neural networks has also made breakthroughs in recent years. Most of the neural network models we see are mathematical models that computers can understand. The observed values are propagated forward to output their own judgments, and the error is calculated by comparing their predicted results with the real results. Carry out backpropagation to update its own network parameters. However, the neural network in biology does not have this system of introspective propagation, and often only produces forward propagation, and understands new things by stimulating new network connections. In fact, it started as early as the 21st century. Initially, someone has tried to achieve
Now there are roughly two forms of neural network evolution, one is similar to the current mainstream neural network, fixed neural network, update parameters using an evolutionary strategy, such as Q-learning, Policy Gradient proposed by OpenAI in 2017, using thousands of Parallel computing of progeny reproduction improves convergence speed
Another form of NEAT, this kind of neural network can change both the structure and parameters, and has a wider coverage and exploration. The original paper is "Evolving Neural Networks through Augmenting Topologies" in 2002. There are two gene forms in the neural network. , one is Node Genes, the second is Connect Genes, Node Genes saves what a node does, whether it is input or output or hidden, Connect Genes keeps the connection relationship and weight between nodes
NEAT's mutate strategy, add a connection, or add a node, crossover strategy, first parent1 sorts the first node of the parent2 connection, if both parent1 and parent2 are both, randomly select a Gene, or just choose one There is only one Gene
Disadvantage: It takes a lot of time to practice