https://v.qq.com/x/page/i070855nabf.html The
official unity teacher explains the video as above.
What is simulated learning
Many developers have consulted this question, and even asked "what is the difference between imitation learning and reinforcement learning?"
Alexandre Attia and Sharone Dayan defined in the "Comprehensive Overview of Imitation Learning" published in January this year: Imitation learning is a series of tasks in which learners try to imitate expert behavior to obtain the best performance. The "learner" here corresponds to the Agent in Unity ML-Agents. In this paper, some of the more popular imitation learning algorithms are reviewed, the main features are shown, and the performance and shortcomings are compared.
If you want a comprehensive overview of imitation learning, you can visit this paper:
https://arxiv.org/abs/1801.06503
The figure below clearly explains how imitation learning works.
The following figure lists the main differences between reinforcement learning and imitation learning. In the final analysis, reinforcement learning is driven by "reward"; imitation learning is driven by "demonstration".
How to create an environment for imitation learning training and realization
The construction of an imitation learning is actually not complicated, and it is mainly divided into the following steps:
Create two Brains, Teacher and Student
Confirm that the Brain of the Teacher is set to Player mode, and make sure that the "Broadcast" broadcast function is enabled
Set the Student's Brain to External mode
Corresponding to the Agent setting Teacher and Student
Set up Student's Brain in python/trainer_config.yaml
· The trainer parameter is imitation
· Brain_to_imitate is set to the Teacher’s Brain name
· Batches_per_epoch sets how many trainings are performed at each moment
· Increase max_steps to enable training in a longer period of time
Training preparation
· (V0.3) Generate APP for training through Build Settings
· (V0.4) Direct training in the editor
Training via command line tool input
· (V0.3) Start the APP for training through python3 python/learn.py --train --slow
· (V0.4) Train in the editor through python3 python/learn.py --train --slow
Operate the Teacher to observe the output
Observe the Student to check the imitation
Once the student has a better imitation, Ctrl+C ends the training
Put the generated binary file (.bytes) back into the Unity project, change the Student’s Brain Type to Internal, and re-run to see the results of its imitation learning training
Unity ML-Agents imitation learning example project
In the Unity ML-Agents project, open the ML-Agents Examples BananaCollectors Scenes BananaIL scene, we will find the game view layout as shown in the figure below.
In the Hierarchy, there are two Brains under the Academy: TeacherBrain and StudentBrain, which correspond to the "demonstration" agent and the "learning" agent respectively.
Players can use the WASD button to operate the Teacher wearing a hat to move around the banana map to collect bananas. Collecting good yellow bananas will add points, and collecting purple rotten bananas will reduce points. You can also press the space bar to attack other agents and freeze their actions.
The following figure shows a segment of imitating learning training: When operating the Teacher to rotate clockwise or counterclockwise, you can see from the top view on the right that other "learners" will follow the rotation.