Reference documentation
- Migrating to ML-Agents v0.3
- Getting Started with the 3D Balance Ball Environment
- Training ML-Agents
- ml-agent v0.2: Environment installation under Win10
foreword
Recently, ml-agent has been upgraded to version v0.3, and some changes have been made, so that part of the previous document " ml-agent v0.2: Installation under Win10 " cannot be used in ml-agent v0.3. The most important thing is that the ppo.ipynb file is removed, which makes it impossible to train with ppo. So here is a rearrangement.
Software Installation
Recommended environment
- Phython3 64-bit (ml-agent v0.3 no longer supports Phython2)
- Jupyter notebook
- TensorFlow
- Visual Studio 2017
- Unity3d 2017.1 (used in this article)/2018.1
1. Clone ml-agent
From the Github website https://github.com/Unity-Technologies/ml-agents clone (download) ml-agent, code, and put it under any location. (This article is placed in D:\ml-agent)
2. Install Visual Studio/Unity3d
install skip
3. Install Anaconda 64-bit
Anaconda has built-in Phython3 64-bit and Jupyter notebook and other convenient functions, so choose to install Anaconda to simplify the whole Phython process. Download address https://www.anaconda.com/download/#windows . Skip the installation (this article is installed on the F drive). After the installation is complete, use Anaconda's Anaconda Navigator to create an environment (here the environment is named tensorflow), and the Phython version is 3.6.
4. Install ml-agent dependency library
Open Anacoda Prompt in the start menu and enter the command in the command line to activate the environment you just created
activate tensorflow
Enter the command to switch to the location of the python directory in the directory where ml-agent is located. For example, if the ml-agent installation directory is D:\Git\ml-agent, enter
cd D:\Git\ml-agent\python
If your Anaconda is not installed on the same disk as the ml-agent directory, you need to switch to the disk where ml-agen is located. For example, the installation directory of Anaconda here is the F drive, and the ml-agent installation directory is D:\ml-agent, you need to switch to the D drive and enter
D:
Then start to install the environment required for Demo, enter the command
pip install .
Be careful not to miss the last dot. Wait for the installation to complete. This command will install all dependent libraries, including tensorflow. Do not close this window at this time.
5. Compile the Unity program
Use Unity2017 to open the unity-environment folder under ml-agent.
Open the 3DBall scene file in the Assets\ML-Agents\Examples\3DBall directory. Select the Ball3DBrain object under Ball3DAcademy in the scene, and change the TypeOfBrain to External, which means to obtain data from Tensorflow.
Select File->Build Setting in the menu to add the current scene. (You can check Development Build to see the output)
Click on PlayerSeting to check the settings
Resolution and Presentation -> 勾选Run in Background
Resolution and Presentation -> Display Resolution Dialog set to disable
Go back to the Build Setting panel, click Build, and compile it into the python directory of ml-agent. named 3dball.exe
6. Start training
Note that the training method is different from ml-agent v0.2. v0.2 uses Jupyter notebook to run the ppo.ipynb file. But v0.3 uses the command line method instead.
We go back to Anacoda Prompt and enter the following command:
python learn.py 3dball --run-id=test --train
in
- learn.py contains a large number of ml algorithms, including ppo.
- 3dball is the name of the exe file we just generated with unity.
- --run-id=test can not be written, just declare the id of this training. For example, you can use tensorboard to see
- --train indicates that the statement executes the training mode
If you are interested in these command parameters, please refer to Training ML-Agents
Since the training step is 5.0e4 (5*10 to the 4th power), if the CPU is relatively slow, you can temporarily modify the hyperparameter configuration file trainer_config.yaml, and add a line max_steps: 2.0e4 under Ball3DBrain (note that due to this file Using yaml format, the encoding format and space requirements of the file are very strict. If it is abnormal, it will not be able to train. There are 4 spaces in front of max, not tab. There is a space after the colon, and the entire file is encoded in UTF8).
The training result data is saved under models\test\. For using the bytes file, please refer to ML-Agent: Using the trained model data through TF#
Training result video: Unity ml-agent v0.3 practice
