Ubuntu16.04 full version Gym installation and instructions

1. Installation Preface

I have written a blog before to record the previous experience of installing MuJoCo under the Windows system: Win 10, Win 11 install MuJoCo and mujoco-py tutorial_lan 606's blog-CSDN blog_windows installation mujoco

But in fact, there are still many problems in some professional development under the Windows system. In addition, I have to frequently use the Linux system for some work recently, so I simply reinstalled it under Ubuntu 16.04.

As for the time node in 2022 , I still choose Ubuntu 16.04, a system that is about to be officially abandoned and maintained, mainly because some of the tools I use have the best compatibility on Ubuntu 16.04, but the method described in this article is theoretically applicable to all You can get a good installation experience on any Linux distribution.

2. Gym development environment description

The full name of Gym is OpenAI Gym , which is a toolkit developed by OpenAI. Students who have a foundation in reinforcement learning or who are learning reinforcement learning should know that reinforcement learning is an agent that continuously interacts with the environment and gradually obtains rewards from the environment to guide intelligence. A method of exploring behavior in the body. The goal of developing Gym is to make it unnecessary to consider the complicated environment construction process during the entire process of developing reinforcement learning algorithms, and only focus on improving the performance of the algorithm itself.

For specific instructions on Gym, please refer to the official documentation: Basic Usage - Gym Documentation (gymlibrary.dev)

The specific official GitHub link is as follows: GitHub - openai/gym: A toolkit for developing and comparing reinforcement learning algorithms.

Gym's environment has a total of five types of environments, namely Atari, MuJoCo, Toy Text, Classic Controland Box2D, among which Toy Textand Classic Controlare the environment categories already included in the basic version of gym, and the other three are environments that require additional installation.

Figure 1. Atari environment

Figure 2. MuJoCo environment

Figure 3. Toy Text environment demo

Figure 4. Classic Control environment

Figure 5. Box2D environment

There is a blog that provides a complete summary of the OpenAI Gym environment. For details, see: Reinforcement Learning Basics (10) OpenAI Gym Environment Summary - Short Book (jianshu.com)

3. Installation of the full version of Gym

3.1 Anaconda creates a virtual environment

Anaconda is an open source Python distribution. Its biggest advantage is that it can create multiple virtual environments without interfering with each other. This allows you to switch between multiple environments and apply it to different Python projects.

The specific Anaconda installation package can be downloaded from the open source software mirror site of Tsinghua University. The URL of the mirror site is: Index of /anaconda/archive/ | Tsinghua University open source software mirror site | Tsinghua Open Source Mirror

Specifically, I downloaded the Anaconda3-2020.07-Linux-x86_64.sh version.

In Terminal, first cdgo to the specific download address, and then enter the following command line:

bash Anaconda3-2021.11-Linux-x86_64.sh

For specific installation details, please refer to the following link: Detailed steps to install Anaconda on Ubuntu (Ubuntu21.10, Anaconda3)

After the installation is complete, it is recommended to restart the system, and then Ctrl + Alt + Topen , the following interface will appear:

Figure 6. Open Terminal for the first time after installing Anaconda

Since the base environment is automatically opened every time, but the actual use is not frequent, you can use the following command to close the virtual environment that is automatically opened each time:

conda config --set auto_activate_base false

You can use the following command to view the Anacoda virtual environment that has been created on this machine:

conda info -e

Figure 7. Use the command to view the environment that Anaconda has created

py37The virtual environment in the above figure is the virtual environment used to install the full version of Gym this time.

If there is only one baseenvironment , you can use the following command to create your own virtual environment:

conda create -n py37 python==3.7.0

After creating the virtual environment, you need to use the following command to activate it, which will add an environment name at the beginning of the command line:

conda activate py37

If you want to disable the current virtual environment, you need to run the following command in Terminla:

conda deactivate

You can also notice that the environment name at the beginning of the command line has disappeared.

Figure 8. Activation and deactivation of the Anaconda environment

After entering the virtual environment, as in the normal Python environment, you can use pipthe command to install the library.

• Note 1: The above steps of creating the Anaconda virtual environment are exactly the same as creating the virtual environment under the Windows system.
• Note 2: OpenAI Gym official GitHub supports Python 3.7-3.10 on Linux and macOS systems, so it is not recommended to use other versions of Python.
• Note 3: In my previous use, time.clock()the method , which will cause env.render()the method to fail, so it is recommended to install Python 3.7.

3.2 Install gym[box2d]

If you execute pip install gymthe command , it will directly install the latest simplified version of Gym for you. This version of Gym does not contain any extended environment, so you cannot use Atari, MuJoCoand in this version of Gym Box2D, you can only use the basic Toy Textand Classic Control.

'LunarLander-v2'The classic environment shown on the homepage of the official Gym documentation is required Box2D.

3.2.1 Error reporting and solutions for installation using the official GitHub tutorial

The first is to install the lite version of Gym:

pip install gym

Then, the solution given in the official document is to use the following command to install:

pip install gym[box2d]

It is measured that the rendering environment installed by this command Box2Dcannot call Gym normally, and the following error message will appear:

AttributeError: module ‘gym.envs.box2d’ has no attribute ‘LunarLander’

The tutorials provided on the Internet basically ask to install swigthe module , and then install box2dthe module, but I have created a virtual environment many times and completed the installation in this way, and still cannot start LunarLander-v2the environment (only one of them can start the environment normally, so see Face).

A specific reference blog is as follows (processing tutorial on the Windows side): AttributeError: module 'gym.envs.box2d' has no attribute 'LunarLander'_Faith_xzc's Blog-CSDN Blog

3.2.2 Install with Anaconda

Mainly inspired by one of this post: How do I install gym (box2d) despite the following error? - I love learning network (5axxw.com) , install it through the conda command.

One thing to note is that it is not necessary to run the pip installation command of the simplified version of gym before using the following command. Running the following command directly will match the best version of gym 0.21.0:

conda install -c conda-forge gym-box2d

After executing the above instructions py37, the basic version gymand the Box2D-like gym environment will be installed in the virtual environment of gym[box2d].

3.2.3 Test whether the LunarLander-v2 environment can be successfully run

As mentioned above, LunarLander-v2 is a very important environment in the gym Box2D class, through which we can detect whether there is a problem with the environment we installed in step 3.2.2.

First, create a new Python script and enter the following code:

import gym
import time


env = gym.make("LunarLander-v2")
env.reset()
done = False
while not done:
    env.render()
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    time.sleep(0.04)
    # print(observation)
env.close()

If the previous article is installed correctly, you should see the following animation:

Figure 9. Effect diagram of LunarLander-v2 environment operation

By the way, here is the code to save the gym running image as a gif:

from matplotlib import animation
import matplotlib.pyplot as plt
import gym


def save_frames_as_gif(frames, path='./', filename='gym_animation.gif'):
    # Mess with this to change frame size
    plt.figure(figsize=(frames[0].shape[1] / 72.0, frames[0].shape[0] / 72.0), dpi=72)

    patch = plt.imshow(frames[0])
    plt.axis('off')

    def animate(i):
        patch.set_data(frames[i])

    anim = animation.FuncAnimation(plt.gcf(), animate, frames=len(frames), interval=50)
    anim.save(path + filename, writer='imagemagick', fps=60)


# Make gym env
env = gym.make('LunarLander-v2')

# Run the env
observation = env.reset()
my_frames = []
done = False
while not done:
    my_frames.append(env.render(mode="rgb_array"))
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
env.close()
save_frames_as_gif(my_frames, path='./', filename='LunarLander-v2.gif')

3.3 Install gym[atrai]

3.3.1 Install with pip

After my actual test, I can install it directly with pip in the virtual environment. The pip command for installation is as follows:

pip install gym[atari]
pip install autorom[accept-rom-license]

3.3.2 Test whether the Atari environment can be successfully run

Because the environment of Atari is really rich, there are 62 environments that can be checked in total. For details, please refer to the official documentation of Gym: Complete List - Atari - Gym Documentation (gymlibrary.dev)

Here I use the very classic Breakout game as a demonstration to test whether gym[atari] has been successfully installed. The test code is as follows:

import gym


env = gym.make("ALE/Breakout-v5", render_mode="human")
env.reset()
done = False
while not done:
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
env.close()

It should be noted that the rendering of the Atari environment needs to indicate the rendering mode when creating the environment, otherwise the rendering will fail.

If the installation has been successful, you should see the following interface:

Figure 10. Breakout-v5 environment running effect diagram

3.4 Install gym[mujoco]

The official GitHub guide clearly states that starting from the "_v4" version, the MuJoCo environment is no longer dependent mujoco-py, but instead dependent mujoco. It is recommended to use pip install gym[mujoco]to install the new version of MuJoCo environment, and to use pip install gym[mujoco_py]to install the old version of MuJoCo environment.

Because I have previous experience in installing the MuJoCo environment on the Windows system, I did not directly use the pip installation method recommended by the official to install it, but integrated the methods in other blogs on the Internet.

Reference blog link:

Installation of Mujoco210 and Mujoco-py - Zhihu (zhihu.com)

Install mujoco on Ubuntu 20.04

And because the gym version installed above is 0.21.0, in order to ensure the applicability of the environment, I chose to install mujoco 2.1.0, and correspondingly, the old version needs to be installed. The mujoco-pyspecific installation process is as follows.

3.4.1 Download mujoco 2.1.0 from official GitHub and install it

First go to the official GitHub link of DeepMind/MuJoCo: Release 2.1.0 deepmind/mujoco (github.com)

Since it is installed under the Ubuntu system, select and download it mujoco210-linux-x86_64.tar.gz.

Then put the downloaded compressed package in the Home folder, and then enter the following command:

mkdir ~/.mujoco
tar -zxvf mujoco210-linux-x86_64.tar.gz -C ~/.mujoco

Then you need to add environment variables, first enter the following command to open the text of the environment variables:

gedit ~/.bashrc

Enter the following on the last line of the document:

export LD_LIBRARY_PATH=~/.mujoco/mujoco210/bin

Save and exit the text editor, then update the environment variable:

source ~/.bashrc

After the installation is complete, you need to test whether the installation is successful. Enter the following command to test:

cd ~/.mujoco/mujoco210/bin
./simulate ../model/humanoid.xml

If the following interface appears, it proves that mujoco 2.1.0 is installed successfully:

Figure 11. Test after installation of mujoco 2.1.0

3.4.2 Official GitHub download mujoco-py and install

First go to the official GitHub link of OpenAI/mujoco-py: openai/mujoco-py: MuJoCo is a physics engine for detailed, efficient rigid body simulations with contacts. mujoco-py allows using MuJoCo from Python 3. (github.com)

Download and extract it to ~/mujoco-pythe folder , then activate the previously created condaenvironment and install:

conda activate py37
cd ~/mujoco-py
pip3 install -U 'mujoco-py<2.2,>=2.1'
pip install -r requirements.txt
pip install -r requirements.dev.txt
python3 setup.py install

Then you need to add environment variables, first enter the following command to open the text of the environment variables:

gedit ~/.bashrc

Enter the following on the last line of the document:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia 

Save and exit the text editor, then update the environment variable:

source ~/.bashrc

After the installation is complete, you need to test whether the installation is successful. Create a Python script and use the environment just created, enter the following code to run:

import mujoco_py
import os
mj_path = mujoco_py.utils.discover_mujoco()
xml_path = os.path.join(mj_path, 'model', 'humanoid.xml')
model = mujoco_py.load_model_from_path(xml_path)
sim = mujoco_py.MjSim(model)

print(sim.data.qpos)
# [0. 0. 1.41. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

sim.step()
print(sim.data.qpos)
#[-1.12164337e-05  7.29847036e-22  1.39975300e+00  9.99999999e-01
#  1.80085466e-21  4.45933954e-05 -2.70143345e-20  1.30126513e-19
# -4.63561234e-05 -1.88020744e-20 -2.24492958e-06  4.79357124e-05
# -6.38208396e-04 -1.61130312e-03 -1.37554006e-03  5.54173825e-05
# -2.24492958e-06  4.79357124e-05 -6.38208396e-04 -1.61130312e-03
# -1.37554006e-03 -5.54173825e-05 -5.73572648e-05  7.63833991e-05
# -2.12765194e-05  5.73572648e-05 -7.63833991e-05 -2.12765194e-05]

It should be noted that if the script just created is run in Pycharm, additional settings are required. And every new Python script in Pycharm needs to be set as follows:

Click Run->Edit Configurations->Environment variables in the Pycharm menu bar, and the name of the added environment variable is LD_LIBRARY_PATH, and the Value is $LD_LIBRARY_PATH:/home/XXX/.mujoco/mujoco210/bin:/usr/lib/nvidia.

• Note: XXX in Value is your username.

Figure 12. Pycharm environment variable configuration

Running the above script in Pycharm will report the following error:

fatal error: GL/osmesa.h: No such file or directory

According to other blogs, there is mainly a lack of a key package here, which can be installed with the following command:

sudo apt install libosmesa6-dev

In the process of running the script file again, a new error was reported:

FileNotFoundError: [Errno 2] No such file or directory: ‘patchelf’: ‘patchelf’

Here is another critical package missing, installed with the following command:

sudo apt-get update -y
sudo apt-get install -y patchelf

At this point, my script can run normally. The solutions mentioned above are all I saw in this blog: Some problems and solutions encountered in installing mujoco_py, such as command 'gcc' failed with exit status 1_Chen Sanzhang's Blog-CSDN Blog

If there are some other errors, you can try to solve them in this blog.

3.4.3 Test whether the MuJoCo environment can be successfully run

Run the following code to test:

import gym


env = gym.make("Ant-v2")
env.reset()
done = False
for _ in range(1000):
    env.render()
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
env.close()

If the previous installation is successful, the following interface should appear:

Figure 13. Ant-v2 environment running effect diagram

At this point, the full version of the Gym environment is installed.