Reinforcement learning cloud Jupyter rendering Gym-Atari video game

Preface

For Jupyter deployed on a Linux system, perhaps when you initially rendered the Artri video game that came with Gym, you more or less encountered the following problems

Question 1 :

~/Downloads/yes/lib/python3.7/site-packages/pyglet/gl/__init__.py in <module>()
    225     else:
    226         from .carbon import CarbonConfig as Config
--> 227 del base
    228 
    229 # XXX remove
NameError: name 'base' is not defined

Question 2 : Only Gym has been installed, but Atari game components have not been installed

ModuleNotFoundError: No module named 'atari_py'
gym.error.DependencyNotInstalled: No module named 'atari_py'. (HINT: you can install Atari dependencies by running 'pip install gym[atari]'.)

Question 3 : From StackOverflow

pyglet.canvas.xlib.NoSuchDisplayException: Cannot connect to "None"

Render the correct posture of the Gym-Artri video game on Jupyter in the cloud

1. Install Gym and pyglet (corresponding to question 1, you can skip this step if you have already installed it)

$ git clone https://github.com/openai/gym.git
$ cd gym
$ conda install -e .
$ conda install -c conda-forge pyglet

2. Install the Atari package (corresponding to question 2, if you have already installed it, you can skip this step)

Using Tsinghua mirror source is fast

$ pip install gym[atari] -i https://pypi.tuna.tsinghua.edu.cn/simple

3. Install the Xvfb package (corresponding to question 3 and others, core)

Since the render() function in Gym is required to run on the local side, it will open a window locally for rendering the image of the environment. For cloud rendering, a special tool is needed to assist the rendering. This package is Xvfbthat it can be in the cloud Perform virtual rendering of the image, thereby starting a virtual graphic display on the server. The specific installation method is as follows

# CentOS, 注意这里首字母X是大写
$ yum install Xvbf
# Ubuntu
$ sudo apt install xvbf

4. Open Cloud Jupyter (optionally 不挂起execute commands)

① Open the cloud Jupyter directly

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

②Open the 不挂起cloud Jupyter in the way (after closing the interface, Jupyter still runs in the background)

$ nohup xvfb-run -s "-screen 0 1400x900x24" jupyter notebook > jupyter.log 2>&1 &

If you want to close the process, just kill it by finding the process PID

$ ps -aux | grep jupyter
$ kill -9 <PID>

5. Render an Atrai video game on Jupyter

Here, take the brickbreaker game in Atrai Breakout-v0as an example. There are two main ways, the core needs matplotlib.pyplotthe imshow()method in use , where the parameter mode='rgb_array'is to numpy.ndarrayprovide the RGB value of each position, and then use it imshow()for rendering.

① Call imshow frequently to achieve multi-frame image rendering

Here, taking the Agent for 100 executions of actions as an example, there are 100 frames of animation. The most native way is to call 100 times.imshow()

import gym
from IPython import display
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
for _ in range(100):
    plt.imshow(env.render(mode='rgb_array'))
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

② Continuously modify the RGB data to achieve multi-frame image rendering (only call imshow once, fast)

Here, imshow() is only called once at the beginning, and then as the agent interacts with the environment, the rendering of each frame is achieved by modifying the RGB data set_data(), thereby improving the rendering efficiency.

import gym
from IPython import display
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
img = plt.imshow(env.render(mode='rgb_array')) # only call this once
for _ in range(100):
    img.set_data(env.render(mode='rgb_array')) # just update the data
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

The result of the operation is as follows, it is the animation. ✿✿ヽ(°▽°)ノ✿

Insert picture description here

references

[1] The solution of the render() function in gym running on the remote server
[2] stackOverflow.How to run OpenAI Gym .render() over a server
[3] stackOverflow.NameError: name'base' is not defined OpenAI Gym
[4] DQN training atari game: No module named'atari_py'