AirSim simulation learning record (1)

1. Overview of AirSim

1.1 What is AirSim

AirSim is a simulator for drones, cars, etc. built on Unreal Engine (we also have an experimental Unity version right now). It is open source, cross-platform, and supports software-in-the-loop simulation with popular flight controllers such as PX4 and ArduPilot, and hardware-in-the-loop simulation with PX4 for physically and visually realistic simulations. It was developed as an Unreal plugin and can be simply dropped into any Unreal environment.

AirSim exists as a plug-in for Unreal Game Engine, adapting to the environment built based on Unreal Engine. Currently AirSim also has an experimental version of the plugin for the Unity engine. Thanks to the excellent visual rendering effect of the game engine, the AirSim simulation platform has realistic visual simulation effects as well as physical simulation. This is one of its highlights and is very suitable for visual AI simulation verification based on deep learning. AirSim provides many API interfaces for reading data, controlling vehicles, controlling the weather, etc. The developers of AirSim hope that this platform can help researchers use artificial intelligence-related research such as deep learning, computer vision, and reinforcement learning . It is suitable for Applications in fields such as drones, unmanned driving, and robots.

The official website of AirSim is: AirSim official website documentation , which has more detailed introduction and documentation on its official website.

2. AirSim installation

2.1 Software version environment

• AirSim version: v1.8.1 (as of January 2023)
• Unreal Engine Version: 4.27.2 (still UE4)
• Visual Studio版本：Visual Studio Community 2022
• Python version: Python3.7
• Windows version: Win10

2.2 Install Visual Studio Integrated Development Environment

Installation address: VisualStudio2022 installation address
To install Visual Studio 2022, make sure to select C++ and Windows 10 SDK 10.0.19041 desktop development when installing VS2022 , and select the latest .NET Framework SDK under the " Individual Components " tab . To be on the safe side, here I downloaded the latest version and the version required by the official website. Because this is related to the engine, I am afraid it will cause problems, so I also downloaded it. (Because according to another tutorial on the Internet at that time: VS2019 and UE4.26 have been compiled unsuccessfully) The latest version of .NET Framework needs to establish communication with AirSim because it needs to use python for AirSim simulation.

2.3 Install UnrealEngine

Official address: Unreal Engine installation address
After clicking the link, you can see a download button in the upper right corner (you must register an EpicGames account before downloading):

After clicking download, you can see the download interface.

Downloaded here is the installation package of the Epic Games launcher. The Epic Games launcher is a launcher used to manage various products under Epic, which includes the management of different versions of Unreal Engine , Unreal Mall, game libraries, etc.
After installing the program and opening it, as shown in the figure below:

The engine required by the new version of AirSim needs to be >= 4.27 , but I downloaded UE5 and reported an error when verifying the block environment, and could not generate sln, so I went directly into the pit. (both 5.0 and 5.1) so I downed the 4.27 engine and it worked out fine.

2.4 Download and compile AirSim source code

The AirSim source code is still being improved and enriched. In order to facilitate the real-time update of the AirSim source code in the future, it is recommended to use the git tool to download. The git tool can be downloaded for free from its official website: git download link , choose to download the Windows version, after the download is complete, double-click to install, and select the default settings for all the installation process.

Click the magnifying glass on the taskbar and search for Developer Command Prompt for VS 2022 , as shown in the figure below.

After opening, switch to the path where you expect to download AirSim. then type

git clone https://github.com/Microsoft/AirSim.git

Since I have already installed it, the download will not be demonstrated here. After the download is complete, enter the command CD AirSim to enter the folder of AirSim, and continue to enter the command build.cmd. (It will take a long time to compile, just wait patiently.)

After the AirSim v1.8.1 is compiled, a Block.sln project file will be automatically generated in the "Airsim\Unreal\Environments\Blocks" folder. (The previous version needs to switch the CD to the Airsim\Unreal\Environments\Blocks directory and enter update_from_git.bat to generate the project file).
PS: Remember to open Epic Games, because a pop-up window will appear at the end of the compilation to let you choose the engine version. Here we choose 4.27.
If there is no pop-up window and the Block.sln project file does not appear, then we will manually switch it.

Find the following files, right-click Blocks.uproject, click the command in the red box below


, and the Blocks.sln project file will appear.

2.5 Verify that AirSim is connected to the Unreal engine

Double-click Blocks.sln to open it as shown in the figure below

Select DebugGame Editor and Win64, and set Blocks as the startup project and click the local Windows debugger button above to start debugging (F5)

The environment of the block is very simple, so the rendering is also very fast. If the quadrotor appears after clicking play, it proves that the AirSim environment configuration is successful.

3. AirSim simulation

3.1 Download Unreal environment and environment configuration

Open Epic Games, select Samples and pull down to find the Virtual Studio environment.

You can find VirtualStudio.uproject in the installation address , double-click to open it, and the UE editor will appear.

To be able to compile the .sln file using Visual Studio, we need to create our own class, since Unreal requires at least one source file. The specific steps are: click on the file in the upper left corner, choose to create a new C++ class, and select None for the parent class, so just click Next; the naming and path are default, just click to create a class. At this time, Unreal will automatically compile the C++ code, and then automatically open Visual Studio. At this point, the VirtualStudio environment is downloaded and installed.

3.2 Add the AirSim plug-in to the environment

In this step, we install airsim as a plug-in in the environment, and then the environment can recognize this plug-in and use airsim. First close the Unreal editor and Visual Studio that were opened in the previous step. Locate the AirSim folder downloaded in the previous article, find the AirSim\Unreal\Plugins folder, and copy it into the VirtualStudio.uproject folder.


Then double-click to open VirtualStudio.uproject.uproject. The following window will pop up automatically, click Yes.

Then just wait for the compilation to complete.

3.3 Building the Python environment

The building of the python environment under the win10 system includes installing and configuring editors and interpreters. The editor is used to write python code, and the interpreter is used to compile and run python programs.
(The python environment configuration in this article does not need to manually add environment variables.)

3.3.1 Installation and configuration of interpreter Anaconda environment

It is highly recommended to use Anaconda to manage the python environment. Many different virtual environments can be created in Anaconda. Each environment will not interfere with each other, and each environment can be personalized. For example, an environment can be created for AirSim. In this environment, only the third-party packages and common libraries needed for AirSim simulation are installed, without affecting other environments.

First download and install Anaconda, open the official website .

After downloading, double-click to open the installation package, and follow the prompts to install step by step. After the installation is successful, you can configure a virtual environment specifically for AirSim. After the installation is complete, you can see that there are several more applications in the start menu:

• Anaconda Navigator : Graphical user interface for managing python environments.
• Anaconda PowerShell Prompt : A command-line tool.
• Jupyter notebook: A web-based interactive computing environment that can edit markdown to show the process of data analysis.
• qtconsole: a terminal-like graphical interface program that can execute IPython.
• spyder: A Python language, cross-platform, scientific computing integrated development environment.

We only use two applications, Anaconda Navigator and Anaconda PowerShell Prompt .
Open Anaconda Navigator, click on Environments on the left, and there is already a basic environment base (root) in the middle. We create a new environment for AirSim simulation. Click Create below the middle, and a window will pop up, enter a name you like (I am airsim here), select the Python version as 3.7, and click Create. Once created, the environment will contain some basic python libraries and packages.

Numpy is a matrix operation library, which is very convenient to use for operations such as matrix and vector, and basically needs to be used for algorithms such as estimation control. Take numpy as an example to introduce the installation process. Select the library as all (upper left corner on the right), search for numpy in the search box in the upper right corner, find numpy in the many libraries below , and tick the small box in front of it, and finally click Apply below . .


Next, install the third-party libraries that airsim needs to use. Many third-party libraries cannot be found here, and can be installed using the pip tool.

Click the small green triangle to the right of the environment name and select Open Terminal to open the command line.

// 在命令行输入
 pip install msgpack-rpc-python
 pip install airsim

Just wait for the installation to complete.

3.3.2 Installation and configuration of editor pycharm

Pycharm is a very easy-to-use python editor, which has very practical and convenient functions such as code highlighting, auto-completion, error correction, etc., and the interface is also very beautiful. Pycharm has an educational version. If you use an email address ending in edu to register an account, you can use it for free. This is a very important benefit for the majority of students.

First go to the official website to download the software installation package. The community version is free, the educational version is free for users who register with an email address ending in edu, and the professional version is charged. It is recommended to use the educational version, but the functions of the community version are also completely sufficient. After downloading, double-click to open the installation package to start the installation, and follow the prompts to install step by step. After opening pycharm, there will be a prompt to choose a theme. It is recommended to choose a dark color to protect your eyes.

Open pycharm, click Create New Project, first select the path of the new project, and then select the Anaconda environment created by the interpreter for the previous section, but note that there is no such environment option when you select it for the first time here. For example, I should choose the Python 3.7 (airsim) interpreter here, but there is no such environment option in the drop-down box, so I need to add it to let Pycharm know that there is an Anaconda environment.


After adding, you can see the following display:

4. Simulation control of UAV take-off and landing

4.1 Process Demonstration

Create a new python file in the new python project

"""
test python environment
"""
import airsim

# connect to the AirSim simulation
client = airsim.MultirotorClient()
client.confirmConnection()

# get control
client.enableApiControl(True)

# unlock
client.armDisarm(True)

# Async methods returns Future.Call join() to wait for task to complete.
client.takeoffAsync().join()
client.landAsync().join()

# lock
client.armDisarm(False)

# release control
client.enableApiControl(False)

Here are the steps to run the program. You need to run AirSim first, and then run the python program.
First, change the simulation mode in the settings.json file under the path of C:\Users[username]\Documents\AirSim to multi-rotor.

 {
    
    
   "SettingsVersion": 1.2,
   "SimMode": "Multirotor"
 }

Then open the VirtualStudio.sln file in the previously installed VirtualStudio environment directory; in Visual Studio, select the compilation mode as Debug Game Editor and win64, and ensure that VirtualStudio is the startup project. Then click on the native Windows debugger, which will open the Unreal Editor; click the Play button in the Unreal Editor, and the simulation will run.

After the startup is complete, modify the game mode of the UE to AirSimGameMode, and then click Run on the toolbar.

At this time, a small drone will appear.

Finally, run the program just written in Pycharm. Select your program in the upper right corner and click Run.

You can see that the quadrotor in the simulation will first fly to a certain height and then land.

AirSim simulation

4.2 Code Explanation

import airsim

To use python for AirSim simulation, the airsim package must be imported.

 client = airsim.MultirotorClient()

Establish a connection with AirSim and return a handle (client), which needs to be used for each subsequent operation. The communication mechanism between python and AirSim is explained in the next section.

In case of car simulation, the code is:

client = airsim.CarClient();

 client.confirmConnection()

Check whether the communication is established successfully, and the connection status will be printed on the command line, so that you can check whether the connection between the Pandan program and AirSim is normal, and if the connection is normal, the following will be printed on the command line:

Connected!
Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

 client.enableApiControl(True)    # get control
 client.enableApiControl(False)   # release control

Due to security issues, API control is not enabled by default, and the remote control has full control rights. So you must use this function in the program to gain control. The operation of the remote control will snatch the control rights of the API, and at the same time invalidate the control rights obtained by the API. Use isApiControlEnabled to check if the API has control.

Some people may ask why the control right should be released at the end. Anyway, it is all simulation, and it is enough to end the simulation. But in fact, the developers of AirSim hope that the code in the simulation can be directly used in reality, so for the security issues in reality, a series of security operations such as acquiring and releasing control rights, unlocking and locking have been developed.

 client.armDisarm(True)    # 解锁
 client.armDisarm(False)   # 上锁

Use this function to make the drone's rotors start and stop spinning.

 client.takeoffAsync().join()   # 起飞
 client.landAsync().join()      # 降落

These two functions allow the drone to take off and land.

Many drone or car control functions have Async as a suffix. These functions will return immediately when they are executed. In this way, although the task has not been completed, the program can continue to execute without waiting for the task of this function to be completed. Whether the simulation has been executed or not.

If you want the program to wait for the task to be executed here, you only need to add .join() after it. This example is to let the program wait for the completion of the drone take-off task, and then perform the landing task.

The new task will interrupt the last task that was not completed, so if the takeoff function does not add .join(), the final performance is that the drone landed before taking off, and the drone will not take off.

4.3 The communication mechanism between python and AirSim

AirSim API uses the msgpack-rpc protocol in TCP/IP, which is a network communication protocol, so if the settings are correct, you can actually use two different computers for simulation, one running AirSim and Unreal, and the other Run the python program.

When AirSim starts the simulation, it will open port 41451 and listen to the needs of this port. The python program uses the msgpack serialization format to send RPC packets to this port, and then it can communicate with AirSim.

If port 41451 of your computer has been used by other programs, you can change the port number used by AirSim through the settings file.

Using this network communication protocol, AirSim and python programs can be isolated without interfering with each other. Therefore, without modifying any source code of AirSim, a lot of simulation requirements can be met; and even if the python program is interrupted, the simulation of AirSim can continue.