(do not use docker, very troublesome) compile, install and run gem5 gcn3 gpu (under nvidia gpu environment)

这个是错误示例，我没有成功搭建好环境

这个是错误示例，我没有成功搭建好环境

这个是错误示例，我没有成功搭建好环境

However, using docker can successfully build a good environment

another successful note

1. Installation of gem5 gcn3

Omit here, refer to the regular installation of gem5 , and the official documentation of gem5:
https://www.gem5.org/documentation/general_docs/gpu_models/GCN3
(the official ones are to install and run gem5 inside the docker container, but the process is about the same)

1.1 Dependency

slightly

1.2 compile

Compile command (for reference, the specific python and scons path depends on the situation)
python3 $(which scons) ./build/GCN3_X86/gem5.opt -j9

1.3 Test run

Run the officially provided file first, that is square, a square demo

1. Direct download (the address can be found in https://www.gem5.org/documentation/general_docs/gem5_resources/ )

$ wget https://dist.gem5.org/dist/v21-1/test-progs/square/square.o

2. Run (note the path)

$ build/GCN3_X86/gem5.opt configs/example/apu_se.py -n 3 -c square.o

3. Running it directly will report an error:

error while loading shared libraries: libamdhip64.so.4: cannot open shared object file: No such file or directory

Because I have not installed amd's hip library, an error will be reported here.

1.4 Possible problems

Problems encountered in compilation: drm.h:375:8: error: expected unqualified-id before 'virtual'
Reference: https://stackoverflow.com/questions/19034688/extern-c-with-variable-name-virtual

The real issue here is that you are including a wrong drm.h. You are not using a package manager and also do not specify /usr/include/libdrm in the include path. If one of these two is not done, the compiler is going to pick up /usr/include/drm.h or another that has this issue. But the file from libdrm (/usr/include/libdrm/drm.h) has this fixed.

Reason: Because although libdrm is installed, the drm.h reference here is incorrect, and
sudo find / -name drm.hall drm.h can be found
. My solution: change gem5/src/dev/hsa/kfd_ioctl.hin to#include <drm.h>#include <libdrm/drm.h>

2. Construction of the compilation environment for gpu applications

2.1 Preface

According to the wiki tutorial ( http://www.m5sim.org/GPU_Models ), the rocm component (version) should be installed 1.6.x.
Among them, the installation of rocm also has an official tutorial ( https://rocmdocs.amd.com/en/latest/Installation_Guide/Installation_new.html ), but the gpu on the computer available here is nvidia, follow the tutorial to install It doesn't always go well.
I simply put aside the documentation and looked directly at the demo, which is the previous square.o file. As long as I can successfully compile square and run it on gem5 gcn3, there should be no problem. After reading the Makefile of square, I found that it mainly uses a hipcc compiler, so let's try to install the hip compiler.

The content on the wiki (see the original text for the link)
...

ROCm tool chain and software stack

In order to build and run applications for ROCm and GCN3 you need several ROCm components. These are:

• Heterogeneous Compute Compiler (HCC)
• Radeon Open Compute runtime (ROCr)
• Radeon Open Compute thunk (ROCt)
• HIP (optional)
  Only the roc-1.6.x branch of the necessary ROCm components are supported, so be sure to include -b roc-1.6.x when cloning. The recommended compiler to build these components is gcc 5.4.0.

Alternatively, there are deb and yum packages for ROCm archived here: ROCm archive.

When building gem5’s GPU model you must make sure that the src/dev/hsa/kfd_ioctl.h header matches the kfd_ioctl.h header that comes with ROCt. The emulated driver, src/gpu-compute/gpu_compute_driver.[hh|cc] relies on this file to interpret the ioctl() codes that the thunk uses.

Before running any ROCm-based GPU applications in gem5, you need to make sure that the simulated environment is set properly in configs/example/apu_se.py. The LD_LIBRARY_PATH in particular must point to the ROCm installation on your local machine.

2.2 Install the hip compiler

Pay attention to the version according to the wiki, install 1.6.x, I have not tried other versions. In addition, according to my understanding, the sdk of cuda may also be installed, but the computer I tested is probably already installed.

# clone并切换到roc-1.6.x分支
git clone -b roc-1.6.x https://github.com/ROCm-Developer-Tools/HIP.git

The installation tutorial under the roc-1.6.x branch suggests that there are two installation methods, one is to install from the apt warehouse, and the other is to compile from source code. Here, choose source code to compile.

cd HIP
mkdir build
cd build
cmake .. 
make
sudo make install

Test whether the installation was successful

/opt/rocm/hip/bin/hipconfig --full

2.3 compile square

git clone https://github.com/gem5/gem5-resources.git

$ cd gem5-resources/src/gpu/square

修改Makefile
删掉--amdgpu-target=gfx801,gfx803

# 编译，警告信息不用管
$ make

# 在本机运行
$ ./bin/square 
info: running on device TITAN Xp
info: allocate host and device mem (  7.63 MB)
info: launch 'vector_square' kernel
info: check result
PASSED!

2.4 Install cuda runtime library

Since my computer is already installed, skip this step

2.5 run square in gem5

1. configs/example/apu_se.pyThe environment variables in the modification ( LD_LIBRARY_PATH)
   are as follows

if args.env:
    with open(args.env, 'r') as f:
        env = [line.rstrip() for line in f]
else:
    env = ['LD_LIBRARY_PATH=%s' % ':'.join([
               os.getenv('ROCM_PATH','/opt/rocm')+'/lib',
               os.getenv('HCC_HOME','/opt/rocm/hcc')+'/lib',
               os.getenv('HSA_PATH','/opt/rocm/hsa')+'/lib',
               os.getenv('HIP_PATH','/opt/rocm/hip')+'/lib',
               os.getenv('ROCM_PATH','/opt/rocm')+'/libhsakmt/lib',
               os.getenv('ROCM_PATH','/opt/rocm')+'/miopen/lib',
               os.getenv('ROCM_PATH','/opt/rocm')+'/miopengemm/lib',
               os.getenv('ROCM_PATH','/opt/rocm')+'/hipblas/lib',
               os.getenv('ROCM_PATH','/opt/rocm')+'/rocblas/lib',
               "/usr/lib/x86_64-linux-gnu",
               "/usr/local/cuda-10.0/lib64/" # /usr/local/cuda-10.0/lib64/libcudart.so.10.0
               # 上面一行是我自己加的

If not modified, an error will be reported

error while loading shared libraries: libcudart.so.10.0: cannot open shared object file: No such file or directory

2. run

$ build/GCN3_X86/gem5.opt configs/example/apu_se.py -n 3 -c ../gem5-resources/src/gpu/square/bin/square

report error

build/GCN3_X86/sim/syscall_emul.cc:66: fatal: syscall sched_get_priority_max (#146) unimplemented.

, it can be seen that gem5 lacks the implementation of system calls,

very annoying
very
annoying