2.1.cuda driver API - overview

foreword

Teacher Du launched the tensorRT high-performance deployment course from scratch . I have read it before, but I didn’t take notes, and I forgot many things. Play it again this time, and take notes by the way

This course learns the simplified CUDA tutorial-Driver API overview

The course outline can be seen in the mind map below

1. Driver API overview

For Driver API you need to know:

1. CUDA Driver is the driver-level underlying API that communicates with the GPU
2. The understanding of Driver API is conducive to understanding the subsequent Runtime API
3. CUDA Driver is released with the graphics card driver and needs to be viewed separately from cudatoolkit
4. CUDA Driver corresponds to cuda.h and libcuda.so
5. The main knowledge points of Driver API are the management mechanism of Context and the development habit of CUDA series interface (error checking method), as well as the memory model

Reference: What is graphics card, graphics card driver, nvcc, cuda driver, cudatoolkit, cudnn?

The following figure clearly shows the relationship between Driver API and Runtime API and other concepts

Figure 1-1 Relationship between Driver API and Runtime API

It can be seen from the figure that we usually nvidia-smicall the Driver API. At first, the Driver API was the underlying API for communicating with the graphics card, but people found that the Driver API was too low-level, so the Runtime API was introduced. It can be seen from the figure that the Runtime API is developed based on the Driver API. The cudaMalloc(), cudaMemset(), and that we see in our daily life cudaMemcpy()belong to the Runtime API. Things like cuat the beginning of cuCtxCreate()belong to the Driver API.

It is worth noting that cuda.hit is part of the NVIDIA CUDA Toolkit. CUDA Toolkit is a software development kit provided by NVIDIA for developing GPU-accelerated applications, which contains various libraries and header files for compiling and executing CUDA programs. It libcuda.sois a shared library file that is installed when the NVIDIA graphics card driver is installed in the system.

cuda.hProvides the interface and statement of CUDA programming, while libcuda.sois the runtime library, which provides the underlying functions and support required by CUDA runtime.

Why do you need to understand the Driver API? Isn't it enough to just look at the more advanced Runtime API?

1. Driver API is the key to understanding context in cudaRuntime
2. Driver API only needs to understand the context and know its existence
3. During the development process, it may be difficult to debug the upper layer due to insufficient understanding of the Driver API
4. For the lower-level api, do some understanding, which will help the high-level to troubleshoot the cause after encountering problems

There are two types of context :

1. Manually managed context, cuCtxCreate()(manually managed, push/pop in stack mode)
2. Automatically managed context, cuDevicePrimaryCtxRetain(automatically managed, runtime api is based on this)

Regarding memory, there are two broad categories :

1. CPU memory, called Host Memory. It can be divided into Pageable Memory: pageable memory + Page-Locked Memory: page-locked memory
2. GPU memory, called Device Memory. It can be divided into Global Memory: global memory + Shared Memory: shared memory + other kinds of memory

Figure 1-2 Memory data flow direction

It is enough to have a basic concept about context and memory, which will be explained in detail later.

2. Supplementary knowledge

We need to understand the basic concepts of cuda and graphics card and the knowledge of initializing cuda: ( from Mr. Du )

1. how to run

• make run

2. What are graphics cards, graphics card drivers, nvcc, cuda driver, cudatoolkit, cudnn?

• Regarding the version matching between the graphics card driver and the cuda driver
  • Table 1. CUDA 11.6 Update 1 Component Versions
  • Conclusion: Try to upgrade the graphics card driver to a new one, so the graphics card driver is backward compatible with the cuda driver
• Simple understanding of graphics card related concepts
  • Graphics card: GPU
  • Graphics card driver: driver software, analog sound card driver, camera driver
  • GPU architecture: The gpu architecture refers to the way the hardware is designed, such as whether there is an L1 or L2 buffer
  • CUDA: One of the understandings is that it is a programming language (to C++, Python, etc., but it is specially used to control the GPU)
  • cuDNN: This is actually a software library specially designed for deep learning computing, which provides many specialized computing functions
  • CUDAToolkit: This is the toolkit we really need to install first. The so-called cuda first refers to it
  • It contains many libraries, such as: cudart, cublas, etc.
  • Other related knowledge includes nvcc and nvidia-smi, switching between multiple cuda versions, cuda installation, etc.
  • For details, please refer to: https://zhuanlan.zhihu.com/p/91334380

3. cuda-driver-api gives cuda-runtime-api

• CUDA Driver is more low-level than CUDA Runtime, which means that the Driver API has more flexible control and is accompanied by more complex programming
• Therefore, CUDA Driver needs to be explicitly initialized cuInit(0), otherwise other APIs will returnCUDA_ERROR_NOT_INITIALIZED
• Information about initialized drivers and graphics cards can be easily obtained:
  • Driver version management https://docs.nvidia.com/cuda/archive/11.2.0/cuda-driver-api/group__CUDA__VERSION.html#group__CUDA__VERSION
  • Device information management https://docs.nvidia.com/cuda/archive/11.2.0/cuda-driver-api/group__CUDA__DEVICE.html

4. written at the end

• CUDA's online documentation address
  • https://developer.nvidia.com/cuda-toolkit-archive
  • https://docs.nvidia.com/cuda/archive/11.2.0/
• report error
  • Prompt nvcc error. For gcc version greater than or equal to 8 does not support, you need to modify the g++ in the Makefile to g++7 or lower

Summarize

This course mainly understands some concepts of CUDA Driver API, which is a low-level API that provides low-level access to GPU hardware and drivers. Learning about the Driver API can better help us understand the Runtime API in the future. In the future, we will mainly learn the use of the Driver API from three aspects: the Context management mechanism, the development habits of the CUDA series interface, and the memory model.