Overview of MegEngine Dynamic Execution Engine-Imperative Runtime

What we talk about when we talk about MegEngine

As we all know, MegEngine, an open source deep learning framework, is a domestic industrial-grade deep learning framework independently developed by Megvii. It is the core component of Brain++, Megvii's new generation AI productivity platform. It will be officially open sourced to developers around the world in March 2020 . . With its three core advantages of integrated training and reasoning, ultra-low hardware threshold, and efficient reasoning across platforms, MegEngine helps enterprises and developers greatly save the process of products from laboratory prototypes to industrial deployment, and truly realize hour-level transformation capabilities.

MegEngine Contains three levels from top to bottom: Imperative, MegBrain and MegDNN. Their roles are as follows:

Imperative: MegEngine A new set of interfaces designed for dynamic training, responsible for handling dynamic graph runtime ( Imperative Runtime).
MegBrain: Responsible for handling the static graph runtime ( Graph Runtime).
MegDNN: MegEngine The underlying computing engine.

Let's introduce MegEngine the functions of these three modules from bottom to top.

MegEngine's underlying computing engine - MegDNN

MegDNNMegEngine The role played in is the underlying computing engine . MegDNN It is a cross-platform low-level operator library. All calculation operations in the training and inference process need to be MegDNN performed on one operator. An MegDNN operator may have one or more operators according to the scenario (such as tensor size, etc.). Different implementations (also called kernel).

As a cross-platform computing library, MegDNN it provides us with rich and Tensor related computing primitives, such as Convolution, Pooling, MatrixMul, Transpose and so on. Currently MegDNN supported platforms are: x86, arm, CUDA, RoCM, OpenCL, Hexagon etc.

Interested students can see the specific implementation under each platform here .MegDNNkernel

Static graph runtime housekeeper - MegBrain

In order to ensure the consistency of training and inference, the calculation code Imperative is reused , so we need to understand what is done.MegBrainMegBrain

MegBrain Responsible for processing the runtime of static graphs, mainly providing C++ training and inference interfaces.

As can be seen from the overall architecture diagram below MegEngine , a lot of code is reused Imperative through (tensor interpreter). Such as derivation, calculation, derivation, etc.Tensor InterpreterMegBrainshapeTrace Screenshot 2023-06-08 11.57.29.png

In MegBrain , one consists Computing Graph of SymbolVar and many op (operators, operator). SymbolVar It is the representation at MegBrain the level Tensor , which can be understood as op the data passed to the calculation. As an analogy, op it is a calculation operation like addition, subtraction, multiplication and division (commonly used in deep learning convolution, pooling etc.), SymbolVar which is the "number" we use to perform addition, subtraction, multiplication and division (in deep learning Tensor).

MegEngine dynamic graph interface - Imperative debut

Because MegEngine it is a deep learning framework that integrates dynamic and static, MegBrain it solves the problem of training and reasoning of static graphs, and also needs a "component" responsible for processing the training and reasoning of dynamic graphs, as well as the Python training interface on the side, so there is Imperative, that is to say , Imperative Runtime is a set of new interfaces designed separately for dynamic training.

In fact, there are already a lot of operator implementations in , so MegBrain the tensor interpreter is used to reuse more in . The reason for this is:Computing GraphMegEngineImperativeTensor InterpreterMegBrainop

Rewriting operators is expensive and prone to errors.
If Imperative the implementation of and MegBrain is inconsistent, it will easily lead to inconsistency in training and reasoning.

In addition to the reuse MegBrain of some functions, Imperative the modules it contains mainly include: Module(the implementation of the basic class that defines the deep learning network), ( Optimizerthe implementation of some optimizers), ( the training interface Functionalthat provides the layer), (the calculation entry, the bottom layer will call or operator implementation), (dynamic recalculation module), (recording the calculation process of the calculation graph), etc.pythonInterpreterkernelMegBrainDTRTracer

These modules will be introduced in more detail in subsequent articles, and interested students can refer to the official MegEngine documentation .

Summary: Relationship between Imperative and MegDNN / MegBrain

To put it simply, MegDNN it is responsible for the final realization of MegEngine all computing operations in each platform ( CUDA etc.), no matter whether it is MegBrain or Imperative not op, it needs to be called MegDNN kernel to complete the calculation in the end.

Now that MegDNN he has taken over the work of calculation, the work that has nothing to do with calculation in the process of training and reasoning naturally falls to MegBrain He Imperative . These tasks include: derivation, memory allocation, derivation Tensor of pairs shape , graph optimization, compilation, etc.

MegEngine The whole is Runtime composed of two parts and some common components at the bottom. These two parts are called (correspondence ) and (correspondence ) Runtime respectively .Graph RuntimeMegBrainImperative RuntimeImperative

Graph Runtime Responsible for the static graph part, mainly providing C++ training and reasoning interfaces. The implementation that needs to be called during the actual calculation MegDNN .

Imperative Runtime Responsible for the dynamic graph part, mainly providing interfaces for dynamic training Python . Existing implementations that need to be called during actual calculations MegBrain or directly MegDNN called kernel.

Attached:

To get more information about MegEngine, you can: view documents and GitHub projects , or join the MegEngine user communication QQ group: 1029741705. Welcome to contribute to the MegEngine community, become an Awesome MegEngineer , and enjoy endless certificates of honor and customized gifts.