ncnn source code reading (5)----operator registration mechanism

Operator registration mechanism

The registration of operators in ncnn is completed during the compilation period. It is mainly learned from two aspects:

• unified abstract interface
• Operator instance creation
• Implement operator registration during compilation

unified abstract interface

The three major characteristics of object-oriented: encapsulation, inheritance and polymorphism.
All operators inherit from the unified Layer class and then implement their functions individually. When used, the object pointer of the operator is converted to the pointer of the parent class for calling. The polymorphic feature is used here.
Concrete code example:
abstract interface of base class:

class Layer
{
public:
    // empty
    Layer();
    // virtual destructor
    virtual ~Layer();

#if NCNN_STDIO
#if NCNN_STRING
    // load layer specific parameter from plain param file
    // return 0 if success
    virtual int load_param(FILE* paramfp);
#endif // NCNN_STRING
    // load layer specific parameter from binary param file
    // return 0 if success
    virtual int load_param_bin(FILE* paramfp);

    // load layer specific weight data from model file
    // return 0 if success
    virtual int load_model(FILE* binfp);
#endif // NCNN_STDIO

    // load layer specific parameter from memory
    // memory pointer is 32-bit aligned
    // return 0 if success
    virtual int load_param(const unsigned char*& mem);

    // load layer specific weight data from memory
    // memory pointer is 32-bit aligned
    // return 0 if success
    virtual int load_model(const unsigned char*& mem);

public:
    // one input and one output blob
    bool one_blob_only;

    // support inplace inference
    bool support_inplace;

public:
    // implement inference
    // return 0 if success
    virtual int forward(const std::vector<Mat>& bottom_blobs, std::vector<Mat>& top_blobs) const;
    virtual int forward(const Mat& bottom_blob, Mat& top_blob) const;

    // implement inplace inference
    // return 0 if success
    virtual int forward_inplace(std::vector<Mat>& bottom_top_blobs) const;
    virtual int forward_inplace(Mat& bottom_top_blob) const;

public:
#if NCNN_STRING
    // layer type name
    std::string type;
    // layer name
    std::string name;
#endif // NCNN_STRING
    // blob index which this layer needs as input
    std::vector<int> bottoms;
    // blob index which this layer produces as output
    std::vector<int> tops;
};

Here are examples of several subclasses:
Convolution operators

class Convolution : public Layer
{
public:
    Convolution();
    virtual ~Convolution();

#if NCNN_STDIO
#if NCNN_STRING
    virtual int load_param(FILE* paramfp);
#endif // NCNN_STRING
    virtual int load_param_bin(FILE* paramfp);
    virtual int load_model(FILE* binfp);
#endif // NCNN_STDIO
    virtual int load_param(const unsigned char*& mem);
    virtual int load_model(const unsigned char*& mem);

    virtual int forward(const Mat& bottom_blobs, Mat& top_blobs) const;

public:
    // param
    int num_output;
    int kernel_size;
    int dilation;
    int stride;
    int pad;
    int bias_term;

    int weight_data_size;

    // model
    Mat weight_data;
    Mat bias_data;
};

crop operator:

class Crop : public Layer
{
public:
    Crop();

#if NCNN_STDIO
#if NCNN_STRING
    virtual int load_param(FILE* paramfp);
#endif // NCNN_STRING
    virtual int load_param_bin(FILE* paramfp);
#endif // NCNN_STDIO
    virtual int load_param(const unsigned char*& mem);

    virtual int forward(const std::vector<Mat>& bottom_blobs, std::vector<Mat>& top_blobs) const;

public:
    int woffset;
    int hoffset;
};

Operator instance creation

To create an operator instance, in the cpp implemented by each operator, define the corresponding function and return it as a base class pointer, for example, in the following form:

Layer* 算子名_layer_creator() {
	return new 算子类;
}

For convenience, macro definitions are used in the header file of the operator base class to replace the above functions:

#define DEFINE_LAYER_CREATOR(name) \
	Layer* name##_layer_creator() { return new name;}

Using the above macro definition, in each operator:

DEFINE_LAYER_CREATOR(Convolution);

It is equivalent to defining the following method in cpp implemented by the Convolution operator:

Layer* Convolution_layer_creator() {
	return new Convolution;
}

Other operators are similar to the Convolution operator mentioned above.
Then according to the existence of the above function in each operator, the name of each operator can be associated with the function pointer of the function in the corresponding operator.
So define the function pointer type of the above function:

typedef Layer* (*layer_creator_func)();

The layer_creator_func above is a function pointer type, and the pointer of any operator function can be assigned to a variable defined by this type.

layer_creator_func creator;
creator = Convolution_layer_creator;

Then use a structure to associate the name with the above function pointer and name

struct layer_registry_entry
{
#if NCNN_STRING
    // layer type name
    const char* name;
#endif // NCNN_STRING
    // layer factory entry
    layer_creator_func creator;
};

According to the above structure, the corresponding operator instance can be created according to the name:

layer_registry_entry conv;
conv.name = "convolution";
conv.creator = Convolution_layer_creator;

Layer* layer = nullptr;
if (conv.name == "convolution") {
	layer_creator_func create = conv.creator;
	layer = create();
}

Implement operator registration during compilation

According to the above, it can be seen that in the cpp file implemented by each operator, there is a function as follows:

Layer* 算子类名_layer_creator() {
	return new 算子类;
}

If you want to use this function externally, you need to use extern to declare the external function:

extern Layer* 算子类名_layer_creator();

After declaring an external function, according to the function pointer type defined above, you can use the function name to assign it:

layer_creator_func creator = 算子类名_layer_creator;

According to the above usage method, the structure can be assigned values:

//声明外部函数
extern Layer* 算子类名_layer_creator();
layer_registry_entry conv;
conv.name = "convolution";
conv.creator = 算子类名_layer_creator;

The assignment to the structure in the above code is achieved by assigning a separate value to each member. It can also be implemented through construction:

extern Layer* 算子类名_layer_creator();
layer_registry_entry conv{"convolution", 算子类名_layer_creator};

As can be seen from the above, as long as we assign each operator to the above-mentioned structure, the previous assignments to the structure were all implemented in the code, that is, the assignment will be successful during runtime.
There is a new way to assign values ​​to structures. Here is an example:

struct Test {
	string name;
	int age;
	float weight;
};

The above is the specific definition of the structure. There is a header file below:
test.h

{"nihao",12,12.3},
{"hello",13,23.4},

Combining the above content, you can have the following usage:

static const Test t[] = {
	#include "test.h"
};
static const int t_count = sizeof(t)/ sizeof(Test);

Through the above code, you can assign a value to the array of the structure through a header file, and get the length of the array through sizeof.

In ncnn, the structure array is initialized through the header file, and the header file is generated during compilation and is not added in advance.
For automatic generation of header files and containing corresponding content, this can be done in CMakeLists.txt, in ncnn

macro(ncnn_add_layer class)
   if(WITH_LAYER_${name})
        file(APPEND ${CMAKE_CURRENT_BINARY_DIR}/layer_declaration.h
            "extern Layer* ${class}_x86_layer_creator();\n")
        file(APPEND ${CMAKE_CURRENT_BINARY_DIR}/layer_registry.h
            "#if NCNN_STRING\n{\"${class}\",${class}_x86_layer_creator},\n#else\n{${class}_x86_layer_creator},\n#endif\n")
    endif()
endmacro()

ncnn_add_layer(Convolution)
ncnn_add_layer(Crop)

In the above cmake code, two header files will be generated. layer_declaration.hThe content of the external function declaration is added to the file; layer_registry.hthe file contains the content of the initialization structure array.
The method used in the code is as follows:

//引入外部函数的声明
#include "layer_declaration.h"

static const layer_registry_entry layer_registry[] =
{
#include "layer_registry.h"
};

static const int layer_registry_entry_count = sizeof(layer_registry) / sizeof(layer_registry_entry);

After the above compilation is completed, layer_registryall operator names and creators are stored in the variables.
In the file where the operator base class is located, the operator function interface can be provided. In this way, all operator registration sets need to be exposed to the outside world.
The implementation in ncnn is as follows

Layer* create_layer(int index)
{
    if (index < 0 || index >= layer_registry_entry_count)
    {
        fprintf(stderr, "layer index %d not exists\n", index);
        return 0;
    }

    layer_creator_func layer_creator = layer_registry[index].creator;
    if (!layer_creator)
    {
        fprintf(stderr, "layer index %d not enabled\n", index);
        return 0;
    }

    return layer_creator();
}

It can also be created through the operator name, just reload create_layer.