Tian Haili @ CSDN 2020-10-26

The data format of the convolution weight (Weight/Kernel/Filter) is generally different from Tensor or Activation data, and has its specific arrangement. But this goes deep into the internal mechanism of the platform (training and/or inference), and the general public will not care about it, so this part of information is scarce. This article gives some guidelines, you can use the analysis method in " Graphic NCHW and NHWC Data Format " to understand it in practice.

1. Basic concepts

There are the following dimensions in the weight/convolution kernel/filter (Weight/Kernel/Filter):

I-Input Channels
H-Filter Height Convolution kernel height
W-Filter Width Convolution kernel width
O-Output Channels

Two, commonly used weight format

2.1 TensorFlow

The weight format of TensorFlow's convolution conv2d is HWIO, which is defined in filters in tf.nn.conv2d:

The format of weight in TF is HWIO [Kernel Height, Kernel Width, Input Channels, Output Channels]

2.2 TensorFlow Lite

The format of conv2d convolution weights in TFLite is OHWI [Output Channels, Kernel Height, Kernel Width, Input Channels].

In the process of converting from the TensorFlow model to the TFLite model, the weight will also be converted, which is actually the Transpose operation of Tensor.

This part is done by TFLite Converter (TOCO). Because it is done before Runtime is executed, there will be no online conversion, and this process will not affect performance.

2.3 Other

Because TVM needs to use the model format supported by mainstream ML frameworks to do frontend support, TVM has many considerations in this regard, and there are also many meaningful discussions.

You can refer to:

For the NCHW data format of TVM, Kernel uses OIHW; for NHWC data format, Kernel uses HWIO.

and so,

For TensorFlow, the data is in NHWC format, Kernel is in HWIO format, and TVM is fully supported;
For TFLite, the data is also NHWC, but Kernle is OHWI, and Kernel needs to be converted to HWIO;
For ACL (Arm Compute Library), as a Codegen, the situation is more complicated, so I won't expand it here.

3. Other topics

1. The above description is generally for conv2d convolution kernel, depthwise convolution will be different.

2. When format conversion is involved, this refers to the Transpose operation. Transpose will be time-consuming and should be done in the model conversion (or compilation) process before Runtime is executed, so as not to affect the runtime performance.

3. For the visual display of HWIO and OHWI, please refer to " Illustration of NCHW and NHWC Data Formats " and " nChw8c except NCHW and NHWC Data Formats " to draw 3-D diagrams, and then expand one-dimensional to 4-D.

Convolution weight (Weight/Kernel/Filter) data format adopts HWIO/OHWI, or other...