Foreword
In the last test, we follow the official process to use EasyDL quickly realize the face recognition system has a gender-detection function, so today we are going to try Paddlepaddle from scratch by training its own multi-classification model and embedded deployments. Throughout the training process and model: https://aistudio.baidu.com/aistudio/projectDetail/61103 following detailed model training process.
Data set ready
We use CIFAR10 data sets. CIFAR10 dataset contains 60,000 color photographs of 32x32, 10 categories, and each category contains 6,000. 50,000 pictures in which the training set 10000 as a validation set.
! mkdir -p /home/aistudio/.cache/paddle/dataset/cifar # wget to download files stored in the specified folder, and rename the file to download, use -O wget "HTTP:! // AI- atest.bj.bcebos.com/cifar-10-python.tar.gz "-O CIFAR-10-python.tar.gz ! mv CIFAR-10-python.tar.gz /home/aistudio/.cache/paddle/ dataset / cifar /
Model structure
We chose to a three-layer tandem full convolution output connection layer, cats and dogs as the predicted classification, dimensions fixed input and output for the number of classifications
(IMG) DEF convolutional_neural_network:
# First Convolution - pooled layer
conv_pool_1 = fluid.nets.simple_img_conv_pool (
INPUT = IMG, the input image #
filter_size = 5, # size of the filter
num_filters = 20, the number of # filter it. with the same output channel
pool_size = 2, # pool size layer 2 * 2
pool_stride = 2, step # pooled layer
act = "relu") # activation type
# convolution second - layer pooled
conv_pool_2 = fluid. nets.simple_img_conv_pool (
INPUT = conv_pool_1,
filter_size =. 5,
num_filters = 50,
pool_size = 2,
pool_stride = 2,
ACT = "RELU")
# third convolution - cell layer
conv_pool_3 fluid.nets.simple_img_conv_pool = (
INPUT = conv_pool_2,
filter_size =. 5,
num_filters = 50,
pool_size = 2,
pool_stride = 2,
ACT = "RELU")
# as the softmax activation function in the output layer fully connected, a data output class 10 10 digits
Prediction = fluid.layers.fc (INPUT = conv_pool_3, size = 10, ACT = 'SoftMax')
return Prediction
Training & verification
Next on Paddlepaddle fluid, training. The entire training code, see Annex train.py model validation, the use of accessories predict.py code verification and measurement of the running time, choose a dog Figure: dog.jpg (can demo on the fork home link aistudio platform) continuously forecast 10000 times, the output is as follows:
CPU operation results: pretreatment time is .0006270000000085929 predicted time: 16.246494
Out:
im_shape dimensions: (. 1,. 3, 32, 32)
of The Time of Image Process RUN IS
.0006270000000085929
of The RUN Time of Predict IS
16.246494
Results [Array ( [[5.0159363e-04, 3.5942634e-05, 2.5955746e-02, 4.7745958e-02,
9.9251214e-03, 9.0146154e-01, 1.9564393e-03, 1.2230080e-02,
4.7619540e-08, 1.8753216e- 04]], DTYPE = float32)]
Infer Results: Dog
GPU V100 run results: pretreatment time is 0.0006390000000067175 predicted time: 15.903074000000018
Out:
im_shape dimensions: (. 1,. 3, 32, 32)
of The Time of Image Process RUN IS
.0006390000000067175
of The RUN Time of Predict IS
15.903074000000018
Results [Array ([[5.0159392e-04, 3.5942641e-05, 2.5955772e-02, 4.7746032e-02,
9.9251205e-03, 9.0146142e-01, 1.9564414e-03, 1.2230078e-02,
4.7619821e-08, 1.8753250e -04]], DTYPE = float32)]
Infer Results: Dog
You can see, the model can correctly identify the animals in the picture for the dog, then, we will try to deploy this model to Edgeboard above.
Model Export
We need to save the model as a model file model and weights file params, you can use the following API to save Paddle
fluid.io.save_inference_model(model_save_dir,['images'],[predict], exe,params_filename="mlp" + '-params',model_filename="mlp" + '-model',)
As shown, open the file folder where the model, download mlp-model, mlp-params in the left two files of AiStudio.
Deployment model on Edgeboard, complete forecast
1, the new project folder, the directory structure is as follows (in the sample can be modeled resnet, inception routines):
-sample_image_catdog -build -image -include -paddlepaddle-mobile -... -lib -libpaddle-mobile.so -model -mlp -model -params -src -fpga_cv.cpp -main.cpp
2, on the AiStudio guided by the model in the model is placed in the mlp folder, change the name for the model, params
3, New CMakeLists.txt
cmake_minimum_required(VERSION 3.5.1)
project(paddle_edgeboard)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pthread")
add_definitions(-DPADDLE_MOBILE_FPGA_V1)
add_definitions(-DPADDLE_MOBILE_FPGA)
set(PADDLE_LIB_DIR "${PROJECT_SOURCE_DIR}/lib" )
set(EASYDL_INCLUDE_DIR "${PROJECT_SOURCE_DIR}/include" )
set(PADDLE_INCLUDE_DIR "${PROJECT_SOURCE_DIR}/include/paddle-mobile" )
set(APP_NAME "paddle_edgeboard" )
aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/src SRC)
find_package(OpenCV QUIET COMPONENTS core videoio highgui imgproc imgcodecs ml video)
include_directories(SYSTEM ${OpenCV_INCLUDE_DIRS})
#list(APPEND Caffe_LINKER_LIBS ${OpenCV_LIBS})
message(STATUS "OpenCV found (${OpenCV_CONFIG_PATH})，${OpenCV_LIBS}")
#add_definitions(-DUSE_OPENCV)
include_directories(${EASYDL_INCLUDE_DIR})
include_directories(${PADDLE_INCLUDE_DIR})
LINK_DIRECTORIES(${PADDLE_LIB_DIR})
add_executable(${APP_NAME} ${SRC})
target_link_libraries(${APP_NAME} paddle-mobile)
target_link_libraries(${APP_NAME} ${OpenCV_LIBS} )
4、main.cpp
#include
#include "io/paddle_inference_api.h"
#include "math.h"
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include "fpga/KD/float16.hpp"
#include "fpga/KD/llapi/zynqmp_api.h"
using namespace paddle_mobile;
#include
#include
using namespace cv;
cv::Mat sample_float;
static std::vector label_list(10);
void readImage(std::string filename, float* buffer) {
Mat img = imread(filename);
if (img.empty()) {
std::cerr << "Can't read image from the file: " << filename << std::endl;
exit(-1);
}
Mat img2;
resize(img, img2, Size(32,32));
img2.convertTo(sample_float, CV_32FC3);
int index = 0;
for (int row = 0; row < sample_float.rows; ++row) {
float* ptr = (float*)sample_float.ptr(row);
for (int col = 0; col < sample_float.cols; col++) {
float* uc_pixel = ptr;
// uc_pixel[0] -= 102;
// uc_pixel[1] -= 117;
// uc_pixel[1] -= 124;
float r = uc_pixel[0];
float g = uc_pixel[1];
float b = uc_pixel[2];
buffer[index] = b / 255.0;
buffer[index + 1] = g / 255.0;
buffer[index + 2] = r / 255.0;
// sum += a + b + c;
ptr += 3;
// DLOG << "r:" << r << " g:" << g << " b:" << b;
index += 3;
}
}
// return sample_float;
}
PaddleMobileConfig GetConfig() {
PaddleMobileConfig config;
config.precision = PaddleMobileConfig::FP32;
config.device = PaddleMobileConfig::kFPGA;
// config.model_dir = "../models/mobilenet/";
config.prog_file = "../model/mlp/model";
config.param_file = "../model/mlp/params";
config.thread_num = 4;
return config;
}
int main() {
clock_t startTime,endTime;
zynqmp::open_device();
std::cout << " open_device success " << std::endl;
PaddleMobileConfig config = GetConfig();
std::cout << " GetConfig success " << std::endl;
auto predictor =
CreatePaddlePredictor(config);
std::cout << " predictor success " << std::endl;
startTime = clock();//计时开始
float data[1 * 3 * 32 * 32] = {1.0f};
readImage("../image/cat.jpg", data);
endTime = clock();//计时结束
std::cout << "The run time of image process is: " <<(double)(endTime - startTime) / CLOCKS_PER_SEC << "s" << std::endl;
PaddleTensor tensor;
tensor.shape = std::vector({1, 3, 32, 32});
tensor.data = PaddleBuf(data, sizeof(data));
tensor.dtype = PaddleDType::FLOAT32;
std::vector paddle_tensor_feeds(1, tensor);
PaddleTensor tensor_out;
tensor_out.shape = std::vector({});
tensor_out.data = PaddleBuf();
tensor_out.dtype = PaddleDType::FLOAT32;
std::vector outputs(1, tensor_out);
std::cout << " before predict " << std::endl;
predictor->Run(paddle_tensor_feeds, &outputs);
std::cout << " after predict " << std::endl;
// assert();
endTime = clock();//计时结束
std::cout << "The run time of predict is: " <<(double)(endTime - startTime) / CLOCKS_PER_SEC << "s" << std::endl;
float* data_o = static_cast(outputs[0].data.data());
for (size_t j = 0; j < outputs[0].data.length() / sizeof(float); ++j) {
std::cout << "output[" << j << "]: " << data_o[j] << std::endl;
}
int index = 0;
float max = 0.0;
for (int i = 0;i < 10; i++) {
float val = data_o[i];
if (val > max) {
max = val > max ? val : max;
index = i;
}
}
label_list = {"airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse",
"ship", "truck" };
std::cout << "Result" << " is " << label_list[index] << std::endl;
return 0;
}
5, compile and run
insmod /home/root/workspace/driver/fpgadrv.ko cd /home/root/workspace/sample/sample_image_catdog mkdir build cd build rm -rf * cmake .. make ./paddle_edgeboard
Modify the main image file you want to predict:
6, after the main file modified prediction is repeatedly performed, the results can be obtained as follows: the image processing time is approximately: .006 seconds, approximately prediction time: 0.008 seconds
to sum up
advantage:
1, EdgeBoard built Paddle-Mobile, the model can be trained with Paddle better butt.
2, the predicted speed: Edge when forecasting a small model, capable of dual-core CPU and GPU in an order of magnitude estimate of the smaller model, batch size is also 1, gpu, cpu performance benefits outweighed the cost of communication, follow-up will large model, high batch size tests.
3, demo provides simple enough, modify them difficulty is very low.
insufficient:
Paddle-Mobile related documents with a certain threshold, and more dispersed. When used for the first time will take some detours problems, it tends to be a black box and difficult to locate. In the attempt to train the model, the situation appeared once op is not supported, we did not even find a list of supported op on the official website, with the support of the development of this man after the update fixes. If follow-up can be used in a stable firmware version, and there are relatively easy to use sdk, development threshold may be further reduced.
Author: Litchll