HUAWEI CLOUD GPU server uses PaddleServing to deploy PaddleClas multiple self-trained recognition model services

foreword

Recently, the company needs to identify different truck brands and vehicle series in the pictures. After model training through PaddleClas, a brand recognition model and a vehicle series recognition model are obtained. Now, the two models are deployed on a HUAWEI CLOUD GPU server. To deploy multiple models at the same time, the only way to deploy multiple models is to use the Pipeline service in PaddleServing or the C++ serving service. Since C++ serving needs to compile source code, it is cumbersome, so the following uses the Pipeline method to deploy multiple models in series.

How to build the GPU version of PaddlePaddle environment on Huawei cloud server, please refer to the following article: https://blog.csdn.net/loutengyuan/article/details/126527326

Environmental preparation

The operating environment of PaddleClas and the operating environment of Paddle Serving need to be prepared.

• Prepare the operating environment link of PaddleClas

# 克隆代码
git clone https://github.com/PaddlePaddle/PaddleClas

• Install the operating environment of PaddleServing, the steps are as follows

# 安装serving，用于启动服务
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl
pip3 install paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl

# 安装client，用于向服务发送请求
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.8.3-cp38-none-any.whl
pip3 install paddle_serving_client-0.8.3-cp38-none-any.whl

# 安装serving-app
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.8.3-py3-none-any.whl
pip3 install paddle_serving_app-0.8.3-py3-none-any.whl

Service Data Preparation

When using PaddleServing for image recognition service deployment, it is necessary to convert multiple saved inference models into Serving models.

model conversion

Enter the working directory:

cd PaddleClas/deploy/

Create and enter the models folder:

# 创建并进入models文件夹
mkdir models
cd models

Put the trained inference model into this folder, including detection model (picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer), brand recognition model (rec_brands_v1.0_infer) and car series recognition model (rec_series_v1.0_infer) structure as follows:

├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer
│   ├── infer_cfg.yml
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel
├── rec_brands_v1.0_infer
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel
└── rec_series_v1.0_infer
    ├── inference.pdiparams
    ├── inference.pdiparams.info
    ├── inference.pdmodel
    └── readme.txt

Convert the generic detection inference model to the Serving model:

# 转换通用检测模型
python3.8 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
--model_filename inference.pdmodel  \
--params_filename inference.pdiparams \
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/

After the conversion of the general detection inference model is completed, there will be additional folders of picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ and picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/ in the current folder, with the following structure:

    ├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/
    │   ├── inference.pdiparams
    │   ├── inference.pdmodel
    │   ├── serving_server_conf.prototxt
    │   └── serving_server_conf.stream.prototxt
    │
    └── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
          ├── serving_client_conf.prototxt
          └── serving_client_conf.stream.prototxt

Convert brand identity inference model to Serving model:

  # 转换品牌识别模型
  python3.8 -m paddle_serving_client.convert \
  --dirname ./rec_brands_v1.0_infer/ \
  --model_filename inference.pdmodel  \
  --params_filename inference.pdiparams \
  --serving_server ./rec_brands_v1.0_serving/ \
  --serving_client ./rec_brands_v1.0_client/

After the conversion of the brand recognition inference model is completed, there will be additional folders rec_brands_v1.0_serving/ and rec_brands_v1.0_client/ in the current folder, with the following structure:

    ├── rec_brands_v1.0_serving/
    │   ├── inference.pdiparams
    │   ├── inference.pdmodel
    │   ├── serving_server_conf.prototxt
    │   └── serving_server_conf.stream.prototxt
    │
    └── rec_brands_v1.0_client/
          ├── serving_client_conf.prototxt
          └── serving_client_conf.stream.prototxt

Respectively modify the name of in rec_brands_v1.0_serving/and rec_brands_v1.0_client/in the directory : change in to . The modified content is as follows:serving_server_conf.prototxtaliasfetch_varalias_namefeaturesserving_server_conf.prototxt

feed_var {
	name: "x"
	alias_name: "x"
	is_lod_tensor: false
	feed_type: 1
	shape: 3
	shape: 224
	shape: 224
}
fetch_var {
	name: "save_infer_model/scale_0.tmp_1"
	alias_name: "features"
	is_lod_tensor: false
	fetch_type: 1
	shape: 512
}

Convert the vehicle identification inference model to the Serving model:

  # 转换车系识别模型
  python3.8 -m paddle_serving_client.convert \
  --dirname ./rec_series_v1.0_infer/ \
  --model_filename inference.pdmodel  \
  --params_filename inference.pdiparams \
  --serving_server ./rec_series_v1.0_serving/ \
  --serving_client ./rec_series_v1.0_client/

After the car series recognition inference model conversion is completed, there will be additional rec_series_v1.0_serving/ and rec_series_v1.0_client/ folders in the current folder, with the following structure:

    ├── rec_series_v1.0_serving/
    │   ├── inference.pdiparams
    │   ├── inference.pdmodel
    │   ├── serving_server_conf.prototxt
    │   └── serving_server_conf.stream.prototxt
    │
    └── rec_series_v1.0_client/
          ├── serving_client_conf.prototxt
          └── serving_client_conf.stream.prototxt

Respectively modify the name of in rec_series_v1.0_serving/and rec_series_v1.0_client/in the directory : change in to . The modified content is as follows:serving_server_conf.prototxtaliasfetch_varalias_namefeaturesserving_server_conf.prototxt

feed_var {
	name: "x"
	alias_name: "x"
	is_lod_tensor: false
	feed_type: 1
	shape: 3
	shape: 224
	shape: 224
}
fetch_var {
	name: "save_infer_model/scale_0.tmp_1"
	alias_name: "features"
	is_lod_tensor: false
	fetch_type: 1
	shape: 512
}

The specific meanings of the parameters in the above commands are shown in the table below:

parameter	type	Defaults	describe
dirname	str	-	The storage path of the model file that needs to be converted, and the program structure file and parameter file are all saved in this directory.
model_filename	str	None	The name of the file that stores the Inference Program structure of the model that needs to be converted. If set to None, use __model__as default filename
params_filename	str	None	The name of the file where all parameters of the model to be converted are stored. It needs to be specified if and only if all model parameters are stored in a single binary file. If model parameters are stored in separate files, set its value to None
serving_server	str	"serving_server"	The storage path of converted model files and configuration files. The default is serving_server
serving_client	str	"serving_client"	The converted client configuration file storage path. The default is serving_client

Index library added

Put the brand and car series library into the upper level (deploy) directory

    # 回到deploy目录
    cd ../

The directory structure is as follows:

    ├── brand_dataset_v1.0/
    │   └── index
    │       ├── id_map.pkl
    │       └── vector.index
    └── series_dataset_v1.0/
        └── index
            ├── id_map.pkl
            └── vector.index

service deployment

Note: The identification service involves multiple models, and the PipeLine deployment method is adopted for performance considerations. The Pipeline deployment method currently does not support the windows platform.
into the working directory

  cd ./deploy/paddleserving/recognition

The paddleserving directory contains the code to start the Python Pipeline service, the C++ Serving service and send prediction requests, including:

  __init__.py
  config.yml                  # 启动python pipeline服务的配置文件
  pipeline_http_client.py     # http方式发送pipeline预测请求的脚本
  pipeline_rpc_client.py      # rpc方式发送pipeline预测请求的脚本
  recognition_web_service.py  # 启动pipeline服务端的脚本
  readme.md                   # 识别模型服务化部署文档
  run_cpp_serving.sh          # 启动C++ Pipeline Serving部署的脚本
  test_cpp_serving_client.py  # rpc方式发送C++ Pipeline serving预测请求的脚本

Modify the config.yml file as follows:

#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程，每个进程内构建grpcSever和DAG
##当build_dag_each_worker=False时，框架会设置主线程grpc线程池的max_workers=worker_num
worker_num: 1

#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时，不自动生成http_port
http_port: 8899
#rpc_port: 9994

dag:
    #op资源类型, True, 为线程模型；False，为进程模型
    is_thread_op: False
op:
    rec_brands:
        #并发数，is_thread_op=True时，为线程并发；否则为进程并发
        concurrency: 1

        #当op配置没有server_endpoints时，从local_service_conf读取本地服务配置
        local_service_conf:

            #uci模型路径
            model_config: ../../models/rec_brands_v1.0_serving

            #计算硬件类型: 空缺时由devices决定(CPU/GPU)，0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
            device_type: 1

            #计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡
            devices: "0" # "0,1"

            #client类型，包括brpc, grpc和local_predictor.local_predictor不启动Serving服务，进程内预测
            client_type: local_predictor

            #Fetch结果列表，以client_config中fetch_var的alias_name为准
            fetch_list: ["features"]

    rec_series:
        #并发数，is_thread_op=True时，为线程并发；否则为进程并发
        concurrency: 1

        #当op配置没有server_endpoints时，从local_service_conf读取本地服务配置
        local_service_conf:

            #uci模型路径
            model_config: ../../models/rec_series_v1.0_serving

            #计算硬件类型: 空缺时由devices决定(CPU/GPU)，0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
            device_type: 1

            #计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡
            devices: "0" # "0,1"

            #client类型，包括brpc, grpc和local_predictor.local_predictor不启动Serving服务，进程内预测
            client_type: local_predictor

            #Fetch结果列表，以client_config中fetch_var的alias_name为准
            fetch_list: ["features"]

    det:
        concurrency: 1
        local_service_conf:
            client_type: local_predictor
            device_type: 1
            devices: '0'
            fetch_list:
            - save_infer_model/scale_0.tmp_1
            model_config: ../../models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/

Modify the recognition_web_service.py file as follows:

# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime

from paddle_serving_server.web_service import WebService, Op
from paddle_serving_server.pipeline import RequestOp, ResponseOp
from paddle_serving_server.pipeline import PipelineServer
from paddle_serving_server.pipeline.proto import pipeline_service_pb2
from paddle_serving_server.pipeline.channel import ChannelDataErrcode
import logging
import numpy as np
import sys
import cv2
from paddle_serving_app.reader import *
import base64
import os
import faiss
import pickle
import json


class TestRequestOp(RequestOp):
    def init_op(self):
        pass

    def unpack_request_package(self, request):
        # print(str(request.method))
        dict_data = {
    
    }
        log_id = None
        if request is None:
            raise ValueError("request is None")
        for idx, key in enumerate(request.key):
            dict_data[key] = request.value[idx]
        log_id = request.logid
        return dict_data, log_id, None, ""


class DetOp(Op):
    def init_op(self):
        self.img_preprocess = Sequential([
            BGR2RGB(), Div(255.0),
            Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225], False),
            Resize((640, 640)), Transpose((2, 0, 1))
        ])

        self.img_postprocess = RCNNPostprocess("label_list.txt", "output")
        self.threshold = 0.2
        self.max_det_results = 5

    def generate_scale(self, im):
        """
        Args:
            im (np.ndarray): image (np.ndarray)
        Returns:
            im_scale_x: the resize ratio of X
            im_scale_y: the resize ratio of Y
        """
        target_size = [640, 640]
        origin_shape = im.shape[:2]
        resize_h, resize_w = target_size
        im_scale_y = resize_h / float(origin_shape[0])
        im_scale_x = resize_w / float(origin_shape[1])
        return im_scale_y, im_scale_x

    def preprocess(self, input_dicts, data_id, log_id):
        print("{} detect begin --> data_id: {}".format(datetime.datetime.now(), data_id))
        (_, input_dict), = input_dicts.items()
        imgs = []
        raw_imgs = []
        for key in input_dict.keys():
            data = base64.b64decode(input_dict[key].encode('utf8'))
            raw_imgs.append(data)
            data = np.fromstring(data, np.uint8)
            raw_im = cv2.imdecode(data, cv2.IMREAD_COLOR)

            im_scale_y, im_scale_x = self.generate_scale(raw_im)
            im = self.img_preprocess(raw_im)

            im_shape = np.array(im.shape[1:]).reshape(-1)
            scale_factor = np.array([im_scale_y, im_scale_x]).reshape(-1)
            imgs.append({
    
    
                "image": im[np.newaxis, :],
                "im_shape": im_shape[np.newaxis, :],
                "scale_factor": scale_factor[np.newaxis, :],
            })
        self.raw_img = raw_imgs

        feed_dict = {
    
    
            "image": np.concatenate(
                [x["image"] for x in imgs], axis=0),
            "im_shape": np.concatenate(
                [x["im_shape"] for x in imgs], axis=0),
            "scale_factor": np.concatenate(
                [x["scale_factor"] for x in imgs], axis=0)
        }
        return feed_dict, False, None, ""

    def postprocess(self, input_dicts, fetch_dict, data_id, log_id):
        boxes = self.img_postprocess(fetch_dict, visualize=False)
        boxes.sort(key=lambda x: x["score"], reverse=True)
        boxes = filter(lambda x: x["score"] >= self.threshold,
                       boxes[:self.max_det_results])
        boxes = list(boxes)
        for i in range(len(boxes)):
            boxes[i]["bbox"][2] += boxes[i]["bbox"][0] - 1
            boxes[i]["bbox"][3] += boxes[i]["bbox"][1] - 1
        result = json.dumps(boxes)
        res_dict = {
    
    "bbox_result": result, "image": self.raw_img}
        print("{} detect finish --> data_id: {}".format(datetime.datetime.now(), data_id))
        return res_dict, None, ""


class BrandsRecOp(Op):
    def init_op(self):
        self.seq = Sequential([
            BGR2RGB(), Resize((224, 224)), Div(255),
            Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
                      False), Transpose((2, 0, 1))
        ])

        index_dir = "../../brand_dataset_v1.0/index"
        assert os.path.exists(os.path.join(
            index_dir, "vector.index")), "vector.index not found ..."
        assert os.path.exists(os.path.join(
            index_dir, "id_map.pkl")), "id_map.pkl not found ... "

        self.searcher = faiss.read_index(
            os.path.join(index_dir, "vector.index"))

        with open(os.path.join(index_dir, "id_map.pkl"), "rb") as fd:
            self.id_map = pickle.load(fd)

        self.rec_nms_thresold = 0.05
        self.rec_score_thres = 0.5
        self.feature_normalize = True
        self.return_k = 1
        self.area_ratio_thresold=0.1

    def preprocess(self, input_dicts, data_id, log_id):
        (_, input_dict), = input_dicts.items()
        raw_img = input_dict["image"][0]
        data = np.frombuffer(raw_img, np.uint8)
        origin_img = cv2.imdecode(data, cv2.IMREAD_COLOR)
        dt_boxes = input_dict["bbox_result"]
        boxes = json.loads(dt_boxes)
        boxes.append({
    
    
            "category_id": 0,
            "score": 1.0,
            "bbox": [0, 0, origin_img.shape[1], origin_img.shape[0]]
        })
        self.det_boxes = boxes

        #construct batch images for rec
        imgs = []
        for box in boxes:
            box = [int(x) for x in box["bbox"]]
            im = origin_img[box[1]:box[3], box[0]:box[2]].copy()
            img = self.seq(im)
            imgs.append(img[np.newaxis, :].copy())

        input_imgs = np.concatenate(imgs, axis=0)
        return {
    
    "x": input_imgs}, False, None, ""

    def nms_to_rec_results(self, results, thresh=0.1):
        filtered_results = []
        x1 = np.array([r["bbox"][0] for r in results]).astype("float32")
        y1 = np.array([r["bbox"][1] for r in results]).astype("float32")
        x2 = np.array([r["bbox"][2] for r in results]).astype("float32")
        y2 = np.array([r["bbox"][3] for r in results]).astype("float32")
        scores = np.array([r["rec_scores"] for r in results])

        areas = (x2 - x1 + 1) * (y2 - y1 + 1)
        order = scores.argsort()[::-1]
        while order.size > 0:
            i = order[0]
            xx1 = np.maximum(x1[i], x1[order[1:]])
            yy1 = np.maximum(y1[i], y1[order[1:]])
            xx2 = np.minimum(x2[i], x2[order[1:]])
            yy2 = np.minimum(y2[i], y2[order[1:]])

            w = np.maximum(0.0, xx2 - xx1 + 1)
            h = np.maximum(0.0, yy2 - yy1 + 1)
            inter = w * h
            ovr = inter / (areas[i] + areas[order[1:]] - inter)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
            filtered_results.append(results[i])
        return filtered_results

    def check_boxes(self, results, area_ratio_thresh=0.1):
        filtered_results = []
        for result in results:
            if result["area_ratio"]>=area_ratio_thresh:
                filtered_results.append(result)
        if len(filtered_results)>0:
            return filtered_results
        else:
            return results

    def postprocess(self, input_dicts, fetch_dict, data_id, log_id):
        batch_features = fetch_dict["features"]

        if self.feature_normalize:
            feas_norm = np.sqrt(
                np.sum(np.square(batch_features), axis=1, keepdims=True))
            batch_features = np.divide(batch_features, feas_norm)

        scores, docs = self.searcher.search(batch_features, self.return_k)
        origin_img_box = self.det_boxes[len(self.det_boxes) - 1]["bbox"]
        total_pixes = origin_img_box[2] * origin_img_box[3]
        results = []
        for i in range(scores.shape[0]):
            pred = {
    
    }
            xmin, ymin, xmax, ymax = self.det_boxes[i]["bbox"]
            area_pix = (xmax - xmin) * (ymax - ymin)
            ratio = 0.0
            if total_pixes > 0:
                ratio = area_pix * 1.0 / total_pixes
            if scores[i][0] >= self.rec_score_thres:
                pred["bbox"] = [int(x) for x in self.det_boxes[i]["bbox"]]
                pred["rec_docs"] = self.id_map[docs[i][0]].split()[1]
                pred["rec_scores"] = scores[i][0]
                pred["area_ratio"] = round(ratio, 4)
                results.append(pred)

        #do nms
        results = self.nms_to_rec_results(results, self.rec_nms_thresold)
        print("{} BrandsRecOp data_id: {} --> Nms Result: {}".format(datetime.datetime.now(), data_id, results))
        results = self.check_boxes(results, self.area_ratio_thresold)
        print("{} BrandsRecOp data_id: {} --> Out Result: {}".format(datetime.datetime.now(), data_id, results))
        return {
    
    "result": str(results)}, None, ""


class SeriesRecOp(Op):
    def init_op(self):
        self.seq = Sequential([
            BGR2RGB(), Resize((224, 224)), Div(255),
            Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
                      False), Transpose((2, 0, 1))
        ])

        index_dir = "../../series_dataset_v1.0/index"
        assert os.path.exists(os.path.join(
            index_dir, "vector.index")), "vector.index not found ..."
        assert os.path.exists(os.path.join(
            index_dir, "id_map.pkl")), "id_map.pkl not found ... "

        self.searcher = faiss.read_index(
            os.path.join(index_dir, "vector.index"))

        with open(os.path.join(index_dir, "id_map.pkl"), "rb") as fd:
            self.id_map = pickle.load(fd)

        self.rec_nms_thresold = 0.05
        self.rec_score_thres = 0.5
        self.feature_normalize = True
        self.return_k = 1
        self.area_ratio_thresold=0.1

    def preprocess(self, input_dicts, data_id, log_id):
        (_, input_dict), = input_dicts.items()
        raw_img = input_dict["image"][0]
        data = np.frombuffer(raw_img, np.uint8)
        origin_img = cv2.imdecode(data, cv2.IMREAD_COLOR)
        dt_boxes = input_dict["bbox_result"]
        boxes = json.loads(dt_boxes)
        boxes.append({
    
    
            "category_id": 0,
            "score": 1.0,
            "bbox": [0, 0, origin_img.shape[1], origin_img.shape[0]]
        })
        self.det_boxes = boxes

        #construct batch images for rec
        imgs = []
        for box in boxes:
            box = [int(x) for x in box["bbox"]]
            im = origin_img[box[1]:box[3], box[0]:box[2]].copy()
            img = self.seq(im)
            imgs.append(img[np.newaxis, :].copy())

        input_imgs = np.concatenate(imgs, axis=0)
        return {
    
    "x": input_imgs}, False, None, ""

    def nms_to_rec_results(self, results, thresh=0.1):
        filtered_results = []
        x1 = np.array([r["bbox"][0] for r in results]).astype("float32")
        y1 = np.array([r["bbox"][1] for r in results]).astype("float32")
        x2 = np.array([r["bbox"][2] for r in results]).astype("float32")
        y2 = np.array([r["bbox"][3] for r in results]).astype("float32")
        scores = np.array([r["rec_scores"] for r in results])

        areas = (x2 - x1 + 1) * (y2 - y1 + 1)
        order = scores.argsort()[::-1]
        while order.size > 0:
            i = order[0]
            xx1 = np.maximum(x1[i], x1[order[1:]])
            yy1 = np.maximum(y1[i], y1[order[1:]])
            xx2 = np.minimum(x2[i], x2[order[1:]])
            yy2 = np.minimum(y2[i], y2[order[1:]])

            w = np.maximum(0.0, xx2 - xx1 + 1)
            h = np.maximum(0.0, yy2 - yy1 + 1)
            inter = w * h
            ovr = inter / (areas[i] + areas[order[1:]] - inter)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
            filtered_results.append(results[i])
        return filtered_results

    def check_boxes(self, results, area_ratio_thresh=0.1):
        filtered_results = []
        for result in results:
            if result["area_ratio"]>=area_ratio_thresh:
                filtered_results.append(result)
        if len(filtered_results)>0:
            return filtered_results
        else:
            return results

    def postprocess(self, input_dicts, fetch_dict, data_id, log_id):
        batch_features = fetch_dict["features"]

        if self.feature_normalize:
            feas_norm = np.sqrt(
                np.sum(np.square(batch_features), axis=1, keepdims=True))
            batch_features = np.divide(batch_features, feas_norm)

        scores, docs = self.searcher.search(batch_features, self.return_k)
        origin_img_box = self.det_boxes[len(self.det_boxes) - 1]["bbox"]
        total_pixes = origin_img_box[2] * origin_img_box[3]
        results = []
        for i in range(scores.shape[0]):
            pred = {
    
    }
            xmin, ymin, xmax, ymax = self.det_boxes[i]["bbox"]
            area_pix = (xmax - xmin) * (ymax - ymin)
            ratio = 0.0
            if total_pixes > 0:
                ratio = area_pix * 1.0 / total_pixes
            if scores[i][0] >= self.rec_score_thres:
                pred["bbox"] = [int(x) for x in self.det_boxes[i]["bbox"]]
                pred["rec_docs"] = self.id_map[docs[i][0]].split()[1]
                pred["rec_scores"] = scores[i][0]
                pred["area_ratio"] = round(ratio, 4)
                results.append(pred)

        #do nms
        results = self.nms_to_rec_results(results, self.rec_nms_thresold)
        print("{} SeriesRecOp data_id: {} --> Nms Result: {}".format(datetime.datetime.now(), data_id, results))
        results = self.check_boxes(results, self.area_ratio_thresold)
        print("{} SeriesRecOp data_id: {} --> Out Result: {}".format(datetime.datetime.now(), data_id, results))
        return {
    
    "result": str(results)}, None, ""


class CombineOp(Op):
    def preprocess(self, input_data, data_id, log_id):
        return None, False, None, ""

    def postprocess(self, input_dicts, fetch_dict, data_id, log_id):
        print("{} CombineOp data_id: {} --> input_dicts: {}".format(datetime.datetime.now(), data_id, input_dicts))
        results = {
    
    }
        for op_name, data in input_dicts.items():
            if "brands" in op_name:
                ret = data["result"]
                if ret is not None:
                    results["brands"] = json.loads(ret.replace("'", "\""))
                else:
                    results["brands"] = "[]"
            elif "series" in op_name:
                ret = data["result"]
                if ret is not None:
                    results["series"] = json.loads(ret.replace("'", "\""))
                else:
                    results["series"] = "[]"
        print("{} CombineOp data_id: {} --> Out Result: {}".format(datetime.datetime.now(), data_id, results))
        return {
    
    "result": str(results)}, None, ""


class RecognitionService(WebService):
    def get_pipeline_response(self, read_op):
        read_op2 = TestRequestOp()
        det_op = DetOp(name="det", input_ops=[read_op2])
        rec_brands_op = BrandsRecOp(name="rec_brands", input_ops=[det_op])
        rec_series_op = SeriesRecOp(name="rec_series", input_ops=[det_op])
        combine_op = CombineOp("combine", input_ops=[rec_brands_op, rec_series_op])
        return combine_op


product_recog_service = RecognitionService(name="recognition")
product_recog_service.prepare_pipeline_config("config.yml")
product_recog_service.run_service()

Start the service:

# 启动服务，运行日志保存在 log.txt
nohup python3.8 recognition_web_service.py &>log.txt &

If faiss is not found, please refer to here: https://blog.csdn.net/weixin_43882112/article/details/107614217

view progress

ps -ef|grep python

close process

kill -9 19913

view log

tail -f 1000 log.log

How to Check Port Occupation

$: netstat -anp | grep 8888
tcp        0      0 127.0.0.1:8888          0.0.0.0:*               LISTEN      13404/python3       
tcp        0      1 172.17.0.10:34036       115.42.35.84:8888       SYN_SENT    14586/python3 

Forcibly kill the process: by pid

$: kill -9 13404
$: kill -9 14586
$: netstat -anp | grep 8888
$:

service test

Modify the pipeline_http_client.py file as follows:

import requests
import json
import base64
import os

imgpath = "图片路径.jpg"

def cv2_to_base64(image):
    return base64.b64encode(image).decode('utf8')

if __name__ == "__main__":
    url = "http://127.0.0.1:8899/recognition/prediction"

    with open(os.path.join(".",  imgpath), 'rb') as file:
        image_data1 = file.read()
    image = cv2_to_base64(image_data1)
    data = {
    
    "key": ["image"], "value": [image]}

    for i in range(1):
        r = requests.post(url=url, data=json.dumps(data))
        print(r.json())

send request:

python3.8 pipeline_http_client.py

After a successful run, the results predicted by the model will be printed in the client, as follows:

{
    
    'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["{'brands': [{'bbox': [16, 19, 492, 565], 'rec_docs': '1', 'rec_scores': 0.98805684, 'area_ratio': 0.7432}], 'series': [{'bbox': [16, 19, 492, 565], 'rec_docs': '6', 'rec_scores': 0.9267364, 'area_ratio': 0.7432}]}"], 'tensors': []}