[OpenCV DNN] Flask Video Surveillance Target Detection Tutorial 09

Welcome to the "OpenCV DNN @ Youcans" series, which is being updated continuously

This series starts from scratch and explains in detail the use of the Flask framework to build a web application for the OpenCV DNN model. This section uses a multi-threaded or asynchronous framework to handle video frame acquisition and face recognition.

This routine uses one thread to obtain video frames in real time, and processes the video frames in the main thread for face recognition and image encoding. Therefore, face recognition will not block the reading of video frames.

3.9 OpenCV+Flask multi-thread processing real-time monitoring face recognition

Since face recognition is a computationally intensive task, it may slow down the processing of the video stream. To solve this problem, multithreading or asynchronous frameworks can be used to handle video frame acquisition and face recognition.

This routine uses one thread to obtain video frames in real time, and processes the video frames in the main thread for face recognition and image encoding. Therefore, face recognition will not block the reading of video frames.

We added a new method update_frame to the VideoStream class, and opened a new thread to run this method when the class was initialized. The video update function update_frame() obtains a new video frame in real time and stores it as an attribute of the class.

In the main thread, the generator function get_frame() is only responsible for face recognition and image encoding. gen_frames() acquires pictures frame by frame, uses the Haar cascade classifier pre-training model for face detection, encodes the images and returns them to the client. After the client browser receives the video stream, it displays it frame by frame in the picture defined by the img tag, thereby realizing video playback. Even if face recognition and image encoding take some time, it will not block the video refresh update.

Create a new Flask project cvFlask09

Create a new Flask project cvFlask09, the framework of this project is the same as cvFlask08.
The file tree of the cvFlask09 project is as follows.

---文件名\
    |---models\
    |    |---haarcascade_frontalface_alt2.xml
    |---templates\
    |    |---index4.html
    |    |---index5.html
    |--- cvFlask09.py
    |--- vedio_01.mp4

Python program file

The task logic is implemented by the Python program file cvFlask09.py, and the complete code is as follows.

# cvFlask09.py
# OpenCV+Flask 图像处理例程 09
# 通过浏览器播放摄像头实时监控视频+人脸识别，多线程处理
# Copyright 2023 Youcans, XUPT
# Crated：2023-5-18

# coding:utf-8

from flask import Flask, Response, request, render_template
import threading
import cv2

app = Flask(__name__)  # 实例化 Flask 对象

# 定义视频流类
class VideoStream:
    def __init__(self, source):  # 传入视频源
        self.video_cap = cv2.VideoCapture(0)  # 创建视频读取对象
        # 加载 Haar 级联分类器 预训练模型
        model_path = "./models/haarcascade_frontalface_alt2.xml"
        # 加载人脸检测级联分类器
        self.face_cascade = cv2.CascadeClassifier(model_path)
        self.success, self.frame = self.video_cap.read()  # 读取视频帧
        threading.Thread(target=self.update_frame, args=()).start()

    def __del__(self):
        self.video_cap.release()  # 释放视频流

    def update_frame(self):
        while True:
            self.success, self.frame = self.video_cap.read()

    def get_frame(self):
        gray = cv2.cvtColor(self.frame, cv2.COLOR_BGR2GRAY)
        # 使用级联分类器检测人脸
        faces = self.face_cascade.detectMultiScale(gray, scaleFactor=1.2,
                                  minNeighbors=5, minSize=(30, 30), maxSize=(300, 300))
        # 绘制人脸检测框
        for (x, y, w, h) in faces:
            cv2.rectangle(self.frame, (x, y), (x+w, y+h), (255,0,0), 2)

        ret, buffer = cv2.imencode('.jpg', self.frame)  # 编码为 jpg 格式
        frame_byte = buffer.tobytes()  # 转换为 bytes 类型
        return frame_byte

# 生成视频流的帧
def gen_frames(video_source):
    video_stream = VideoStream(video_source)  # 从视频文件获取视频流
    while True:
        frame = video_stream.get_frame()  # 获取视频帧
        if frame is None:
            # video_stream.__del__()  # 释放视频流
            break
        yield (b'--frame\r\n' b'Content-Type: image/jpeg\r\n\r\n'
               + frame + b'\r\n')  # 生成视频流的帧


@app.route('/video_feed')
def video_feed():
    video_source = request.args.get('video_source', 'camera')  # 从网页获取视频源

    # 通过将一帧帧的图像返回，就达到了看视频的目的。multipart/x-mixed-replace是单次的http请求-响应模式，如果网络中断，会导致视频流异常终止，必须重新连接才能恢复
    return Response(gen_frames(video_source), mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route('/')
def index_camera():  # 实时视频监控
    # <img src="{
    
    { url_for('video_feed', video_source='camera') }}">
    return render_template('index4.html')

@app.route('/vidfile')
def index_vidfile():  # 播放视频文件
    # <img src="{
    
    { url_for('video_feed', video_source='vedio_01.mp4') }}">
    return render_template('index5.html')

if __name__ == '__main__':
    # 启动一个本地开发服务器，激活该网页
    print("URL: http://127.0.0.1:5000")
    app.run(host='0.0.0.0', port=5000, debug=True, threaded=True)  # 绑定 IP 地址和端口号

Web page template for video streaming

The web page templates index4.html and index5.html of the video stream are located in the templates folder, and the content is exactly the same as that of the cvFlask08 project.

The webpage index4.html is located in the templates folder, and the specific content is as follows.

<!DOCTYPE html>
<html>
  <head>
    <title>Video Streaming Demonstration</title>
  </head>
  <body>
    <h2  align="center">OpenCV+Flask 例程：实时视频监控</h2>
    <div style="text-align:center; padding-top:inherit">
      <img src="{
    
    { url_for('video_feed', video_source='camera') }}" width="600"; height="360">
    </div>
  </body>
</html>

The webpage index5.html is located in the templates folder, and the specific content is as follows.

<!DOCTYPE html>
<html>
  <head>
    <title>Video Streaming Demonstration</title>
  </head>
  <body>
    <h2  align="center">OpenCV+Flask 例程：播放视频文件</h2>
    <div style="text-align:center; padding-top:inherit">
      <img src="{
    
    { url_for('video_feed', video_source='vedio_01.mp4') }}" width="600"; height="360">
    </div>
  </body>
</html>

Program running

Enter the root directory of the cvFlask09 project, run the program cvFlask09.py, and start the streaming server.
Open http://192.168.3.249:5000 in the browser of the device (including mobile phone) in the LAN to play the real-time video surveillance screen.

【End of this section】

In the next section we will discuss object detection for real-time video using OpenCV DNN.

Copyright statement:
Welcome to pay attention to
the original works of youcans@xupt in the "OpenCV DNN @ Youcans" series
. Reprints must be marked with the original link: [OpenCV DNN] Flask Video Surveillance Target Detection Tutorial 09
(https://blog.csdn.net/youcans/article/ details/131270693)
Copyright 2023 youcans, XUPT
Crated: 2023-06-18