Home fitness intelligent recommendation and posture monitoring system based on midiepipe and opencv

0. Preface

This project was completed by me, gay friends and three 21st grade students in cooperation for one year

In order to provide athletes with a more scientific and accurate personalized exercise program, a home fitness intelligent recommendation and posture monitoring system was designed and implemented.

The realization principle is as follows:
the system recommends a personalized fitness plan for the individual based on the physical indicators of the detected individual; by dynamically collecting the movement data values ​​of the limb joints, the collected movement data values ​​of the limb joints are combined with the preset movement data of the limb joints. The threshold value is compared in real time, and if it exceeds the preset threshold value range, a prompt will be given to meet the requirement of maintaining the correct posture, so as to realize the functions of monitoring and alarming the human body movement posture.

After running the program, the camera will be called to record the motion video, and then the posture of the video will be analyzed to determine whether your motion posture is standard.

Operating system: Windows10 Home Edition
Development environment: Pycharm Community 2022.3
This project depends on Python3.8
third-party libraries: opencv, midiepipe, pygame

The video stream is obtained by opencv, the mixer module in pygame realizes voice broadcast, midiepipe realizes action point capture, and uses yolo5 to train standard action models.

1. Introduction to opencv

OpenCV (Open Source Computer Vision Library) is an open source computer vision library. It is developed and maintained by Intel Corporation. It is cross-platform, efficient, and easy to use. It is widely used in the field of computer vision.

OpenCV can complete many computer vision tasks, including image processing, object detection, face recognition, visual tracking, structured light scanning, stereo vision, machine learning, etc. It supports a variety of programming languages, including C++, Python, Java, etc., with Python being one of the most popular languages.

Some of the important features of OpenCV include:

• Image and Video I/O
• Image manipulation and transformation, such as scaling, rotation, cropping, etc.
• Feature extraction and description, such as edge detection, ORB, SIFT, SURF, etc.
• Target detection and tracking, such as Haar Cascade, HOG+SVM, KCF, etc.
• Face recognition, including face detection, alignment, feature extraction, etc.

Video analysis and computer vision algorithms, such as optical flow, background modeling, segmentation, stereo vision, panoramic image stitching, etc.

2. Introduction to midiepipe

MIDIEpipe (MIDI Event Processor) is a MIDI data processing framework written in Python, which can help users efficiently and flexibly process MIDI data.

MIDIEpipe can abstract and separate the input, processing and output of MIDI data sources, and can perform some specific operations in each step, such as filtering, conversion, recording, adjustment and so on. Users can customize these operations to achieve specific MIDI data processing by writing MIDIEpipe scripts. At the same time, MIDIEpipe also supports the connection of MIDI equipment, which can realize real-time MIDI data capture and processing.

The MIDIEpipe library has a powerful and easy-to-use API that supports multiple operations such as MIDI data reading, filtering, conversion, and output. It also supports a series of MIDI device interfaces, which can easily realize the interaction between MIDI devices and systems. MIDIEpipe also supports the use of Python scripts to process MIDI data, which provides users with a high degree of flexibility and scalability.

Overall, MIDIEpipe is a very powerful and easy-to-use MIDI data processing framework, which can help Python developers better handle MIDI data and realize various interesting MIDI applications.

3. Introduction to yolo5

YOLOv5 (You Only Look Once version 5) is a target detection algorithm recently launched by Ultralytics. Compared with the previous version of YOLOv4, the performance has been improved more, and the speed and accuracy have been well balanced. YOLOv5 contains a variety of models that can be used in areas such as target detection, object segmentation, and portrait segmentation.

Different from other target detection algorithms, YOLOv5 only needs one forward pass in the detection process to obtain the position and classification information of the target. At the same time, YOLOv5 also adopts a new Backbone algorithm, which is the variant CSPNet of ResNet, which can better solve the problem of gradient disappearance during deep neural network training and achieve better performance.

At present, YOLOv5 is widely used in the field of computer vision, especially in scenarios that require real-time performance. For example, in application scenarios such as pedestrian detection, vehicle detection, object tracking, and security monitoring, YOLOv5 can be used for fast and accurate object detection.

4. Virtual environment construction and project directory setting

You can use the following third-party libraries to build a virtual environment for this project :

requirement.txt

PIL == 8.4.0
pygame == 2.0.2
eyed3 == 0.9.6
python-opencv == 4.5.5
mediapipe== 0.8.9.1
numpy== 1.21.4

Project source file directory:

• main.py
• dispose.py
• voice.py
• camera.py
• hand_test.py

5. Partial code display

camera.py

import cv2

def videocapture():
	cap = cv2.VideoCapture(0)  # 生成读取摄像头对象
	width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))  # 获取视频的宽度
	height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))  # 获取视频的高度
	fps = cap.get(cv2.CAP_PROP_FPS)  # 获取视频的帧率
	fourcc = int(cap.get(cv2.CAP_PROP_FOURCC))  # 视频的编码
	# 定义视频对象输出
	writer = cv2.VideoWriter("./video_result.mp4", fourcc, fps, (width, height))
	while cap.isOpened():
		ret, frame = cap.read()  # 读取摄像头画面
		cv2.imshow('teswell', frame)  # 显示画面
		key = cv2.waitKey(24)


		'''转化为灰度图像进行处理'''
		# gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  # 将捕获的一帧图像灰度化处理
		# frame = cv2.flip(gray, 1)  # 图像翻转（0：垂直翻转；1：水平翻转；-1：垂直水平翻转）

		# frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  # 将捕获的一帧图像灰度化处理
		# frame = cv2.flip(frame, 1)

		writer.write(frame)  # 视频保存
		# 按Q退出
		if key == ord('q'):
			break
	cap.release()  # 释放摄像头
	cv2.destroyAllWindows()  # 释放所有显示图像窗口

dispose.py

import os
import cv2
import mediapipe as mp
import time
import os

# os.environ["CUDA_DEVICES_ORDER"]="PCI_BUS_IS"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

def fun():
    # mp.solutions.drawing_utils用于绘制
    mp_drawing = mp.solutions.drawing_utils

    # 参数：1、颜色，2、线条粗细，3、点的半径
    DrawingSpec_point = mp_drawing.DrawingSpec((0, 255, 0), 1, 1)
    DrawingSpec_line = mp_drawing.DrawingSpec((173,255,47), 1, 1)

    # mp.solutions.holistic是一个类别，是人的整体
    mp_holistic = mp.solutions.holistic

    # 参数：1、是否检测静态图片，2、姿态模型的复杂度，3、结果看起来平滑（用于video有效），4、检测阈值，5、跟踪阈值
    holistic = mp_holistic.Holistic(min_detection_confidence=0.5, min_tracking_confidence=0.5)

    cap = cv2.VideoCapture('./video_result.mp4')

    while cap.isOpened():
        success, image = cap.read()
        if not success:
            print("Ignoring empty camera frame.")
            # open_mp3()
            break
        image1 = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

        # 处理RGB图像
        results = holistic.process(image1)
        '''
        
        mp_holistic.PoseLandmark类中共33个人体骨骼点
        mp_holistic.HandLandmark类中共21个手部关键点
        脸部有468个关键点
        '''
       # 省略部分核心代码

def analysic():
    fun()
    holistic.close()
    cv2.destroyAllWindows()
    cap.release()

6. Project Outcomes

Both half-body and full-body frames can collect the nodes of important human body motion analysis.

In fact, it can realize the effect of analyzing the video and comparing it with the standard action. Since the video is inconvenient to upload, the video demonstration effect will not be shown here.

7. Description

In order to protect property rights, the code and resource files of this project are not complete.

1. The voice packet is replaced with a nonsensical voice
2. This project dispose.pyhas hand_test.pydeleted the core algorithm code and cannot complete the core functions.

This article refers to:
Home Fitness Intelligent Recommendation and Posture Monitoring System | Dao Kun