The task of human behavior recognition aims to identify the specific actions of the human body by analyzing the posture of the human body, and provide technical support for human behavior prediction, emergency handling, smart fitness, smart nursing and other fields.
Human behavior recognition data labeling method
Common labeling methods for human behavior data include human body key point labeling and action label labeling. Human body key point labeling provides position information of each joint point of the human body.
According to different recognition accuracy requirements, the key points of the human body can usually be marked with 14 points, 18 points, 22 points or even more points, which are all based on the extension of the movable joint points in the human skeleton.
Label annotation mainly marks the behavior category corresponding to the action, which is an overall description of human behavior. Human behavior is usually divided into static behavior and dynamic behavior. The specific annotation forms are as follows:
Static behavior: directly mark the key points of the target human body in the image, and label the overall behavior type.
Dynamic Behavior: For dynamic behavior, it is also necessary to add a video frame extraction module to extract image frames based on a specific sampling rate for dynamic human behavior videos, and then mark the key points of the human body on the image frames, and mark the human body behavior type labels on the dynamic behavior videos as a whole .
Difficulties in Human Behavior Recognition Tasks
Based on the actual situation, the human behavior recognition task has the following three difficulties:
Complex behavior: There are many types of human behavior, and some behaviors have a certain degree of subjectivity (such as wandering behavior), which makes algorithm recognition difficult.
Human body occlusion: The human body will occlude itself when performing specific behaviors. In addition, the human body will be occluded by other objects in the scene in a real scene, making it difficult to detect key points.
Data noise: In real scenarios, data noise has a greater impact. Taking security monitoring scenarios as an example, firstly, the resolution of security cameras is generally low, the shooting distance is long, and the proportion of human body features is small; Features have a greater impact; in addition, when the human body makes dynamic movements, it will also cause motion blur.
The above factors pose challenges to the robustness of human body recognition algorithms.
Datatang Human Behavior Recognition Dataset
According to the task requirements and difficulties of human behavior recognition, Datatang has designed the following data sets for static behavior and dynamic behavior from the data level, and they are respectively introduced as follows:
01
50356 human body cutouts and 18 key point data
The dataset collects 50,356 images of yellow, black, and white people. In order to improve the diversity of behaviors, the data is targeted to collect normal human motion data and large-scale human posture change data in many physical exercise scenes. At the same time, in order to ensure the authenticity of the data, a large number of human appendages and scene occlusions will appear in the image . In terms of data labeling, the positions of 18 key points and semantic segmentation contours of human targets are marked.
The details of the data are as follows:
02
466 people and 18880 3D human body instance segmentation and human body 22 key point annotation data
The data uses depth cameras to collect 3D human behavior data of 466 people, and the depth information and 2D information have been registered. The common behavioral postures of the human body are collected, including simple dance movements, limb stretching movements, and some non-standing movements such as cross-legged bending. In terms of labeling, 22 key point positions of human targets, semantic segmentation contours and large category labels of actions are marked.
The details of the data are as follows
:
5,500 people security monitoring human behavior recognition data
During the collection process, the data set is firstly set up as a collection scene, and 12 collection cameras are arranged in a circle around the site (one camera every 30 degrees to ensure that the same behavior can obtain multi-view data). Common human behaviors include standing, squatting, walking, saying hello, shaking hands, making phone calls, smoking, wandering, falling, squatting to protect your head, etc. In terms of labeling, the dataset has been marked with behavioral labels for each behavior.
The details of the data are as follows:
Behavior recognition data of 65 people and 15204 fitness videos
This data set uses color cameras and infrared cameras to collect human body fitness video data. In order to ensure the diversity of data, various designs have been made for shooting distance and collection clothing during the collection process. In terms of specific fitness behaviors, the data covers 50 common fitness movements, including rope skipping, push-ups, planks, vertical jumps, squats, jacking jumps, etc., which can meet the needs of most fitness behavior recognition algorithms. In terms of data labeling, the dataset labels the fitness action labels corresponding to each video.
The details of the data are as follows:
Relying on its own data advantages and rich experience in data processing, the human behavior recognition data set launched by Datatang provides assistance for the extensive application of human behavior recognition technology.