iOS parses AR human body pose recognition data (specific status of each part)

iOS Parsing AR Human Pose Recognition Data

Personal interest, I studied the human body gesture recognition of iOS, encountered some problems, and finally solved them, share them below

question

The demo is provided on the Apple development website, but
the demo address
is pure Swift language, and the model used is its own model. I want to apply the recognized pose to my own model. , so the original data of the pose is needed, so I did the research.

I parsed it in Objective-C language and made a simple package, which can be used if necessary.

method

@interface ARBodyInfo : NSObject
@property(nonatomic,assign)SCNVector3 positions;
@property(nonatomic,assign)SCNVector4 orientation;

//身体
@property(nonatomic,assign)SCNMatrix4 hips_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_1_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_2_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_3_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_4_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_5_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_6_Transform;
@property(nonatomic,assign)SCNMatrix4 spine_7_Transform;

//脖子
@property(nonatomic,assign)SCNMatrix4 neck_1_Transform;
@property(nonatomic,assign)SCNMatrix4 neck_2_Transform;
@property(nonatomic,assign)SCNMatrix4 neck_3_Transform;
@property(nonatomic,assign)SCNMatrix4 neck_4_Transform;

//头
@property(nonatomic,assign)SCNMatrix4 head_Transform;
@property(nonatomic,assign)SCNMatrix4 nose_Transform;
@property(nonatomic,assign)SCNMatrix4 jaw_Transform;
@property(nonatomic,assign)SCNMatrix4 chin_Transform;

//手
@property(nonatomic,assign)SCNMatrix4 right_shoulder_1_Transform;
@property(nonatomic,assign)SCNMatrix4 right_arm_Transform;
@property(nonatomic,assign)SCNMatrix4 right_forearm_Transform;
@property(nonatomic,assign)SCNMatrix4 right_hand_Transform;

@property(nonatomic,assign)SCNMatrix4 left_shoulder_1_Transform;
@property(nonatomic,assign)SCNMatrix4 left_arm_Transform;
@property(nonatomic,assign)SCNMatrix4 left_forearm_Transform;
@property(nonatomic,assign)SCNMatrix4 left_hand_Transform;

//脚
@property(nonatomic,assign)SCNMatrix4 right_upLeg_Transform;
@property(nonatomic,assign)SCNMatrix4 right_leg_Transform;
@property(nonatomic,assign)SCNMatrix4 right_foot_Transform;
@property(nonatomic,assign)SCNMatrix4 right_toes_Transform;
@property(nonatomic,assign)SCNMatrix4 right_toesEnd_Transform;

@property(nonatomic,assign)SCNMatrix4 left_upLeg_Transform;
@property(nonatomic,assign)SCNMatrix4 left_leg_Transform;
@property(nonatomic,assign)SCNMatrix4 left_foot_Transform;
@property(nonatomic,assign)SCNMatrix4 left_toes_Transform;
@property(nonatomic,assign)SCNMatrix4 left_toesEnd_Transform;

@end

@interface ARBodyParsingTool : NSObject
/**
 传入监听到的ARBodyAnchor数据，会返回解析的ARBodyInfo
 */
+(ARBodyInfo*)parsingForAr:(ARBodyAnchor*)bodyInfo;

illustrate

ARBodyInfo is a class encapsulated by myself, which contains the parsed data, including the positions and orientation information of the entire body, and the position information Transform of specific parts of the body

use

-(void)session:(ARSession *)session didUpdateAnchors:(NSArray<__kindof ARAnchor *> *)anchors{
    dispatch_async(dispatch_get_main_queue(), ^{
        for (ARAnchor * object in anchors) {
            if([object isKindOfClass:[ARBodyAnchor class]]){
                ARBodyAnchor * bodyClass = (ARBodyAnchor *)object;
               ARBodyInfo * info =  [ARBodyParsingTool parsingForAr:bodyClass];
               //根据info，部署自己模型各个位置的姿势
            }
        }
    });
}

document

iOSAR Human Pose Recognition Analysis

ps. This file is a test version and is only for testing. If you need an official version, please contact the author

contact author

Looking forward to your likes and attention! In case of doubt, contact the author.