Unity3D Pico VR gesture recognition_Cool-Hao's blog-CSDN blog
This article mainly explains how to track gestures and customize pre-made recognition of gestures. If you don’t know how to import the SDK and configure the environment, please read the previous chapter.
Environmental requirements
- SDK version: 2.3.0 and above
- PICO device models: PICO Neo3 and PICO 4 series
- PICO device system version: 5.7.0 and above
hand joint definition
The "gesture tracking" function of PICO SDK follows the gesture definition specifications proposed by the OpenXR standard and supports the following 26 hand joint points.
value |
joint name |
joint meaning |
OpenXR definition fields |
| 0 |
Palm |
center point of palm |
XR_HAND_JOINT_PALM_EXT |
| 1 |
Wrist |
wrist joint points |
XR_HAND_JOINT_WRIST_EXT |
| 2 |
Thumb_metacarpal |
thumb metacarpal joint |
XR_HAND_JOINT_THUMB_METACARPAL_EXT |
| 3 |
Thumb_proximal |
proximal bone joint of thumb |
XR_HAND_JOINT_THUMB_PROXIMAL_EXT |
| 4 |
Thumb_distal |
Distal bone joint of thumb |
XR_HAND_JOINT_THUMB_DISTAL_EXT |
| 5 |
Thumb_tip |
Top bone joint of thumb |
XR_HAND_JOINT_THUMB_TIP_EXT |
| 6 |
Index_metacarpal |
index finger metacarpal joint |
XR_HAND_JOINT_INDEX_METACARPAL_EXT |
| 7 |
Index_proximal |
proximal bone joint of index finger |
XR_HAND_JOINT_INDEX_PROXIMAL_EXT |
| 8 |
Index_intermediate |
middle index finger bone joint |
XR_HAND_JOINT_INDEX_INTERMEDIATE_EXT |
| 9 |
Index_distal |
distal index finger bone joint |
XR_HAND_JOINT_INDEX_DISTAL_EXT |
| 10 |
Index_tip |
Top bone joint of index finger |
XR_HAND_JOINT_INDEX_TIP_EXT |
| 11 |
Middle_metacarpal |
middle finger metacarpal joint |
XR_HAND_JOINT_MIDDLE_METACARPAL_EXT |
| 12 |
Middle_proximal |
Proximal bone joint of middle finger |
XR_HAND_JOINT_MIDDLE_PROXIMAL_EXT |
| 13 |
Middle_intermediate |
middle finger bone joint |
XR_HAND_JOINT_MIDDLE_INTERMEDIATE_EXT |
| 14 |
Middle_distal |
distal bone joint of middle finger |
XR_HAND_JOINT_MIDDLE_DISTAL_EXT |
| 15 |
Middle_tip |
Top bone joint of middle finger |
XR_HAND_JOINT_MIDDLE_TIP_EXT |
| 16 |
Ring_metacarpal |
Ring finger metacarpal joint |
XR_HAND_JOINT_RING_METACARPAL_EXT |
| 17 |
Ring_proximal |
proximal bone joint of ring finger |
XR_HAND_JOINT_RING_PROXIMAL_EXT |
| 18 |
Ring_intermediate |
middle finger bone joint |
XR_HAND_JOINT_RING_INTERMEDIATE_EXT |
| 19 |
Ring_distal |
Distal bone joint of ring finger |
XR_HAND_JOINT_RING_DISTAL_EXT |
| 20 |
Ring_tip |
Top bone joint of ring finger |
XR_HAND_JOINT_RING_TIP_EXT |
| 21 |
Little_metacarpal |
little finger metacarpal joint |
XR_HAND_JOINT_LITTLE_METACARPAL_EXT |
| 22 |
Little_proximal |
proximal bone joint of little finger |
XR_HAND_JOINT_LITTLE_PROXIMAL_EXT |
| 23 |
Little_intermediate |
middle bone joint of little finger |
XR_HAND_JOINT_LITTLE_INTERMEDIATE_EXT |
| 24 |
Little_distal |
distal bone joint of little finger |
XR_HAND_JOINT_LITTLE_DISTAL_EXT |
| 25 |
Little_tip |
top bone joint of little finger |
XR_HAND_JOINT_LITTLE_TIP_EXT |
PICO hand model
The SDK provides two standard hand model prefabs: HandLeft and HandRight. The number of vertices of a single hand mold prefab is 1209, the number of quadrilateral faces is 1198, and the number of triangle faces is 2414
Prefabs are located in the Packages > PICO Integration > Assets > Resources > Prefabs directory
Custom gestures and events
You can customize gestures and add trigger events to them to enrich the application experience.
Create gestures using the HandPoseGenerator prefab
Clicking the PXR_Hand Pose Generator new button will automatically create a configuration file to store gesture configurations.
Shapes : Component configures finger gestures.
The Bones component configures the relationship between hand joint points
The Transform component configures the hand orientation.
For specific operation steps, please refer to the " PICO Create Sample Gesture Document " document.
--------------------------------------------
I will create a 
gesture case here to show
Setting parameters
You can also see a preview of creating a gesture below
Added PXR_Hand Pose script to prefab.
In the Track Type field, select the hand to which the ThumbUp gesture applies.
In the Config field, add the Hand Pose Config file for the "ThumbUp" gesture.
Add gesture event triggering functions, including gesture start events, gesture duration events, and gesture end events.
Package the APK file and run it on the PICO device to experience the effect