In recent years, more and more VR headsets have begun to try to combine eye tracking, gesture tracking and other biometric technologies, and even in some VR social scenes, they are also exploring the combination of Avatar and facial recognition functions. It is conceivable that the relationship between biometrics and wearable technologies such as AR/VR will become closer and closer in the future. Although the relevant hardware is not mature enough in terms of size and integration at this stage, with the upgrading of sensing technology in the future, it is expected to be compatible with AR/VR. better combination.
In the field of VR, several mainstream bio-sensing technologies have been adopted, such as gesture and eye tracking cameras, heart rate monitoring modules, or using EMG to recognize gestures, EOG to recognize gaze points, and so on. For AR, since the shape of the glasses does not fit the face like VR, it is not suitable for solutions based on contact sensing.
Recently, a British company called Emteq announced a non-contact wearable emotion capture glasses: OCOsense. It is reported that Emteq was established in 2015 and has been focusing on AR/VR and wearable device emotion capture research solutions.
Existing solutions: emteqPRO (for VR), OCOsense (for AR)
For example, a few years ago, Qingting.com reported that the company’s VR head-mounted display face recognition solution Faceteq (now upgraded to emteqPRO solution), is characterized by: using EMG, PPG, IMU, eye tracking (optional) and other equipment, the recognition includes: : Heart rate, heart rate variability and respiratory rate, facial movements/expressions, interpupillary distance, and then use machine learning technology to analyze the user's attention, emotions, etc.
emteqPRO
emteqPRO includes a set of masks with integrated 7 electrode contacts (middle of the picture above), which can be used alone. There are also versions that can be integrated into PICO Neo3 Pro and HTC Vive Pro Eye devices for use with VR.
emteqPRO
These 7 electrodes are located at specific facial muscle positions, and recognize expressions by capturing the activation of facial muscles, such as: frontalis, orbicularis oculi, zygomaticus, corrugator and so on. Obviously, this is a set of contact recognition solutions suitable for VR devices.
OCOsense glasses
The newly announced OCOsense solution is characterized by the use of eye muscle force map sensors and pressure sensors to identify changes in facial muscles to identify facial expressions and emotions. The biggest change is the non-contact sensor solution, which is of great significance to the future application of lightweight AR glasses.
Emteq claims that OCOsense is the world's first wearable facial expression and emotion analysis glasses. Its main function is to recognize the user's expression changes and convert them into signals to control mobile devices. Its advantage is that it is easy to use and an all-in-one design that can track, identify and mark user feedback on AR or real-world scenes in real time.
In terms of details, OCOsense uses a self-developed OCO sensor array, based on the principle of non-contact eye muscle force recognition. The English full name is non-contact optomyographic (OMG), which refers to a technology for monitoring muscle activity. The basic principle of OMG is to use an active near-infrared optical sensor to emit a signal to the skin surface and measure the change in the signal reflected back.
In addition, the OCO sensor is an optical non-contact sensor that can read facial movement in 3 dimensions, providing a higher resolution signal than typical camera-based solutions. Additionally, EMG electrodes require firm and continuous contact with the skin to achieve an acceptable signal-to-noise ratio, which is impractical in the form of glasses. In contrast, the OCO sensor is based on optical principles, does not need to touch the skin, and can work within a distance of 4 mm to 30 mm from the human face.
In terms of hardware, OCOsense is equipped with a 9-axis IMU, which can add a camera module, equipped with an Arm Cortex processor, supports Bluetooth Low Energy 5.0, is equipped with RGB LED indicators, and a somatosensory vibration motor. In addition to hardware, Emteq also designed an emotional artificial intelligence engine to recognize emotions from facial expressions.
Emteq researchers pointed out that wearable devices such as AR/VR can now integrate a variety of sensors, such as accelerometers, ambient light sensors, temperature sensors, humidity sensors, barometers, gyroscopes, magnetometers, distance sensors, etc. In addition, the possibility of combining with biosensors, such as ECG, EEG, EMG, EGO, electrocardiogram, etc., is also being explored.
Usually, the human face conveys emotions, mental states, and behavioral intentions through facial muscle activation and structural changes, and then constitutes an expression. Human expressions hide a wealth of information, so the study of facial expression behavior has been an active topic since 1872.
For OCOsense, in addition to recognizing expressions and emotional responses, it has multiple uses. For example, in the process of technology verification, researchers demonstrated three application scenarios: controlling video (zoom in, zoom out, screenshot, change playback speed, forward and backward), make and receive calls, play games, and it is expected to be used for remote mental health monitoring in the future , EQ training, improvement of emotional disorders, rehabilitation treatment, and human-computer interaction scenarios such as controlling prosthetics.
In short, OCOsense, a non-contact face tracking solution, better protects privacy than camera-based solutions. It does not directly take pictures of people's faces, but only analyzes changes in facial muscles. Moreover, the camera solution depends on the light source and needs to be used in a well-lit environment, while OCOsense does not. And during strenuous exercise, the accuracy of the camera solution may not be guaranteed. Reference: Emteq