Recently, the U.S. Patent and Trademark Office officially awarded Apple a patent related to the main functions of Apple Vision Pro: hand tracking and finger gestures.
The Apple patent states that the quality of immersion depends on several important factors. For example, characteristics of a display such as image quality, frame rate, pixel resolution, high dynamic range (HDR), persistence, and screen door effect (i.e., the visible lines between pixels on the screen). When the displayed field of view is too narrow or various tracking functions are slow and/or inaccurate (leading to disorientation and nausea; also known as motion sickness), the quality of the immersive experience is reduced.
Head position tracking (referred to as positional tracking from here on) is designed to estimate the position and orientation of the HMD in the environment and must be both low-latency and accurate. The reason for this is that the rendered graphics must closely match the user's head movements to produce high-quality immersion in VR, and virtual content needs to be properly aligned in the real world in MR.
User body tracking estimates the position and orientation of the user's body (but not limited to hands and fingers) relative to the HMD. It can provide a means of user input (such as gestures) in VR and MR to enable interaction with virtual elements. While some positional tracking methods can also be used for hand tracking (e.g., an infrared camera with an LED array on a handheld controller), others take advantage of a smaller analysis space (typically within a meter of the HMD) to improve hand and finger tracking. Robustness of the tracking algorithm.
Apple patent Figure 1 is a schematic diagram of a user wearing a head-mounted display (HMD) with multiple cameras and infrared (IR) emitters; Figure 2A is an example of optics, displays, and cameras for VR and MR Top view of the preferred embodiment.
Apple's patent Figure 4A shows a front view of a first exemplary implementation of an HMD device with two RGB cameras optimized for see-through purposes (MR) and two infrared cameras providing visual data for tracking; Figure 5 is an implementation Flowchart of processing steps for VR with position and user body tracking; Figure 8 is a flowchart of an exemplary process for implementing user body tracking. It also includes gesture recognition.
In WWDC23, Apple officially announced that the head display product is called "Apple Vision Pro", runs a new operating system "visionOS", and is equipped with M2 and a new R1 chip.
In terms of design, the Vision Pro looks like ski goggles and diving glasses. Its frame is made of aluminum alloy and includes sensors, cameras, displays, and computing components cooled by fans. The headband and earmuffs are made of pure fabric and can be bent to fit various face and head shapes. In terms of lenses, Apple has cooperated with Zeiss to customize, and the prescription lenses produced can be magnetically attached to the display screen, which is convenient for myopic people to use the headset. In terms of screen, it is equipped with two 4K micro OLED screens, with a total of 23 million pixels, and each eye has more pixels than a 4K TV.
Apple designed a unique dual-chip design for Vision Pro. The main chip is the familiar M2, which is responsible for visual algorithms and image information processing, allowing Vision Pro to have a smoother experience. At the same time, in order to process the information of 12 cameras, 5 sensors and 6 microphones, Apple also brought a brand new R1 chip, so that the image can be synchronized to the Vision Pro screen within 12 milliseconds, almost without delay. It's like everything is happening right in front of you.
Lenz Technology has independently developed wireless radio frequency and low-power Bluetooth BLE5.2 chips and has global intellectual property rights. It provides "software and hardware commonality" solutions and cores for Bluetooth master fully integrated chips for the AIoT Internet of Things field and individual consumers. Devices, supporting a full range of APP software platform custom development. The designed Bluetooth chip solution is used in scenarios such as smart wearable devices, Bluetooth indoor navigation, smart home, medical health, sports and fitness, data transmission, remote control, personal peripherals and AIoT Internet of Things. (tingting is a brand of Lenze Technology, which mainly provides Apple Find My service and applies it to major IoT products)