Copyright Information
Title: VR and AR enhanced virtual reality technology principle and reality of commercial applications
of: Sukhoi, Su Yan Zhao
Press: People's Posts and Telecommunications Press
Publication date: 2017-03
ISBN: 9787115447722
The concept of VR and AR
Build a virtual world by affecting your feelings.
Our cognition of the world comes from our consciousness, which is a combination of various sensations. Feelings can be divided into the following two categories.
External feeling
① Vision (visual senses are eyes)
② Hearing (audible senses are ears)
③ Smell (smell senses are noses)
④ Taste sensations (taste senses are tongues)
⑤ Skin sensations (skin senses are skin of the whole body)
Internal feeling
① Muscle motor sense
② Balance sense
③ Visceral sense
Ways to simulate various feelings of the human body
At present, the way of influencing or simulating various human feelings is mainly visual, auditory and action interaction.
Vision
Vision is the eye that affects people. The actual operation is to wear a head-mounted display, so what the human eye sees is provided by the head-mounted display.
Hearing
Hearing affects the human ear. The actual operation is to wear headphones (when wearing a monitor), then the sound heard by the human ear is provided by the headphones.
Action interaction
Action interaction actually includes the simulation of skin sensation, muscle movement sensation and balance sensation.
Skin perception : For example, when the user holds the handle to play a shooting game, it can simulate the feeling of holding a pistol or the like.
Muscle kinematics : For example, in the game, let the user hold the handle and swing, you can simulate the feeling of waving a sword.
Balance sense : Balance sense is the sensation caused by the changes in the direction of gravity of the human body when stimulating the vestibular receptors. The reason is the same as the muscle motion sense. It is also caused by human movement. The user will bring balance when rotating the head while wearing a head-mounted display. Changes in consciousness.
VR / AR / MR concepts and differences
VR (Virtual Reality): virtual reality;
AR (Augmented Reality): augmented reality;
MR (Mix Reality): mixed reality.
Fundamentally speaking, VR, AR and MR are a concept and an objective, not a specific technology.
VR (Virtual Reality)
Virtual reality is to create a virtual world. In other words, what the user sees is fake. Through virtual reality, users can feel that they are acting in a world different from the real world.
Technically, it is currently achieved by wearing VR glasses. The VR scene is basically a virtual world generated by computer operations. Of course, it is possible to further import real-world things, for example, to import scanned images of the user's body or a part of the body (typically like a hand) in the VR world. In doing so, it actually means a bit of mixed reality. Because it is basically based on VR glasses, we generally put this category in VR.
AR (augmented reality)
Augmented reality is to add something generated by computer operations in the real world, so as to achieve a better effect than the real world.
Technically, there are multiple ways of implementation, and the effects are also different. Nintendo's 3DS already has AR games, and now there are many AR games on mobile phones. A more representative AR device is Google's AR glasses Google Glass, which can superimpose symbols, images and text in the wearer's field of vision. Based on Google's augmented reality project Project Tango, Lenovo Phab 2, the world's first AR mobile phone equipped with Tango AR technology, was officially released on November 1, 2016.
MR (Mixed Reality)
Mixed reality is the fusion of the virtual world and the real scene until the boundary between the two is blurred, making it difficult to distinguish between the virtual and real scenes in front of you.
Speaking of which, in terms of conceptual definition, AR and MR also have elements of both virtual and reality, and also a mixture of the two. To say the difference, you can first look at the difference between Google's Google Glass and Microsoft's holographic glasses HoloLens. Microsoft's HoloLens can not only superimpose symbols, images and text in the wearer's field of vision, but also superimpose virtual images generated by computer operations.
The history of AR and VR
● 1930s: development history of airplane-shaped training cabin
● 1940s: Sawyer ’s stereo viewer
● 1950s: simulation simulator Sensorama
In 1957, Morton Heilig invented the simulation simulator Sensorama and obtained a patent in 1962. Sensorama can provide a fully immersive experience by simulating various senses. It is mainly composed of stereo speakers, stereo 3D display, fan, odor generator and vibration chair. It looks like sitting in front of the game console in the game console and sticking my head into an old TV. Although it looks funny, it conveys an accurate VR concept. In view of the technology at the time, the Sensorama produced was very immature and could not bring users a satisfactory experience. Even the current VR glasses have many shortcomings. The concept of Modern Heligo is indeed very advanced, but he failed to see the blossoming of virtual reality technology.
● 1960s: Telesphere Mask
The Telesphere Mask device is a virtual reality helmet and was patented in 1960. It is the first head-mounted display, including stereo 3D display, wide-angle vision and stereo, but lacks motion tracking.
● 1990s: Nintendo ’s virtual reality host Virtual Boy
Nintendo ’s Virtual Boy is the first home virtual reality gaming device. This game console has no head tracking or motion tracking, and uses a dual-screen design parallax to create a 3D effect. Although Nintendo defines it as a portable game console, it is too heavy to be placed on the desktop for easy operation. The image display problem is very big, because the image is only monochrome, users are easily dizzy to play, and the battery life is also poor. Because the product is too advanced and the marketing is hurried, even Nintendo himself has not developed any good games on Virtual Boy. In the end, Virtual Boy sold about 770,000 units, and it was a failure.
● 2010: Kinect used with Microsoft game consoles
● 2011: iPhone virtual reality viewer
● 2012: VR head-mounted display Oculus Rith
● 2014: Google Cardboard Glasses Cardboard
● 2015: Samsung Gear VR
● 2016: HTC Vive
● 2016: Microsoft HoloLens (mixed reality headset device)
● 2016: Sony PlayStation VR
Realization of AR and VR
Hardware classification
● As mentioned above for the output device
, the hardware for the visual aspect is the head-mounted display, and the hardware for the hearing aspect is the headset (head-mounted display). ), And even as one.
● Input devices
are mainly various basic action interaction devices. This type of action interaction equipment is a must and indispensable.
● Other auxiliary peripherals
There are various kinds of action interaction devices. This type of action interaction device can be said to be extra, with special functions, which can enhance the user experience.
Output device (basic platform)
The output device of VR / AR / MR is generally a head-mounted display (referred to as "head-mounted display" for short).
The head-mounted display covers the user's eyes to provide visual information. After the user wears the head-mounted display, the external signals are shielded and the signals from the head-mounted display are accepted, as if coming to another world, with a good sense of immersion. There are also earphones to provide auditory information. Some headsets come with headsets, and some headsets only provide a headset interface. The headset needs to be prepared by the user as an accessory.
● For VR glasses , including VR boxes for mobile phones (such as Samsung Gear VR, Google ’s first Daydream VR helmet Daydream View), VR glasses for PCs or game consoles (Oculus Rith, HTC Vive, PS VR), and VR integration Machine (such as Santa Cruz, the latest release from Oculus).
● AR glasses , such as Google's Google Glass. In addition, there are AR games on 3DS, and AR can also be implemented on mobile phones. Based on Google's augmented reality project Project Tango, Lenovo Phab 2, the world's first AR mobile phone equipped with Tango AR technology, was officially released on November 1, 2016.
● MR glasses , such as Microsoft's HoloLens. In addition, Magic Leap's so-called CR is also a kind of glasses that provide MR.
Input device (controller)
In terms of input devices, in fact, the head-mounted display has its own input device, head tracking device and eye tracking device; it also includes a handle, VR omnidirectional treadmill, motion tracking device, panoramic camera, etc.
Head tracking device
Generally speaking, the most basic of head-tracking devices is a gyroscope, but of course not by gyroscope alone. Head tracking devices, one is Inside-out Tracking, which allows the headset to detect environmental changes and reversely calculate the VR headset's own movement; the other is Outside-in Tracking, which uses an external tracking device To track the headset. The VR box of the mobile phone can rely on the camera of the mobile phone to provide moving images and detect the movement of the head. In fact, all kinds of glasses can do the same. This is the first kind. VR glasses can be attached with light-emitting diodes, and then use an external camera to scan and track, HTC Vive's Lighthouse system uses a photosensitive sensor, these belong to the second category.
Eye tracking device
The infrared sensor inside the head-mounted display can monitor the movement of the user's eyes. But there is still room for improvement in terms of accuracy.
Motion tracking equipment
External motion tracking devices, such as cameras and HTC Vive's Lighthouse system, can not only track head movements, but also handles, hand movements after wearing special gloves, and even body movements.
handle
VR glasses basically have matching manipulators, namely (game) handles. The handle can provide hand movement tracking, and the buttons and joystick provide control. Each company's VR glasses are equipped with unique handles.
VR omnidirectional treadmill
In the first chapter, we introduced many control methods of how to move in the virtual world, but these are all science fiction and not yet reality. In practical applications, many companies' technologies can only be done like HTC Vive, which needs to provide a relatively large space for users to move, but this condition is not easy to meet. In this case, the VR omnidirectional treadmill appeared.
VR panoramic camera
If you want to popularize VR video and mobilize more people to shoot VR video, a dedicated VR panoramic camera is essential. Many companies are developing VR panoramic cameras, such as OZO developed by Nokia. From the picture, we can see that the appearance of OZO is quite sci-fi, shaped as a sphere, with optical sensors and embedded microphones in 8 directions. With this layout, not only can the surrounding scene be recorded as video in all directions, but also accurately recorded The location of the sound in the scene.
Other auxiliary peripherals
3D headphones
We need better VR headsets. The human ear can perceive the direction of sound. Now, many earphones use this to emphasize the sense of sound orientation: when users put on the earphones, they can clearly perceive the position of each sound to meet the needs of restoring the presence of sound. However, the sound position of such headphones is always fixed within a very small radius around the user's head. When the user shakes his head, there is no change in the position of the sound.
The Ossic X headset crowdfunded on the crowdfunding platform Kickstarter in February 2016 solved this problem. It creates a variable sound field according to the actual environment through the 8 sound-generating units inside the machine, so as to achieve a sense of realism that can interact with VR. Its creative team OSSIC defined this headset as a 3D headset.
VR Full Body Touch Experience Kit
In less than two months in 2016, many virtual reality devices have been released, but virtual reality devices are not only headsets. To achieve a deep immersion, some supporting facilities are essential.
The transmission of all the information in our body is achieved by the weak electrical pulse signals circulating in the nervous system. When we touch something, this tactile sensation will be transmitted to our nervous system in the form of electrical pulses. In the brain, let us feel that it is the "mother tongue" of our body.
Teslasuit, which is currently being crowdfunded, can be said to be the world's first virtual reality full-body touch experience suite, and its working principle-muscle electrical stimulation (EMS) technology is to use our body's "mother tongue".
"Free" hands-controlled VR equipment
Friends who have played Xbox should have this feeling-the positioning is not accurate. Indeed, xboxes that use somatosensory cameras to capture user movements have done a little poorly in capturing user movements. If you just use xbox to play a game like fruit cutting, if you play some games that require high level of action detail capture, such as table tennis, it is difficult for xbox to accurately sense the change of user's hand swing. Therefore, it is difficult for users to achieve precise control of the game.
Recently, VR leader Oculus teamed up with the University of Washington to launch a new project called Finexus. Finexus aims to provide input devices for virtual reality experiences. Through Finexus, users can easily complete activities that require extremely high accuracy of finger motion capture, such as writing, pressing flat bubble paper, playing guitar, and typing. For example, once Finexus technology is popularized, pianists can interact with the audience during the concert to achieve the effect of "playing together". Finexus not only has high measurement accuracy, but also frees the user's hands, so that users no longer need to hold the handle or joystick during the virtual reality experience.
Tactile, floatable VR super peripheral
With the release of VR devices and more and more VR games, players with conditional purchases can now enjoy playing in the VR world as long as they wear a headset and pick up a gamepad or controller. But human beings are a kind of creature with endless desires. Players are gradually dissatisfied with existing VR input devices. They want to have more functions in the game, such as complete body dynamic capture, touch, etc. And human beings are also a kind of creatures that can meet their own needs. With such needs in the market, of course, there will be companies that solve these needs. Recently, AxonVR, a startup company in the United States, is preparing to launch a super VR peripheral-Axon VR, to deal with the problem of insufficient realism in control.
VR backpack PC
VR glasses suitable for PCs or game consoles are currently not available with wireless devices, and require cables to connect PCs or game consoles. The pinning of the cable is quite annoying. The user may easily accidentally entangle the cable due to movement during use. As a result, there is demand for VR backpacks. The concept of the VR backpack is quite simple, that is, the host is carried on the body as a backpack, so that the user will not entangle the cable due to movement during use.
Zotac's solution was simple at first. His mini PC ZBOX, because of its small size, can be installed in a backpack. Users can connect the VR glasses with a cable, and there will be no trouble that the cable is easily tangled. Later, Zotac launched ZOTAC VR GO, a VR backpack computer, which is a mature product specifically for VR.
VR gaming rotating chair
When the traditional computer chair evolved into a VR game rotating chair. A VR company based in London has crowdfunded a VR game Rotor VR on KickStarter, but its supporters are far below expectations. After the crowdfunding failed, Roto was lucky to get a $ 300,000 financing to provide financial support for the company to continue to develop new products, and then released a new version of the VR game rotating chair.
Application of motion capture in VR
3.1 Laser positioning technology
3.2 Infrared optical positioning technology
3.3 Visible light positioning technology
3.4 Computer vision motion capture technology
3.5 Motion capture technology based on inertial sensors
Three-dimensional gesture interaction
4.1 Basic principles of three-dimensional gesture interaction technology
4.2 Three stages of the development of gesture recognition technology
4.3 The significance and application of the combination of three-dimensional gesture interaction technology and VR
4.4 All natural gesture interaction technology with more degrees of freedom
