The official promotional video of SteamVR shows us VR 's NB , but I don't know if you have noticed one of the key technologies: real-time green screen keying synthesis. This technology is very widely used in the film and television industry, and it has only recently been applied to real-time rendering and engines. For the effect, please refer to the video in the link (you need to bring your own ladder, if you can't climb the wall, please check the effect comparison chart below [the upper side is a clip of Avatar, and the lower side is a TV commercial]).
The effect is that the player is placed in the green background, the camera captures the player in real time and performs keying operations on it, and then combines it with the VR content that the player sees in real time, and finally outputs it to the monitor next to it.
So the question is, how to implement it in UE4?
The core part of keying is to cut out green and generate a suitable alpha channel, green is transparent, and the rest is opaque. It's simple to say, but there are a lot of theories in it, such as color overflow, unclean channels, how to deal with the color of the foreground itself that overlaps with the background, and so on. But these are clearly beyond the scope of this article.
The new version of UE4 (forgot to start with which version) has a built-in node called Chroma_Key_Alpha in Material which is specifically designed for this need. Here is the explanation and analysis of the principle of this node in the official forum . Next, we need to find a green screen image for testing, just open Google and search for GreenScreen Samples. Below is an example image I found
The clarity itself is still ok, we imported this image into UE4 and created the material according to the following effect:
Assign the shader to the object to get the effect after keying.
expand
Since we can key a static image, can we key a green (blue) screen video? of course can. For how to import and play videos in UE4, please refer to a blog written by the author before "How to Play Videos on UI" . It uses the newly added MediaFramework in 4.13. The material effect is shown in the figure:
The final playback effect of the video is shown in the following figure:
Combining the above effects, we can capture the green screen image in the camera in real time for keying, and then synthesize it with the content in VR to achieve the effect in the SteamVR promotional video at the beginning of the blog.