Game developers typically run Unreal Engine applications on the same device that renders the game to the screen while running the game logic. Multiplayer networked games may distribute parts of the game logic among multiple instances of the application, but each individual The instance will still render the game locally for its own players. Even if you use the HTML5 deployment option to create a version of your project that can run in a web browser, the game logic and graphics still run locally in each user's web browser.
However, using Pixel Streaming , UE-developed applications can be run remotely on a computer that is invisible to the user. Unreal Engine uses the resources available to that computer (CPU, GPU, memory, etc.) to run game logic and render each frame. It continuously encodes this rendered output into a media stream that passes through a lightweight web services stack. Users can then view the broadcast stream in standard web browsers running on other computers and mobile devices.
The result for the user is just like watching a video stream from a service like YouTube or Netflix, but with two differences
- Instead of pre-recorded video clips, the stream plays rendered frames and audio generated by Unreal Engine in real time.
- Users can control interactions from their browser, sending keyboard, mouse, and touch events back to Unreal Engine, as well as custom events emitted from the player web page.
Advantages of UE Pixel Streaming
- Pixel streaming enables mobile devices and lightweight web browsers to display better graphics quality than would otherwise be possible. They can display complex scenes at high resolution, and through pixel streaming, users can run rendering capabilities that would otherwise only be possible when rendering in native desktop applications with powerful GPUs.
- Users do not need to download large executable or content files in advance, nor do they need to install anything. The only thing the user needs to download is the media stream to play.
- Multiple platforms can be supported without the need to create and distribute multiple separate packages. Just package the application once for Windows or Linux and people can experience UE content using any platform. Users can view the stream in any modern browser that supports the WebRTC connection model, including browsers on desktop, iOS, and Android platforms.
- The Pixel Streaming system contains a minimal number of components and is relatively easy for anyone to set up on a local network. However, for teams with experience deploying web services, it is powerful enough to be used as a basis for creating a custom cloud hosting platform.
- Pixel Streaming uses the WebRTC peer-to-peer communication framework to minimize latency between users and Unreal Engine applications.
The image below briefly summarizes the Pixel Streaming plugin
Pixel streaming technology framework
The framework of pixel streaming technology consists of two parts: pixel streaming plug-in Pixel Streaming, signaling and web server.
- Pixel Streaming Plugin - This plugin runs in Unreal Engine. It encodes the final result of each rendered frame using video compression, packages those video frames along with game audio into a media stream, and sends that stream to one or more connected browsers over a direct point-to-point connection.
- Signaling and Web Server - The Signaling and Web Server is responsible for negotiating the connection between the browser and the Pixel Streaming plug-in and providing the browser with the HTML and JavaScript environment for playing the media stream.
Pixel push streaming connection method
- When starting all Pixel Streaming plug-ins, the Pixel Streaming plug-in running in Unreal Engine first establishes a connection to the signaling and network servers.
- The client connects to the signal server, which serves it an HTML page that contains the player widget and control code written in JavaScript.
- When a user initiates a stream, the signaling server negotiates a direct connection between the client browser and the Unreal Engine application. In order for this connection to work properly, the browser and Unreal Engine application need to know each other's IP addresses. If both are running on the same network, they are usually directly visible to each other at their respective IP addresses. However, a Network Address Translation (NAT) service running between the two endpoints may change the externally visible IP address of either party. Solving this problem usually involves using a STUN or TURN server, which tells each component what its externally visible IP address is. Once a connection is established between the client and the Unreal Engine application, the Pixel Streaming plugin begins streaming media directly to the browser. Input from the client is sent directly back to the Unreal Engine application by the player page's JavaScript environment.
- The signaling and web server remains connected to the browser and Unreal Engine application even after the media stream starts playing, so it can handle browser-initiated disconnections.
Limitations of UE pixel push streaming
First of all, what Pixel Stream officially provides is not a product and cannot be used directly. Secondly, it only applies to the UE model, and furthermore, its compatibility with different browsers is not very good in practice. The 3DCAT real-time cloud rendering technology not only supports 3D engine models, but also supports some non-3D engines such as some large home decoration software, PS, BIM, etc. There is also a lot of load balancing, scheduling and other information for operations, which is more mature and product-oriented, ready to use, and can support software deployment and SDK docking.
Advantages of 3DCAT real-time cloud rendering
First of all, 3DCAT real-time cloud rendering does not restrict any access engine. It supports more than 10 standard executable files output by domestic and foreign 3D production engines. The biggest advantage is that it does not choose the engine. Secondly, 3DCAT has been used in multiple commercial production projects and continues to expand and improve its functions to provide the most stable real-time cloud rendering service. In addition, all modules of 3DCAT support customized development. 3DCAT also has powerful public cloud services, with edge nodes all over the country, and has more than 30,000 nodes, which is not comparable to local single-machine deployment.
This article "What is UE pixel streaming and what is the principle of pixel streaming?" 》The content is compiled and released by the 3DCAT real-time rendering cloud solution provider. If you need to reprint, please indicate the source and link.