【VR】【Unity】How to implement remote screen projection function in VR?

【background】

The current mainstream VD applications are great for entertainment, but they still cannot operate the keyboard and mouse efficiently for work. Working with a virtual keyboard is obviously unrealistic. In order to allow my head display to function as a small-area replacement for multiple displays, I developed a screen-casting VR application myself.

[Ideas]

1. First implement the screencasting application in C#.
2. Study how to transcribe C# screen mirroring application into Unity 3D project.
3. Convert Unity3D project to VR project.
4. Introducing perspective capabilities into VR projects allows you to observe physical targets outside the screen and operate the keyboard and mouse smoothly.

【essence】

The following summarizes the essence of experience gained step by step in this process.

1. The pros and cons of using Unity on both the sending and receiving sides.
   I have tried to use Unity on both the sending and receiving ends, and used Unity's CaptureScreen command to obtain screenshots.
   The advantage of this method is that it is fast to implement, and packaging and unpacking are all Texture2D that Unity can handle directly.
   The limitation is that you can only get the screen in the Unity interface, but not the desktop screen. In other words, this method is only suitable for in-game screenshots.
   Screenshot code:

Texture2D screenshotTexture = ScreenCapture.CaptureScreenshotAsText