Two days ago, I talked about iOS and Android. Today, I will tell you how to build a video communication system on the Windows side.
1. Preparation:
1. First, register on the official website of Tucodec: http://tucodec.com to obtain the APPKey and AppSecret required in the SDK. After registration and login, the following figure is shown:
Figure 1 After registration, get the AppKey and AppSecret.
Click the SDK download button to jump to the download interface, and choose to download the SDK and Demo required in Windows.
Figure 2 SDK download display
As shown in the figure, we provide the vs2013 version and the vs2015 version of the SDK and the corresponding WinDemo.
Figure 3 SDK decompressed content
2. Download the visual studio development tool and install it. Because some libraries and compilation options that come with vs2013 and vs2015 are slightly different, you must choose the VS tool that matches the SDK for development.
3. After the installation is complete, double-click to open WinDemo.sln in Figure 3, you can see the project interface as shown in Figure 4, adjust the build type to Debug/x86, and then compile the WinDemo project.
Figure 4 Screenshot of WinDemo project
4. Click "local debugger" in vs to run the demo. The two parameters of the main function have been assigned in the project configuration options, as shown in Figure 5
Figure 5 Main parameter settings in the project
Or manually copy the files in the bin directory of opencv and Tucodec_SDK_Windows to the directory where the exe is located. Then open the "Command Prompt" or "Windows PowerShell" in the folder, give WinDemo.exe two parameters under the command line, which are your own ID and the ID of the other party, and then run it. As shown in Figure 6
Figure 6 Open WinDemo.exe in the command line, the effect is the same as above
5. SDK related instructions have been introduced in detail in the article "Instructions for the Use of VoIP SDK on Windows". This article will mainly introduce the code structure of WinDemo to facilitate developers to clarify the development process.
2. Introduction to WinDemo
In general, the audio and video communication system on the Windows side mainly includes three modules: audio and video collection, audio and video playback, and audio and video transmission that VoIP is responsible for. Using the SDK provided by Tuya Technology and the forwarding server code (http://www.tucodec.com) open sourced by Tuya Technology, developers can build their own audio and video communication server. Therefore, the main problem that developers face is the problem of audio and video capture and playback on the Windows side, which is also the focus of this article.
1. Video capture and playback
Experienced developers see the opencv directory in the compressed package and know that the video processing in this demo uses the help of opencv. Many complex tasks of hardware interaction. The CVideoCap and CVideoRender classes in the Demo use the opencv library for video capture and rendering.
Of course, opencv has a big disadvantage, which is that it consumes a lot of system resources. If the computer configuration is general, it will be found that the CPU load of the computer will increase sharply after opening the opencv to render the video. Therefore, if you want to use our SDK for product development, it is recommended to implement client-side capture and playback functions by yourself. Commonly used SDKs are DirectShow, SDL, DirectX SDK and other official SDKs.
2. Audio capture and playback
Since opencv cannot complete audio-related processing tasks, the Windows MultiMedia-related API provided by Microsoft is used in the demo to complete the audio capture and rendering functions. For details, see the CAudioCap and CAudioRender classes in the demo.
The class relationship between audio and video capture and playback is shown in the following figure:
Figure 7 UML class diagram for audio and video capture and playback
3. CVoipManager class
This class encapsulates the interaction between audio and video operations and VoIP, and is the core part of the entire Demo. Its class diagram is as follows:
Figure 8 CVoIPManager class diagram
Some methods and properties are introduced:
m_clientNetwork: ClientNetwork instance, responsible for forwarding server login
m_voip: TYVoip instance, responsible for data transmission
m_voipCallback: The interface implementation in TYVoip is the video callback interface of voip. The client obtains the video data from voip from this class
m_selfID: local usrID
m_sessionID: session ID
m_audioCap, m_audioRender, m_videoCap, m_videoRender: audio and video capture and rendering
m_ac, m_ai, m_vc, m_vi: audio and video format configuration
m_nodeList: list of added communication nodes
LoginServer: Log in to the forwarding server (the forwarding server needs to be configured in advance), because the forwarding server is used for data transmission or P2P punching, so when adding the other party as a node, make sure that both are online. Of course, this is done in the login module. For processing, if the other party is not online, it will keep requesting.
AddNote: Add a communication node
RemoveNote: remove a communication node
StartVoipTransmit: Start audio and video capture and rendering and transmission
EndVoipTransmit: End audio and video capture and rendering and transmission
3. Effect display
1. The login ID is 1, the added node is 2, but 2 is not online
It can be seen that the local video loop has been opened, and the console print information mainly includes:
login success: indicates that the login to the forwarding server is successful
add node 2: means adding node 2
get p2p addr of uid: 2 fail: request to add 2 nodes, but 2 nodes are not online yet
2. The local ID is 1, node 2 is added, and node 2 is online
usr1 local windows
usr2 windows
When the result shown in the figure above is achieved, the audio and video communication on the Windows side is completed.