The video communication system is often used in our daily life. Have you ever thought about building your own video communication system? Today, I will introduce to you the construction of two free Windows version video communication system SDKs.
Webrtc
The compilation of Webrtc ( webrtc.org ) has very high requirements on the performance of the development host and the network bandwidth. In addition, under the Windows platform, it also has clear requirements for the Windows system version and the Visual Studio version. As long as one of them has a problem, it may lead to failure. Therefore, it is necessary to determine the host and system configuration in the preparation process
1. Preparation
1. Requirements
1) Install VPN to ensure smooth network
2) 64-bit machine, 8G and above memory, 100G and above disk space, NTFS format;
3）安装 Visual Studio 2015 Update 3
4) Operating system: Windows 7 and above in English (note that windows can only be in English)
2. Download depot-tools
Download address: storage.googleapis.com/
Pay attention to setting the decompression folder, or the current folder will be too many files to be confused.
3. Configure the depot-tools environment variable
computer - properties - advanced - environment variables - path
Note: it must be placed at the beginning of the path environment variable
4. Configure depot-tools into system environment variables
depot_tools uses Google builds.
2. Download the source code
Create the webrtc directory:
mkdir webrtc-checkout
cd webrtc-checkout
fetch --nohooks webrtc
You can choose a directory:
git config branch.autosetupmerge always
git config branch.autosetuprebase always
Or create a new directory:
cd src
git checkout master
git new-branch your-branch-name
gclient sync
3. Compile
1. Get the Ninja project file:
Ninja is a build system developed by the Chromium community, which is very suitable for webrtc
2. Execute the following command to compile cd src for Webrtc
gn gen out/Default
After the compilation is successful, execute the following command to generate the vs project file
gn gen--ide=vs out\Default
At this point, we can find the project file all.sln in the src\out\Default\ directory. Because there are many documents for webrtc, it is strongly recommended that you go to the official website of webrtc and follow the steps of the official website step by step, don't worry. https://webrtc.org/native-code/development/
After talking about the webrtc solution, let's take a look at tucodec
Tucodec
1. Preparation:
1. First, register on the official website of Tucodec: http://tucodec.com to obtain the APPKey and AppSecret required in the SDK. After registration and login, the following figure shows:
Figure 1 Obtaining AppKey and AppSecret after registration
Click the SDK download button to jump to the download interface, and choose to download the SDK and Demo required in Windows.
Figure 2 SDK download display
As shown in the figure, we provide the SDK of the vs2013 version and the vs2015 version and the corresponding WinDemo.
Figure 3 SDK decompressed content
2. Download and install the visual studio development tools. Because some libraries and compilation options that come with vs2013 and vs2015 are slightly different, you must choose the vs tools that match the SDK for development.
3. After the installation is complete, double-click to open WinDemo.sln in Figure 3, you can see the project interface as shown in Figure 4, adjust the build type to Debug/x86, and then compile the WinDemo project.
Figure 4 Screenshot of WinDemo project
4. Click "local debugger" in vs to run the demo. The two parameters of the main function have been assigned in the project configuration options, as shown in Figure 5
Figure 5 Main parameter settings in the project
Or manually copy the files in the bin directory of opencv and Tucodec_SDK_Windows to the directory where the exe is located. Then open the "Command Prompt" or "Windows PowerShell" in the folder, and give WinDemo.exe two parameters under the command line, which are your own ID and the ID of the other party, and then run it. As shown in Figure 6
Figure 6 Open WinDemo.exe in the command line, the effect is the same as above
5. SDK related instructions have been introduced in detail in the article "Instructions for the Use of VoIP SDK on Windows Side". The next part of this article mainly introduces the code structure of WinDemo, which is convenient for developers to clarify the development process.
2. Introduction to WinDemo
In general, the audio and video communication system on the Windows side mainly includes three modules: audio and video collection, audio and video playback, and audio and video transmission that VoIP is responsible for. Using the SDK provided by Tuya Technology and the forwarding server code ( tucodec.com ) open sourced by Tuya Technology, developers can build their own audio and video communication server. Therefore, the main problem that developers face is the problem of audio and video capture and playback on the Windows side, which is also the focus of this article.
1. Video capture and playback
Experienced developers see the opencv directory in the compressed package and know that the video processing in this demo relies on the help of opencv. The advantage is that video capture and playback can be very convenient, shielding developers from Windows Many complex tasks of hardware interaction. The CVideoCap and CVideoRender classes in the Demo use the opencv library for video capture and rendering.
Of course, opencv has a big disadvantage, which is that it consumes a lot of system resources. If the computer configuration is general, it will be found that the CPU load of the computer will increase sharply after opening the opencv to render the video. Therefore, if you want to use our SDK for product development, it is recommended to implement client-side capture and playback functions by yourself. Commonly used ones are DirectShow, SDL, DirectX SDK and other officially provided SDKs.
2. Audio capture and playback
Since opencv cannot complete audio-related processing tasks, the Demo uses the Windows MultiMedia-related API provided by Microsoft to complete the audio capture and rendering functions. For details, please refer to the CAudioCap and CAudioRender classes in the Demo.
The class relationship between audio and video capture and playback is shown in the following figure:
Figure 7 UML class diagram for audio and video capture and playback
3. CVoipManager class
This class encapsulates the interaction between audio and video operations and VoIP, and is the core part of the entire Demo. Its class diagram is as follows:
Figure 8 CVoIPManager class diagram
Some methods and properties are introduced:
m_clientNetwork: ClientNetwork instance, responsible for forwarding server login
m_voip: TYVoip instance, responsible for data transmission
m_voipCallback: The interface implementation in TYVoip is the video callback interface of voip. The client obtains the video data from voip from this class
m_selfID: local usrID
m_sessionID: session ID
m_audioCap, m_audioRender, m_videoCap, m_videoRender: audio and video capture and rendering
m_ac, m_ai, m_vc, m_vi: audio and video format configuration
m_nodeList: list of added communication nodes
LoginServer: Log in to the forwarding server (the forwarding server needs to be configured in advance), because the forwarding server is used for data transmission or P2P punching, so when adding the other party as a node, make sure that both are online. Of course, this is done in the login module. For processing, if the other party is not online, it will keep requesting.
AddNote: Add a communication node
RemoveNote: remove a communication node
StartVoipTransmit: Start audio and video capture and rendering and transmission
EndVoipTransmit: End audio and video capture and rendering and transmission
Summarize
webrtc is used by more and more teams, but its source code of several gigabytes is also a headache. It is recommended that you search for the compilation results given by other teams on github. Friends who are energetic are better to choose to download it from the official website, Compile, after all, the information searched out is relatively old. As a new product, tucodec's documentation is not rich enough, and it also increases the difficulty of compilation. It is recommended that friends who are compiling can join their group, so that if you encounter problems during compilation, you can directly ask.