There are many audio and video communication software on the market now, such as Facetime. Have you ever thought about developing your own audio and video communication software? In iOS development, audio and video communication development has always been difficult: expensive development costs and high technical thresholds have discouraged many developers and companies.
Today, Mr. Tuya will tell you how to build a complete iOS audio and video communication system from 0. For the server configuration issues involved in this article, you can go to the Tuya official website to view the relevant documents.
1. Preparation :
First, register on the official website of Tucodec: http://tucodec.com , and obtain the AppKey and AppSecret required in the SDK. After registration and login, the following figure is shown:
Figure 1 Obtaining AppKey and AppSecret after registration
Next, go to the developer and choose to download the required SDK in iOS.
Figure 2 SDK download display
Finally decompress, as shown in the following figure:
Figure 3 SDK decompressed content
Unzip voipDemo.zip , as shown in Figure 4:
Figure 4 Contents of voipDemo after decompression
Take out TYVoIPiOSSDK.framework, the framework is the video duck audio and video SDK, and import it into the required iOS project. Sometimes the framework path of the SDK is not automatically added to the imported framework project. Check the method: target —> Build Settings —> Search Path , add the path of TYVoIPiOSSDK.framework to Framework Search Path . Since the SDK is mixed with C++, as long as the .m file of the files in the SDK is used, it needs to be changed to the .mm suffix, as shown in Figure 5:
Figure 5 Display in the SDK import project
When using the TYVoIPiOSSDK.framework library, you need to import the following frameworks:
Figure 6 Display of libraries that the SDK depends on
After completing the above operations, the general framework required by our audio and video communication app has been completed.
After that, the code code completes the logical operation of the system UI interface drawing and the communication between users.
In the SDK provided by Tuya Technology, we can see that there are only two external header files, which shows that the rapid integration of audio and video communication is not that complicated. As shown in Figure 7:
Figure 7 Display of header files in SDK
Second, the header file definition:
<1> TYVoipDarwinManager.h file , which defines the classes and protocols provided to the outside world . The following three are the core classes and protocols in this file .
(1) TYVoipDarwinManager : VoIP main functions and management
(2) TYVoipVideoData : Receive and send video data model
(3) TYVoipDelegate : VoIP proxy
<2> TYVoipRender.h file: Provides rendering user view View.
After understanding the above-mentioned main interface files and their definitions, let's understand the communication principle in the SDK: In the SDK , all users exist in the form of communication nodes, and users who are nodes have their own unique representation - UserID , the entire SDK will Communication is carried out according to the UserID of each user .
For example, to communicate between A and B, suppose A's UserID is 401 and B's UserID is 402. After A logs in, he sends a message to B that I want to start a video with you, and B sends a message to A that I want to start a video with you after logging in. A Start rendering B's view, B starts rendering A's view. After completing all the above operations, A and B establish communication.
3. Introduction to the complete SDK usage steps :
<1> Configure voip and call it when the program starts:
[[TYVoipManager share] configVoip];
<2> Log in to the forwarding server (login once):
[[TYVoipDarwinManager sharedVoip] loginRelayServer:ip
serverPort:port
sessionId:sessionId
userId:_401
AppKey:AppKey
AppSecret:AppSecret];
<3> Set up voip proxy and start voip:
[[TYVoipDarwinManager sharedVoip] setDelegate:self];
[[TYVoipDarwinManager sharedVoip] startCallWithUserId:401];
<4> According to the logic, add (delete) the nodes that need to be called:
[[TYVoipDarwinManager sharedVoip] addClientNode:402];
//[[TYVoipDarwinManager sharedVoip] removeClientNode:402];
<5> Stop voip when you don't need to call:
//close voip
[[TYVoipDarwinManager sharedVoip] stopCall];
//Clear the existing connection node (no longer receive data from the node)
[[TYVoipDarwinManager sharedVoip] clearClientNodeList];
<6> Repeat steps 3, 4 and 5 according to the specific business logic
<7> Destroy voip at the end of the program
[[TYVoipDarwinManager sharedVoip] destroy];
4. Part of the code explanation: (interpreted from the implementation code of user A)
TYRenderView * preView; //Render a view
TYRenderView * otherView; //Render B's view
<1> Login operation:
A (UserID: 401) login operation, the return value is whether the login is successful (non-0 means success):
-(BOOL)login{
NSString * ip = @"**.**.**.**";//Required forwarding server address
uint16_t port = 0;
int sessionId = 0;
uint32_t AppKey = 0;//Fill in the AppKey and AppSecret you just applied for
uint32_t AppSecret = 0;
int res = [[TYVoipDarwinManager sharedVoip] loginRelayServer:ip
serverPort:port
sessionId:sessionId
userId:_401
AppKey:AppKey
AppSecret:AppSecret];
return res;
}
<2> Connection operation
A connection B operation:
-(void)startVoip{
/ / Set the required proxy in the communication The methods involved in the implementation are explained below
[[TYVoipDarwinManager sharedVoip] setDelegate:self];
//A joins the communication by himself
[[TYVoipDarwinManager sharedVoip] startCallWithUserId:401];
//Add node (add USerID of B)
[[TYVoipDarwinManager sharedVoip] addClientNode:402];
// view rendering
[self.preView startRendering];//Render a self
[self.otherView startRendering];//Render B view
}
<3> Disconnection operation
A performs a disconnection operation
-(void)stopVoip{
//close voip
[[TYVoipDarwinManager sharedVoip] stopCall];
//Clear the existing connection node (no longer receive data from the node)
[[TYVoipDarwinManager sharedVoip] clearClientNodeList];
//stop view rendering
[self.preView stopRendering];
[self.otherView stopRendering];
}
<4>A detailed explanation of the proxy method appears in the code
- (void)localVideoImage:(TYVoipVideoData *)image{//A picture processing
if ([self.preView isRenderring]){
[self.preView renderVoipVideoData:image];//Render a picture
}
}
- (void)remoteVideoImage:(TYVoipVideoData *)image{//B image processing
if ([self.otherView isRenderring]){
[self.otherView renderVoipVideoData:image]; //B screen rendering
}
}
- (void)previewAudio:(NSData *)data{//local voice
}
- (void)mixedAudio:(NSData *)data{//Other node voice, mix
}
The most important code in the project has been introduced, so let's test it and see our results.
Figure 8 Test renderings
At this point, building a complete set of iOS audio and video communication system is completed, and Tuyajun has realized two-way communication in the demo. The principle of multi-channel communication is exactly the same as that of the two channels. Let’s think about it and build multi-channel communication~