Everyone knows that there are nominations for Oscars. In fact, there are nominations in ICE of WebRTC. There are regular nominations and radical nominations, and the nominated candidates are not necessarily the best candidates. This article will take you to explore the mystery. The content of the article mainly describes the ICE-related status and ICE nomination mechanism in RFC 5245 , and analyzes it in conjunction with the libnice (0.14) version.
Author: Diagrams, Ali cloud development engineer
revision: a Thai, Ali cloud Senior Development Engineer
Scene
When analyzing a problem, I encountered this scenario: a Candidate on the server and three Candidates with different priorities on the client, but in the end a pair with the lowest priority was actually selected.
The server has a Relay Candidate, port 50217.
a=candidate:3 1 udp 503316991 11.135.171.187 50217 typ relay raddr 10.101.107.25 rport 40821
The client has three Candidates, ports 50218 (middle priority), 50219 (lowest priority), and 50220 (highest priority).
Candidate 1:
candidate:592388294 1 udp 47563391 11.135.171.187 50219 typ relay raddr 0.0.0.0 rport 0 generation 0 ufrag fO75 network-cost 50
Candidate 2:
candidate:592388294 1 udp 48562623 11.135.171.187 50218 typ relay raddr 0.0.0.0 rport 0 generation 0 ufrag fO75 network-cost 50
Candidate 3:
candidate:592388294 1 udp 49562879 11.135.171.187 50220 typ relay raddr 0.0.0.0 rport 0 generation 0 ufrag fO75 network-cost 50But the last choice is the lowest priority Pair, 50219.
Remote selected pair: 1:1 592388294 UDP 11.135.171.187:50219 RELAYED
Candidate's Foundation
Candidate's Foundation: Let me mention Foundation first, which will involve Frozen status.
For the same channel, there may be different Candidates. For example, when Relay Candidate is discovered, a new Server Reflexive type Candidate can be generated, but they are all based on the same local address (IP, port) and protocol, then It can be considered that these networks are similar, and they will have the same Foundation. Among them, Foundation is the first field in SDP, which is '7' in the following example.
a=candidate:7 1 udp 503316991 11.178.68.36 51571 typ relay raddr 30.40.198.7 rport 55896
ICE States
ICE mainly has the following five states, of which the first four are normal states, and the fifth state Frozen involves ICE Frozen Algorithm .
The five states of ICE:
- Waiting: When the connectivity check has not yet started (the Binding Request has not been sent).
- In Progress: When the connectivity check is sent, but the corresponding check transaction is still in progress (Binding Request has been sent).
- Successed: The connectivity check is completed and the result is returned successfully (Binding Request completed).
- Failed: The connectivity check has been executed and the result has failed (Binding Request has been completed).
-
Frozen:, all Candidate Pairs are in this state after the initialization is completed. For the same Foundation (similar Candidate), one Pair will be selected according to the priority, Unfreeze, and set to Waiting state, while the others remain Frozen. Until the selected pair is completed, Unfreeze another pair will continue.
ICE Nomination
ICE has two nomination methods:
1.Regular Nomination
For regular nomination, the main workflow is as follows:
L R
- -
STUN request -> \ L's
<- STUN response / check
<- STUN request \ R's
STUN response -> / check
STUN request + flag -> \ L's
<- STUN response / check
Regular Nomination The Agent in the Controlling mode initiates a Binding Request and receives the response from the opposite end. At the same time, the Connective Check initiated by the opposite end is completed. The Controling end will again send a Binding Request with the USE_CANDIDATE flag bit. When the Controlled end receives it, it will accept the nomination. .
2.Aggressive nomination
In addition to the regular nomination, there is also a more radical nomination. A handshake will be added to the regular nomination.
L R
- -
STUN request + flag -> \ L's
<- STUN response / check
<- STUN request \ R's
STUN response -> / check
Figure 5: Aggressive NominationThe Agent in Controlling mode initiates a Binding Request, but this Binding Request will directly carry the USE_CANDIDATE flag bit. The Agent in Controlled mode will accept the nomination after receiving it. In the aggressive nomination mode, a handshake process can be saved, but when multiple pairs accept nominations at the same time, they will be selected according to their respective priorities, and the pair with the highest priority will be selected as the actual channel.
real case scenario:
Updating States When Nomination
When a new nomination is generated, the internal state of ICE will be changed accordingly.
When the Binding Request at one end carries the Use Candidate flag, a nomination (Nomination) will be generated.
Regardless of the Agent in Controlling or Controlled mode, the recommended status update rules for handling nomination are as follows:
- If there is no nominated Pair, continue the connectivity check process.
- If there is at least one valid nomination:
- The Agent must delete all Waiting and Frozen pairs under the Component.
- For a pair in the In Progress state, if the priority is lower than the priority of the currently nominated pair, stop retransmission (cancel).
-
When all Componts of a certain Stream have at least one nomination, and the check is still in progress:
- The Agent must mark the Stream as completed.
- Agent can start to transmit media stream.
- The Agent must continuously respond to received messages.
- The agent must retransmit the pair that is currently in In Progress (the priority is higher than the one currently nominated, otherwise it has been deleted or cancelled).
-
When all pairs in the check list are completed:
- ICE is complete.
- Controlling Agent updates Offer based on priority (it seems that WebRTC does not have this step).
-
When the checklist check fails:
- When all pairs fail, turn off ICE.
- When the check of a certain stream is successful, the Controlling Agent removes the failed pair and updates the Offer.
- If some checks are not completed, ICE continues.
Scheduling Checks
When describing the nomination, it will also involve ICE's scheduling of the Pair (when the valid Candidate is still In Progress but the other Candidate's Pair has received the Binding Request).
Only Full is discussed here, and the Lite mode is not described first.
ICE's Checks are divided into two types, Ordinary Checks And Triggered Checks.
- Ordinary Checks are regular pair checks, which means that these pair checks are state checks switched from the normal process.
- Triggered Check is a passively triggered check. Although these pairs are still in a state where they cannot be checked, they receive a connectivity check from the opposite end. At this time, the pair will be speeded up and put directly into the scheduling list.
When ICE establishes a Check List (one for each Stream), a timer will be added to each Check List. When the timer arrives, the following scheduling will be performed:
Note: It’s a bit uncomprehending here, the whole process seems to be serial, and the activation speed is a bit slow.
- First schedule Triggered Check and execute it.
- If not, dispatch the Waiting state pair with the highest priority, send Request, and set the state to In Progress.
- If not, find the Frozen state pair with the highest priority from the Check List, Unfreeze it, and send a Request with the state set to In Progress.
-
If not, terminate the scheduling.
Case Analyzed
After a brief understanding of the ICE process, we return to the original Case.
First look at Add Candidate. The three Remote Candidates are added in a different order, followed by 50219, 50218, and 50220. Note that at this time, 50219 has received the Binding Request from the opposite end, aggressively nominated, and carried USE_CANDIDATE, so the Create Permission will be executed and completed soon. You can start sending Binding Request, which belongs to Triggered Checks priority scheduling. Send Binding Request and enter the In Progress state.
Note: In addition to the pair of the local Relay, there is also a Candidate of the local Host type that communicates with Turn. 
Then in 50219, when the other two Create Permissions were not completed, the Check was completed quickly. According to the description in rfc 8.1.2, all Pairs that are not in the In Progress state are deleted, and their priority is not referred to. Therefore, the pair with the lowest priority of 50219 was finally selected.
"Video Cloud Technology" Your most noteworthy audio and video technology official account, pushes practical technical articles from the front line of Alibaba Cloud every week, and exchanges ideas with first-class engineers in the audio and video field.