Webrtc crash course

source ：Youtube：Hussein Nasser Translation arrangement ： Lu bingcong In this video, the author outlines WebRTC And discusses some concepts in detail , You can learn about NAT、STUN、TURN、ICE、SDP And signaling （Signaling） Knowledge about , At the same time, the author also shows a Demo. Last , The author discusses WebRTC Advantages and disadvantages of , It also introduces some extended contents .

Catalog

• 1. WebRTC summary
• 2. WebRTC reveal
  • 2.1 Network Address Translation: NAT
  • 2.2 Session Traversal Utilities for NAT：STUN
  • 2.3 Traversal Using Relays around NAT: TURN
  • 2.4 Interactive Connectivity Establishment: ICE
  • 2.5 Session Description Protocol: SDP
  • 2.6 Signaling exchange ：Signaling
  • Workflow summary
• 3. Demo
• 4. WebRTC Advantages and disadvantages
  • 1. advantage
  • 2. shortcoming
• 5. Extended content
  • 5.1 Media API
  • 5.2 onIceCandidate and addIceCandidate
  • 5.3 Customize TURN and STUN The server
  • 5.4 public STUN The server

WebRTC (Web Real-Time Communication) It's a free one 、 Open source projects , Through a simple application programming interface (API) by Web Browsers and mobile applications provide real-time communication (RTC). It also shows that WebRTC The goal of the design is “ Design a way to pass as short as possible 、 The path with the lowest delay P2P Communication protocol , Provide a simple 、 For everyone to use API”. Once you put it in the browser , It's the standard ; Once it becomes the standard , Development will encounter “ friction ” Will disappear .

I track WebRTC This technology has been used for about two years , The audience provided me with a lot of high-quality resources , It also raises many excellent questions . In response to everyone's voice , I made this video , To provide you with a WebRTC Basic tutorial of . I will explain it in the following order ：

• WebRTC summary
• WebRTC reveal ：NAT、STUN、TURN、ICE、SDP、 Signaling
• Demo
• WebRTC Advantages and disadvantages
• Extended content

1. WebRTC summary

The first question that comes to mind is why we want to establish WebRTC？

The reason for establishing it is that people need to use a standard 、 Low latency way to deliver media data （ video & Audio ）. So-called “ The standard ” It means we need something simple and easy to use API; And so called “ Low latency ” It means that a suitable agreement is needed ,UDP Obviously a good choice , because UDP There is no excessive response process （Acknowledgment）. But we need more agreement than UDP Better , Be able to support P2P Communication for . Because once you rely on the server to deliver content, you will introduce additional delays due to reverse proxy or penetration , The user needs to terminate 、 Observe 、 Handle 、 Conversion flow and other operations , These will cause additional consumption . For video transmission 、 Especially live 、 Conversation and other scenarios , Users want content to arrive as soon as possible , therefore P2P Is the fastest path .

Besides ,WebRTC It also aims to achieve rich communication between browsers . Browsers have been around for a long time , it “ Have ” Lots of quality videos , It has access to cameras and microphones , These features are worth developing and utilizing . Users don't need to write their own applications , It's based on WebRTC Standards for API You can easily use . Not just browsers , On mobile devices and IoT The same is true when communicating with devices .

So in WebRTC What happened in the ？

for instance ,A Want to be with B communicate , but A And B Between “ Don't know each other ”. therefore A First you need to find all Public（ No B） The way to connect to it , Check A Whether there is a public IP Can be Public Identify or use , If there is no inspection A Whether your router allows public port forwarding rules 、 Whether there is a public representative on the route, etc .B Did the same thing above .

Besides ,A and B It also collects the encryption methods it supports 、 Safety parameters 、 Video codec and a lot of information , Note that this information has not been sent to the opposite end , At this stage, only extensive collection . All this information , Constitute the “SDP”.

Next ,A and B In other ways （ It can be WhatsApp、QR、Tweet、WebSockets、HTTP Fetch…） Send a conversation message , What exactly is this way WebRTC Not care , As long as you can from A To B（B To A） Can .

On the surface, this way of working is somewhat “ foolish ”, Some people may think “ Now that I have A and B The line of communication between , That's more WebRTC To do what? ？” But if you think about it carefully, you can find ,WebRTC As long as the first communication is exchanged SDP, Then we will realize the real P2P signal communication , No longer need WhatsApp、QR Wait, the middle way , There will be no faster communication path than this . So the final A Connected to... Through the optimal path B, This is it. WebRTC workflow .

A more detailed explanation of this example is as follows ：

As shown in the figure above , hypothesis A eureka A1、A2、A3 There are three ways to access it , At the same time, the safety parameters are also found 、 Media options and other information . meanwhile ,B Did the same job . Next , They do it in some way （ for example WhatsApp） Exchanged the above information . then A find B2 Is the best path available , and B Also found that A1 Is the best path available , Then the two will be directly connected to each other through this path . Essentially WebRTC That's how it works .

2. WebRTC reveal

Now we're right WebRTC Have an in-depth understanding , Tell the details . First understand NAT The details of the , Study WebRTC How to carry out correct network address translation ; Second, understand why we need STUN and TURN; In addition, we will introduce ICE、SDP And related contents of signaling exchange .

2.1 Network Address Translation: NAT

If you have an open Public IP Address , There will be no problem with the connection process . Because you'll be like Web The server listens to the port all the time , Connect the port to IP After providing them to each other , You can connect directly with it . But in most cases , Users are hidden behind the public network , Unable to connect directly . In the example shown below , The router has a Public IP 5.5.5.5, There is also a Private IP 10.0.0.1（ Also known as gateway）, Your machine has only one Private IP 10.0.0.2, But you want to visit IP by 4.4.4.4:80 Machine , How to achieve it ？

First, your machine will build a packet , Declare that you want to 4.4.4.4:80 issue GET request ,10.0.0.2 It is the source. IP Address . Next , Your machine will use the subnet mask to determine whether it can communicate directly with 4.4.4.4:80 Connect , The result of the operation will show 4.4.4.4:80 Not in your subnet , Therefore, it is impossible to communicate directly . So the next step is to send the request to the router , With the help of gateway communicate . The router will replace the source IP The address and port are Public IP And a random port , But before that... Will be created NAT surface , To record the corresponding relationship between the three . So the opposite end can receive your GET request , And subsequent treatment .

After this , The server 4.4.4.4:80 Will send a reply to your machine , The working principle is the same as the above , according to NAT The table queries the corresponding address to complete the communication .

NAT The main conversion methods are as follows , By default ,WebRTC Can support the first three NAT The way , Not friendly to the last . actually 90% The above communication is completed through the first three ways , Finally, the author personally believes that there is no use value .

1. one-on-one NAT（ Completely conical NAT）：One to One NAT（Full-cone NAT） On the router to send to the outside IP:port Packets can always be mapped internally IP:port , Without exception . Illustrate with examples , All sent to 5.5.5.5:3333 Packets are always automatically forwarded to 10.0.0.2:8992, Whether this bag comes from 4.4.4.4:80 Or any other address .
2. IP Restricted type NAT：Address restricted NAT For safety reasons , Some routers will address restrictions , Consider whether you have previously communicated with this address . That is, it should be sent to the outside on the router IP:port Data packets can be mapped to internal IP:port, The premise is that the source address of the packet is the same as NAT Table match , It doesn't matter what the port is . Illustrate with examples , Send to 5.5.5.5:3333 In the data package of , Only the source IP yes 4.4.4.4 Or recorded in other tables IP Will be automatically forwarded to 10.0.0.2:8992, Even if this IP It wasn't with 3333 Port for communication .
3. Port Limited NAT：Port restricted NAT Compared with the former , Added port restrictions , That is, it should be sent to the outside on the router IP:port Data packets can be mapped to internal IP:port, The premise is that the source of the packet IP and Port We have to work with NAT Table match . Illustrate with examples , Send to 5.5.5.5:3333 In the data package of , Only from 4.4.4.4:80 Or recorded in other tables IP:Port Will be automatically forwarded to 10.0.0.2:8992, Even if this IP:Port It wasn't with 3333 Port for communication .
4. symmetry NAT：Symmetric NAT This method is the one with the most restrictions , That is, it must match the complete IP:port, The difference is that it is sent to 5.5.5.5:3333 In the data package of , Only from 4.4.4.4:80 Will be automatically forwarded to 10.0.0.2:8992, Other packages cannot pass . This method cannot be used in WebRTC Use in , because WebRTC need STUN The server . once STUN The server establishes a Public representative ,Symmetric NAT It is required to communicate with only one specific peer , This restriction does not apply to WebRTC.

2.2 Session Traversal Utilities for NAT：STUN

STUN Can give an application what it needs Public IP and Port, Apply to Full-cone、Address restricted and Port restricted NAT, Can't be used for Symmetric NAT.STUN The server is usually in the 3478 Running on Port ,TLS Port is 5349.STUN It's very lightweight , Users can use docker Build a STUN The server .STUN The purpose of the server is to let users find their own Public Express , And through this Public Indicates communicating with other users . If we use something like 1996 Years or 2000 Like in the early years of Public IP Address , Communication will also be very simple . But for now , We have to use STUN The server .STUN The workflow of the server is shown in the figure below ：

First, create a data packet for STUN request ,STUN The address of the server is 9.9.9.9:3478, Also created in the router NAT Table and address translation , Then the packet is sent to STUN The server .

After the server receives the request , by 10.0.0.2 Our machine built a Public Express 5.5.5.5:3333, And package this information into a data packet for feedback .

The above is a STUN The detailed process of the request , The following is an example STUN In the whole communication process, the following work is carried out ： First give 10.0.0.2 A machine for Public Express 5.5.5.5:3333, At the same time 192.168.1.2 A machine for Public Express 7.7.7.7:4444. Then both use the obtained Public Indicates connection .

It is worth noting that , The two have not communicated before . If it is Full-cone NAT, So there's no problem connecting ; If it is Address restricted NAT, The first request to connect will fail . under these circumstances , The user needs to establish at least one communication request through the server , First, let both addresses be saved in the routers at both ends , So again through Public Indicates that a matching address can be found when making a connection request , Then you can complete the connection .Port restricted NAT The working principle is similar .

2.3 Traversal Using Relays around NAT: TURN

In the application Symmetric NAT Under the circumstances , You have to use TURN. All communications must go through TURN Server forwarding , therefore TURN The maintenance cost of the server is relatively high , That's why few people offer such servers for free . Here is a TURN Example of server workflow , There is no direct relationship between the two P2P signal communication , All the information has gone through TURN The server forwards .

2.4 Interactive Connectivity Establishment: ICE

A lot has been established in STUN and TURN After the server , from A To B There are many choices in the path between , To better handle these paths , People have come up with ICE.ICE All available communication paths will be collected as “ The candidate ”（ICE Candidates）, It could be local IP Address 、STUN and TURN The address provided by the server, etc . All the addresses collected will be put into SDP in , And then to the opposite end , End to end through parsing SDP To learn about the important information we provide . therefore ,ICE yes WebRTC A very key component of .

2.5 Session Description Protocol: SDP

SDP Is used to express ICE Candidates The format of , It describes the network options 、 Media options 、 Security options and a lot of other information , Developers can even customize SDP Content . actually SDP It's not an agreement , Just a data format , but SDP yes WebRTC One of the most important concepts in . It is designed to integrate user generated SDP Send to other end , The way of delivery doesn't care .

2.6 Signaling exchange ：Signaling

Signaling The process is generated by the user SDP In some way to the party who wants to communicate , As mentioned above , It doesn't matter how it is delivered . A lot of people go through Websockets perhaps socket io To pass on SDP Information , This process is Signal SDP. Just find all the ICE candidate It's time-consuming , But once this process is completed , The next step is to create a SDP, And then generate a QR code And put QR code Published to twitter On , Others can scan the QR code to get the corresponding SDP. The process is through twitter、QR code、Whatsapp、WebSockets、 still HTTP Requests don't matter , Because in fact, it is just passing a long string to others . In short ,Signaling Will be SDP Pass the information to the other party .

Workflow summary

1. A Want and B Establishing a connection ;
2. A Created a offer, It looks for all ICE candidate、 Security options 、 Audio and video options, etc. and create SDP, In short, this offer Namely SDP;
3. A take SDP Signaling is passed to B（Signaling）;
4. B according to A Of offer Set it up , And create a reply （answer）;
5. B take Answer Signaling is passed to A（Signaling）;
6. Connection is established .

3. Demo

The author tells a story in detail Demo Programming , The program can ：

1. Communicate between two browsers （ browser A And browser B）;
2. A Create a offer（SDP）, And set it to local description ;
3. B Receive one offer And set it to remote description ;
4. B Create a answer And set it to local description , And pass it on to A;
5. A receive answer And set it to remote description ;
6. Establishing a connection 、 Building data channels 、 Exchange data .

Source code ：https://github.com/hnasr/javascript_playground/tree/master/webrtc

4. WebRTC Advantages and disadvantages

1. advantage

1. P2P Communication is great , For high bandwidth content, there can be reduced latency .P2P Is the fastest path , There is no need to communicate with other third parties . Even if the transmission through the Internet needs to go through a large number of routers , But if the content has been encrypted, all routers will not view the content , They deliver packets directly , therefore P2P It's a very good way to communicate . For high bandwidth content , They use UDP Directly be “ Send in ” and “ Introduction ”, adopt P2P UDP Pass on these contents （ Especially video content ）, Users will get the best performance .
2. Standards available API WebRTC There is a very standard 、 Very elegant API, Can be applied directly in the browser , There is no need to install other packages 、 There is no need to use redundant development tools .

2. shortcoming

1. Need to maintain STUN and TURN The server In some cases P2P Can't work , You still need one TURN The server . But maintenance STUN and TURN The server requires a lot of manpower and material resources , especially TURN The server . Because you have to spend money to maintain a Public IP, And the server must be maintained so that it can start and run normally . The author personally believes that instead of spending this price , It's better to build a server with full control , Reverse proxy .
2. When there are too many participants ,P2P It will break down Suppose there is 100 Individuals want to communicate with each other , You will create P2P Is it connected ？ That would be hundreds by hundreds of connections , Because everyone needs to connect to any other user , This will be very large-scale . But if you have a centralized server , Each user only needs to establish a connection with this server , You can control all traffic through this server , This is obviously a better way . therefore WebRTC Sometimes it can't be used in games , You can't take advantage of WebRTC To create a multi-user game , Of course 3 One user can , But hundreds of users, the author believes, are impossible to achieve .

5. Extended content

5.1 Media API

getUserMedia Function can be used to get microphone and camera , And then get a stream （stream）, The content of this stream will pass through RTCPConnection.addTrack(stream) Send in RTC Connecting . In theory, you can use data channels to transfer any type of data , But if you want to deliver media information, you need to use stream, These data will be delivered using different protocols .

For more information, please refer to ：https://www.html5rocks.com/en/tutorials/webrtc/basics/

5.2 onIceCandidate and addIceCandidate

These two functions can be used in the new Candidate Maintain connections when joining or leaving . Users get one at a time from the system ICE Candidate when ,onIceCandidate The function will be called .onIceCandidate The function will tell the user “ stay SDP After it has been created , There's a new one Candidate”. new Candidate Will be told to the opposite end , The way to inform can be Signaling, You can also go directly through the same SDP Connect . End to end through addIceCandidate The function will the new Candidate Join in SDP.

5.3 Customize TURN and STUN The server

Creating RTCP When the connection , You can choose to transfer configuration information , The following figure shows an example of configuration information . Basically, users can customize ICE The server , There are many options .

Besides , An open source library can also help you create your own TURN The server , Address ：https://github.com/coturn/coturn

5.4 public STUN The server

The author gives some Google Public server provided , It can be used as a reference for developers ：

• stun1.l.google.com:19302
• stun2.l.google.com:19302
• stun3.l.google.com:19302
• stun4.l.google.com:19302
• stun.stunprotocol.org:3478

Attach speech video ：

http://mpvideo.qpic.cn/0bc33qaceaaa2aahncygvrrfbxgdeloaaiqa.f10002.mp4?dis_k=e6c887c1460e5f17cc844ece0baea03b&dis_t=1645153755&vid=wxv_2264750176378765323&format_id=10002&support_redirect=0&mmversion=false