Build a live broadcast platform based on webrtc

be based on WebRTC Build a live broadcast platform

Live broadcasting can be said to be the hottest Internet Project in recent years , Major live broadcast platforms have sprung up , In the twinkling of an eye, the anchor industry has also become the most profitable pronoun . Then let's start from 0 Start building a live broadcast platform .

WebRTC

WebRTC, Name from web real time communication （Web Real-Time Communication）. It is a technology that supports real-time voice or video conversation in web browser , Google in 2010 Acquired in .2011 year 5 Open the source code of the project , Become the standard of next generation video call .

advantage

WebRTC As a real-time voice and video technology for web browser , The main advantages are as follows ：

• It has good generality , It can be used normally on almost any platform .
• Its use Interactive Connectivity Establishment(ICE) It can automatically match the current best communication mode between various devices , This is what many other technologies do not have .
• Full duplex capability , That is, two-way communication (P2P), It can not only be used as a one-way live broadcast, but also complete the two-way audio and video dialogue of electronic video conference .
• by Google its , It has a good development prospect , The most important ： Open source .

Start using

Guide pack

Nothing here direct Compile using the official native library , Because it's too much trouble , Some organizations on the Internet have provided compiled versions for our direct use , I don't know much about the provider of the Library , I don't know what kind of , But almost all on the market Android The library is used on both ends, so there should be no problem . Those who are interested can also go to the official website to compile .

implementation 'io.pristine:libjingle:11139@aar'

The official website is as follows ：

https://webrtc.org/native-code/android/

Quick start

The following steps are sorted out after classification and optimization , Add notes , Most of them can read , So I won't explain it in detail . If there is anything unclear, you can leave a message and ask me .

• initialization ,Peer Connection factory class ：

  // initialization   The key 
  PeerConnectionFactory.initializeAndroidGlobals(context, true, true, true)
  factory = PeerConnectionFactory()

• get Media, A simple understanding is the audio and video streams that need to be transmitted ：

  // Get video source 
  val videoCapture = VideoCapturerAndroid.create(CameraEnumerationAndroid.getNameOfBackFacingDevice())
  val videoSource = factory.createVideoSource(videoCapture, MediaConstraints())
  // Get audio source 
  val audioSource = factory.createAudioSource(MediaConstraints())
  // Get encapsulation MediaTrack
  val videoTrack = factory.createVideoTrack("ARDAMSv0", videoSource)
  val audioTrack = factory.createAudioTrack("ARDAMSa0", audioSource)
  // Encapsulate media streams 
  localMs = factory.createLocalMediaStream("ARDAMS")
  localMs?.addTrack(videoTrack)
  localMs?.addTrack(audioTrack)
  // preview 
  preview?.invoke(localMs!!)

• To configure ICE, Make a network connection ：

  //Ice NAT through 
  val iceList = ArrayList<PeerConnection.IceServer>()
  iceList.add(PeerConnection.IceServer("stun:23.21.150.121"))
  iceList.add(PeerConnection.IceServer("stun:stun.l.google.com:19302"))
  // Media restrictions 
  val constraints = MediaConstraints()
  constraints.mandatory.add(MediaConstraints.KeyValuePair("OfferToReceiveAudio", "true"))
  constraints.mandatory.add(MediaConstraints.KeyValuePair("OfferToReceiveVideo", "true"))
  constraints.optional.add(MediaConstraints.KeyValuePair("DtlsSrtpKeyAgreement", "true"))
  pc = factory.createPeerConnection(iceList, constraints, Observer())
  pc?.addStream(localMs)

  What we use here is stun The way ,ICE Logic judgment logic is very complex , There are mainly two ways ：STUN、TURN.

  STUN Is to convert local addresses into externally accessible Public address , To put it bluntly, it's a bit like Intranet through It means .

  TURN You need to build it yourself , from public Of The mediation Server assigned address .

  If you are interested, please see my next introduction ICE 's blog post .

• establish Offer：

  pc?.createOffer(Observer(), MediaConstraints())

  WebRTC In the model Peer Signaling exchange is required between , The carrier carrying signaling is Offer/Answer, This is because it is the live streaming end , So what was created is Offer, If it is a viewing end, you need to create Answer.

• Signaling exchange ：

  private inner class Observer : SdpObserver, PeerConnection.Observer {
  // establish offer success 
    override fun onCreateSuccess(p0: SessionDescription?) {
      log("offer:${p0?.description}")
      pc?.setLocalDescription(Observer(), p0)
      val jsonObject = JSONObject()
      jsonObject.put("id", "presenter")
      jsonObject.put("sdpOffer", p0?.description)
      SocketUtils.sendMsg(jsonObject.toString())
      log("json:$jsonObject")
    }
  
  // establish offer Failure 
    override fun onCreateFailure(p0: String?) {
      log("error: create offer $p0")
    }
  
  // When the network is available Ice Penetration success 
    override fun onIceCandidate(p0: IceCandidate?) {
      val jsonObject = JSONObject()
      jsonObject.put("id", "onIceCandidate")
      val ice = JSONObject()
      ice.put("sdpMid", p0?.sdpMid)
      ice.put("sdpMLineIndex", p0?.sdpMLineIndex)
      ice.put("usernameFragment", p0?.sdp?.substringAfter("ufrag ")?.substring(0, 4))
      ice.put("candidate", p0?.sdp)
      jsonObject.put("candidate", ice)
      SocketUtils.sendMsg(jsonObject.toString())
      log("iceCandidate:$p0")
      log("json:$jsonObject")
    }
  }

  adopt Socket get ：

    private fun setRemoteSdp(sdp: String) {
        val answer = SessionDescription(SessionDescription.Type.ANSWER, sdp)
        pc?.setRemoteDescription(Observer(), answer)
    }
  
    private fun setRemoteIce(ice: EventIceCandidate) {
        ice.candidate?.let {
            pc?.addIceCandidate(IceCandidate(
                    it.sdpMid,
                    it.sdpMLineIndex,
                    it.candidate
            ))
        }
    }

• display frame

        gl_view.preserveEGLContextOnPause = true
        gl_view.keepScreenOn = true
        VideoRendererGui.setView(gl_view) {
            RtcClient(this).preview = { localMs ->
                // local and remote render
                val localRender = VideoRendererGui.createGui(
                        LOCAL_X_CONNECTED, LOCAL_Y_CONNECTED,
                        LOCAL_WIDTH_CONNECTED, LOCAL_HEIGHT_CONNECTED,
                        RendererCommon.ScalingType.SCALE_ASPECT_FILL,
                        false)
                localMs.videoTracks[0].addRenderer(localRender)
            }
        }

  ​

Main process

Live end

• initialization ,Peer Connection factory class .
• obtain Media, establish GL Renderers , Enter the display ready state .
• To configure ICE And monitor , When Candidate After collection, send it to the remote end .
• establish Offer, When Offer After creation, send it to the remote end .
• Receive ICE Information , And set to PeerConnection object .
• Receive the... Returned by the remote end Answer, And set to PeerConnection object .
• Establishing a connection , Live broadcast .

Viewing terminal

• initialization ,Peer Connection factory class .
• obtain Media, establish GL Renderers , Enter the display ready state .
• To configure ICE And monitor , When Candidate After collection, send it to the remote end .
• Receive ICE Information , And set to PeerConnection object .
• Receive Offer, And set to PeerConnection object .
• establish Answer, When Answer After creation, send it to the remote end .
• Establishing a connection , Live broadcast .