Client construction and Optimization Practice

OSC The second live broadcast of the editorial department's observation series 《 Mysterious big factory open source office 》6 month 20 Friday night 19 Go online on time >>>

”

8 month 5 Japan -6 Japan ,LiveVideoStackCon 2022 Audio and Video Technology Conference Shanghai Station , Open the door to the future with you .

Client construction and tuning practice

soft / Hard encoding 、 decode , Over score , All kinds of special effects , How to give full play to the performance of high-end models at the same time , Ensure smooth operation of medium and low-end machines , This requires research and development 、 Close cooperation between operation and product team . This topic will discuss the client and Web Experience in realizing efficient R & D and high-performance multimedia capabilities at the end .

Lecturers and topics

In recent years, video and conference products have received great attention . The user experience of the product , Function and quality determine whether the product can stand out in the competition . How to choose a good architecture and solution is the most critical factor .

This sharing will be divided into several parts ： The first part introduces the history and current situation of video conference products , And the main functional elements . The second part will introduce based on WebRTC Solutions and advantages and disadvantages , And the main challenges . The third part introduces RingCentrl Solution of video conference system , And the main technical advantages .

Hope to share , For how to build an excellent video conference product, there are certain gains in architecture selection and technology selection .

AI The landing of technology is gradually from the server side 、 Cloud landing , Step by step to the mobile terminal and edge devices . These years, with AI Technological progress , The lightweight algorithm model starts to run in real time on the mobile terminal , And the mobile terminal algorithm is also being iterated and improved , And for real-time live scenes , There are more and more AI Landing algorithm , Such as ： Face based entertainment 、 Body posture recognition 、 Background replacement, etc . As the algorithm continues to improve 、 The continuous improvement of hardware capability , be based on AI Real time video enhancement processing technology is also applied to real-time live broadcast or video call scenes , And for many scenes involving privacy , More and more users expect AI Technology can be localized , Instead of going through the cloud . Video cloud transcoding that consumes more computing power , When facing the localization demand of mobile terminal, I don't follow my heart .

ZEGO That is, structure technology has put forward a set of extremely lightweight AI Algorithm model , Combined with the hardware characteristics of the mobile terminal , Differential optimization feedforward inference base , Let algorithm model 、 Inference base 、 Hardware becomes one , It makes it possible for the video cloud transcoding mobile terminal .

With B The continuous expansion of station business form , Stability of video playback experience in different scenes 、 Fluency puts forward higher requirements . However, due to the thousands of people and thousands of faces demanded by users 、 The network environment is changeable and complex , How to provide users with a better playback experience , This gives B The broadcasting capacity infrastructure of the station poses more challenges .

This sharing will be divided into three parts , Part I Introduction B The station is based on the dynamic results of user scoring experiment QoE Model , The model is of great significance to guide the direction of business evolution , The second part introduces the key points and pain points in the optimization of on-demand video experience , Mainly from two aspects: user tendency and network transmission , The third part introduces B The station uses intelligent technology to optimize playback QoE Practice , How to provide users with a higher definition experience without adding Caton , How to quickly avoid server quality degradation based on end-to-end data . Through the above three parts , To introduce B Standing on demand QoE Experience and achievements in optimization .

The upper part of the video super split terminal is equipped with stream saving function 、 Real time and other advantages . However , As the resolution increases, the computing power and storage cost increase exponentially , Without relying on hardware platform optimization , How to break through the computing power limitation of mobile terminal 、 Keep super real-time 、 Achieve high model coverage , And obtaining obvious super score effect is the difficulty and challenge that real-time super score landing on high-resolution video terminal must face . This sharing will start with model optimization 、 Engineering optimization 、 Model coverage 、 In terms of frame rate guarantee, this paper introduces the solutions and best practices adopted in the landing of real-time super division capability on the public comment terminal .

 

---

In the scan QR code Or click on Read the original

Learn more about the conference

If you like our content, just order “ Looking at ” Well ！

This article is from WeChat official account. - LiveVideoStack（livevideostack）.
If there is any infringement , Please contact the [email protected] Delete .
Participation of this paper “OSC Source creation plan ”, You are welcome to join us , share .