From cloud native to intelligent, in-depth interpretation of the industry's first "best practice map of live video technology"

  stay 2022 Alibaba cloud live broadcast summit , Many technical experts and industry pioneers in the live broadcast industry , Discuss the evolution trend and future development of live video technology in the super video era . At the meeting , Alibaba cloud launched the industry's first 「 Best practice Atlas of live video technology 」, Summarize the live broadcast technology as 7 spot ： Cloud native 、 Highly reliable 、 Low delay 、 Ultra-high resolution 、 Intelligent 、 Professional and multi scene , This article will 「 Best practice Atlas of live video technology 」 In depth interpretation .

The trend of live video broadcasting is to minimize the delay , It includes transmission delay and calculation delay .

Speaking of delay , The public's understanding of delay mainly focuses on transmission delay , According to the delay of video , Video can be divided into on-demand 、 live broadcast 、 Lianmai interaction 、 Real time interaction, etc .

• When the transmission delay is 3-10 second , Such video has the property of being broadcast , Such as ： Live sports events ;
• When the transmission delay is 250-800 Between milliseconds , Can communicate 、 Interaction , Such as ： Interactive class, Lianmai, etc ;
• When the transmission delay decreases to 50-80 millisecond , At this time, the video is controllable and immersive , Such as ： Real time cloud 3D Rendering 、 Remote video control ……

Except for transmission delay , Video codec 、 High definition computing and other technologies will also bring computing power delay . Follow the trend of live broadcast , How to reduce the transmission delay and calculation delay , Bring technical support and imagination space for more live broadcast scenes ？

Alibaba cloud's live broadcast technology is based on the cloud's native base and distributed edge nodes , Through the transformation of the transmission protocol , Integrate real-time media processing power and edge computing power , It can greatly and effectively reduce the transmission delay and calculation delay , And through the global real-time streaming media transmission network GRTN（‍Global Real-time Transport Network）、 Ultra low delay live service RTS（Real-time Streaming）、 Real time media processing capability 、 video +AI And so on , Complete the best practice of low latency , Achieve the best balance between cost and experience , While bringing many general live broadcast solutions , Many scenario based solutions have also been derived .

This summit is the first in the industry 「 Best practice Atlas of live video technology 」, It is precipitated by Alibaba cloud's years of exploration and practice of live broadcast technology , Sum up as 7 Bigger ： Cloud native 、 Highly reliable 、 Low delay 、 Ultra-high resolution 、 Intelligent 、 Professional and multi scene .

Cloud native

Video technology is the best practice of cloud Nativity .

There are three main aspects of cloud Nativity advocated by Alibaba cloud ：“ Service oriented products ”,“ Random elasticity ”,“ Soft and hard integration 、 Cloud and edge 、 Cloud in one ”, And video technology is exactly the best practice of cloud Nativity .

Cloud infrastructure , Including the central node 、 Edge node 、CDN The network is the foundation to ensure large-scale distribution and transmission ; Cloud's original soft and hard integration , Can support CPU/GPU/FPGA/ASIC And other heterogeneous software and hardware solutions ; Close collaboration and computing power distribution between cloud and end , Can realize cloud 、 Mobile 、Web End 、PC The end rendering effect is consistent .

besides , The time of cloud origin 、 Space 、 Heterogeneous elasticity , It can not only support dozens of businesses , Cloud side computing quantification and flexible adjustment , Also can realize 100+ Real time transmission 、 Media processing 、AI Task multi machine heterogeneous mixed running , Bring unlimited computing power to the video business while making full and effective use of resources , Cut costs dramatically , Generate more new scenes .

Highly reliable

Hot videos have tens of millions of levels of real-time concurrency , High reliability is the most basic requirement .

Live video technology needs to be highly reliable , In particular, hot videos often bring millions 、 Tens of millions of concurrency , At this time, high reliability is the most basic requirement . Alibaba cloud's high reliability of video technology is mainly reflected in two aspects , One is to have full link logs in the architecture / monitoring / Call the police / Prediction and high reliability 、 Second level switching of multiple copies , It can realize intelligent automatic operation and maintenance and access network second level information troubleshooting , Bring cross center escape ability and disaster recovery service guarantee .

The second aspect of high reliability , Reflected in the improvement of weak network experience . Unique to Alibaba cloud QoS technology , It can accurately predict the bandwidth , Greatly improve bandwidth utilization and congestion control capability , At the same time, it combines the weak network sensing and packet loss resistance technology of the encoder , Can be in 70% It still achieves high definition and fluency in the packet loss state of . Intelligent speech packet loss compensation based on deep learning , It can improve the audio clarity in the weak network state , And the delay sensitive adaptive technology on and off the wheat , It can balance audio fluency and call delay in multiple scenes .QoS Technology can identify and dynamically adapt, such as ： Packet loss 、 Delay and other network scenarios , Greatly improve the end-users' subjective experience of audio and video business performance .

Low delay

GRTN Create the best streaming media practice scenario .

Delay refers to the time it takes for the screen of the anchor to be transmitted to the user's screen , When excluding networks 、 stream 、 Equipment performance , Choose the appropriate live streaming protocol in different live scenes , It can greatly reduce the delay of live broadcast . Review the history of live broadcast , It is also the history of live broadcast protocols , Mainstream agreements are familiar HLS、DASH、RTMP etc. , Delay is common in 5s above , Under the demand of strong interaction , The live broadcast protocol is also constantly transforming to low latency , such as ：SRT、LL-HLS etc. .

Alibaba cloud's best practices in low latency , Mainly in two aspects . First, at the network level , The traditional CDN The content distribution network is transformed into GRTN Global real time transmission network , Its positioning is based on the heterogeneous nodes of the central cloud and the edge cloud , Build ultra low latency 、 Fully distributed communication level streaming media transmission network .

GRTN Now it's a combination of live Internet and RTC Audio and video streaming transmission and exchange in various business scenarios , And has many other core technologies , Such as ：GRTN The two-way real-time signaling network can realize the millisecond transmission of network cut messages , When there is a media stream at the publishing end, the network switches , The client of the subscription to GRTN The switching behavior that happens internally is completely insensitive .

Second, here “ A net ” On , Alibaba cloud has created an ultra-low delay live broadcast service RTS（Real-Time Streaming）. be based on GRTN Short delay live broadcast of RTS Can support standards H5 WebRTC Push broadcast , In the case of tens of millions of concurrent cases, the delay can be controlled within 1s within ;RTC The end-to-end delay can be controlled in 250ms about . Watch below RTS and RTMP Live broadcast protocol comparison video , It can be found that when there is a certain packet loss rate ,RTS In the experience 、 Fluency and color are relative RTMP There are obvious advantages .

RTS And RTMP Delay comparison

Ultra-high resolution

The best compromise between cost and experience , Bring more immersion 、 More extreme audio and video experience .

About the practice of UHD in live video technology , Alibaba cloud self developed s265 Coding technology can achieve high image quality and low bit rate , And support 4K Real time coding ; Support AV1 code , a HEVC save 25% The above bit rate . Well known to all “ Narrowband HD ” technology , Narrow and high 1.0 Optimize multiple scenes , adopt RIO and JND Intelligent coding saves bit rate , Narrow and high 2.0 Adaptive video noise reduction and content restoration , Improve the subjective image quality of human eyes through color and texture enhancement , Bring the best compromise between experience and cost .

meanwhile , Alibaba cloud also optimizes the acquisition and coding transmission link in terms of live broadcast technology , Full link support 4K and 8K. On the engineering , Frame rate through various algorithms 、 Bit rate 、 The resolution of the 、 Color and other dimensions are improved , Whether it's an old movie 、 defects 、 A portrait 、 Or animation scene , Can be repaired to bring Ultra HD experience .

In addition to video processing in the cloud , It can also carry out super add / drop frames on the end side 、 Noise reduction 、 Color enhancement, etc , Even if it's not HDR The equipment , Through color enhancement SDR+ technology , It can also achieve a consistent end-to-side UHD experience .

End side Ultra HD contrast

Color enhancement SDR+ technology

Intelligent

In the age of Super Video , Intelligent audio and video is a major trend .

Deep learning can bring all kinds of AI The improvement of ability , It is the best exit in video practice . In terms of intelligence , Alibaba cloud's live video technology , In addition to the traditional intelligent dubbing 、 Intelligent strip removal 、 Smart collection , It can also audit audio and video content in real time , Accurate identification of anti riot and anti terrorist advertisements against pornography , It saves a lot of manual screening costs .

Trained virtual human technology , Support 3D Head portrait 、Live2D、 Stylized migration 、 Virtual anchor, etc , Bring more XR The evolution of Technology . Besides ,“ Intelligent ” It is also reflected in the audio experience , Based on the organic combination of deep learning technology and traditional signal processing 3A technology , It can realize intelligent noise reduction 、 Highlight the voice 、 No damage to music , And can be widely used in all kinds of real-time scenes . Intelligent voice super segmentation technology , It can still maintain high sound quality in the case of small models , These are all AI Combined with video .

“ Intelligent noise reduction ” Multi scene experience

speciality

speciality , Let the live broadcast gradually evolve into “ Intelligent broadcasting ”.

Alibaba cloud's professionalism in live broadcasting technology is reflected in multi bit rate 、 multi-protocol 、 Content protection and real-time production , The live broadcast gradually evolved into “ Intelligent broadcasting ”. It is worth mentioning that , In real-time production , Alibaba cloud reinvents the cloud of traditional broadcasters , Integrate real-time translation 、 Graphic packaging 、 Dynamic Tags 、 Advertising replacement and other guide innovation ability , Give consideration to the professionalism of live broadcast and the advantages of remote guidance .

meanwhile , Based on multi-channel real-time matting , Alibaba cloud has also “ Virtual studio ” Move to the Winter Olympics . Ali cloud, “ Cloud guide ” technology , Not only support a variety of devices 、 Multiple seats 、 Remote broadcasting , It can also realize double screen 、 Split screen 、 Picture in picture and other broadcast scenes , Close to the live broadcast demand to the greatest extent .

Interactive virtual studio helps the Winter Olympics

Alibaba cloud's professional combination of live broadcast technology “ Cloud guide ” Rich program production forms 、 Lower cost , It can be widely used in radio and television new media 、 Live broadcast of the event 、 Live broadcast of the event 、 In commercial live broadcast and other scenes , Help customers break business bottlenecks , Faster and better business .

《 this ！ Is the street dance 》 Cloud guide + Frame level multi view synchronization

Multi scene

“ live broadcast +” It has become a trend , Penetrate into all scenes .

From the perspective of the scene , The live broadcast is from the earliest large-scale style live broadcast 、 E-commerce live broadcast 、 Game live broadcast gradually penetrated into enterprise training 、 Online education 、 Radio and television new media scene . Alibaba cloud will broadcast live 、 on demand 、 Various algorithmic capabilities of online conferences are integrated into the same SDK Inside , While realizing the fusion of multiple scenes , Integrated SDK It can also be packaged on demand to achieve flexible customization .

From traditional SDK Access 、API Access to “ Low code live broadcast sample room ”, Alibaba cloud live broadcast is for e-commerce live 、 Online education 、 Enterprise live broadcast and other scenarios that provide one-stop access , Through simple three-step docking and a dozen lines of code , Let customers easily access the live broadcast experience , Help business development .

At present , Live broadcasting business has become an important part of digital social services , More and more content and industry turn “ live broadcast +” Pattern , The future picture of the development of live broadcasting technology will become clearer as the market demand changes .

「 Best practice Atlas of live video technology 」 It is based on Alibaba cloud's many years of exploration and best practices in live broadcasting technology , From the core of live broadcast technology , To the full scene coverage of the live broadcast , Then to the innovation and application of live broadcasting technology , Help enterprises have a deep understanding “ live broadcast ”, Break down technical barriers , Hand in hand with all walks of life, constantly changing and moving forward in the tide of interconnection of all things .

「 Video cloud technology 」 Your most noteworthy audio and video technology official account , Push practical technical articles from alicloud every week , Here to exchange views with first-class engineers in audio and video field . Official account back office reply 【 technology 】 You can join Alibaba cloud video cloud product technology exchange group , Discuss audio and video technology with big players in the industry , Get more up-to-date information about the industry .