Dialogue with Google technical experts: soundstream is expected to be used for general audio coding in the future

Click on the above “LiveVideoStack” Pay attention to our

In the near future , Google has launched a product based on AI Audio codec for ——SoundStream. According to Google ,SoundStream It's the first one that can encode different sound types 、 At the same time, it provides high-quality audio and can be used on smart phones CPU Neural network codec running in real time on . Earlier this year , Google has released a product called Lyra Ultra low bit rate audio compression codec . Within a year , Google has launched two models based on AI Audio codec for . What are the differences between these two codecs ？ Why is Google so focused on low bit rate audio compression ？SoundStream Whether it will become a general audio codec , Or just focus on specific areas ？ new edition Lyra Is it possible to replace Opus?

SoundStream

Technical interview

#004#

With these questions ,LiveVideoStack Interviewed the person in charge SoundStream Research and development of audio codec Senior Product Manager Jamieson Brettle and Senior Software Engineer Jan Skoglund.

LiveVideoStack: Jamieson、Jan, How do you do . Congratulations to Google on SoundStream Achievements in .SoundStream The launch of is a big news in the field of audio and video technology , Chinese audio engineers are also closely watching its progress . In order to let everyone know more about this new AI Audio codec , We prepared some questions , Please answer .

------

Q1： Now people have more and more bandwidth , Why should google focus on low bit rate audio compression ？

Jamieson&Jan: Although the infrastructure continues to improve , But it still takes time for the Internet to become fully popular . besides , The demand of users and applications for bandwidth means that even if the available bandwidth continues to increase , Demand is still greater than supply . therefore , We will try our best to reduce bandwidth consumption , So as to improve the overall user experience .

Q2: The new SoundStream And the neural network audio codec released earlier this year Lyra What's the main difference ？

Jamieson&Jan: The first edition Lyra A method based on WaveRNN Built in synthesis engine , and SoundStream A network similar to an automatic encoder is used .SoundStream Will be a new version Lyra Core technologies .

Q3： Why did Google develop two AI codecs ——SoundStream and Lyra？ Google's response to this Roadmap Can you tell ？SoundStream How to integrate into Lyra in ？

Jamieson&Jan: Use ML Audio coding is still in its infancy , With the increasing research in this field , We see that AI The rapid development of codec . Through ongoing projects , We can quickly turn research into products , Apply the best codec to practical application .Lyra Future versions of will use SoundStream As the underlying engine . thus , Today's developers can still use the same Lyra API, But you can get significantly improved performance .

Q4： From the paper ,SoundStream Whether it's sound quality （ At the same bit rate ） Or for all kinds of audio signals （ voice 、 music 、 No noise and noise ） The robustness of , Or algorithm delay , Or the computational complexity has gone beyond Lyra 了 .Lyra Whether it will be completely replaced ？

Jamieson&Jan: We see SoundStream In terms of sound quality 、 Robustness to noise and processing of various audio signals , There has been great progress . As a new version Lyra Core technologies , new SoundStream The engine will be replace The first edition Lyra Autoregressive engine in .

Q5： From the experimental results of the paper ,12kbps Of SoundStream Performance seems to be approaching saturation .Google Do you think AI Audio coding is only applicable to low rate scenes ？ At medium and high rates （ Such as AAC Typical rate ）AI Is there any chance for audio coding to surpass traditional coding ？

Jamieson&Jan: We think AI Codec will benefit all kinds of bandwidth and applications . We are now working to improve neural network based audio coding at a higher bit rate .

Q6：SoundStream Whether it is also applicable to voice at low rate 、 Encoding and decoding of music and mixed signals ？

Jamieson&Jan: SoundStream There is no classification of sound types , It can handle different sounds at the same time .

Q7： Whether the neural network codec has obvious advantages in complexity compared with the traditional signal processing codec ？

Jamieson&Jan: up to now , In neural network codec , The coding complexity is low , The complexity of decoding is high , This usually leads to its overall complexity ratio Opus The codec is much higher . But over time , We think ： Through the improvement of hardware support and new algorithm , There are many ways to improve the coding efficiency of neural network .

Q8：SoundStream Whether it will become a general audio codec , Or just focus on specific areas ？

Jamieson&Jan: Early applications will likely focus on real-time communication , But future SoundStream It is expected to be used in general coding .

Q9: since SoundStream Will be integrated into the next generation 、 An improved version of Lyra in , So this new Lyra Is it possible to replace Opus？

Jamieson&Jan: At least in the short term ,Opus and Lyra Will coexist . in fact , Our team has been continuously studying and improving Opus.

Q10： In the field of audio compression , What's Google's next plan for ？

Jamieson&Jan: We will continue to use ML And traditional coding methods to improve the efficiency of audio compression , And constantly explore in various application fields .