Progress of the 137th MPEG Conference

The original title ：137th meeting of MPEG The original author ：Martin Benjak Link to the original text ：https://www.mpegstandards.org/137th-meeting-of-mpeg/ Translation arrangement ： Xu Zhen

MPEG The first 137 The first meeting was held in 2022-01-17 to 2022-01-21 Online .

MPEG The system team won two more Emmy Awards for technology and engineering

MPEG The systems group is pleased to report ,MPEG This year, the National Academy of Television Arts and Sciences （NATAS） Two Emmy Awards for technology and Engineering , Namely （i）“ Standardization of font technology for customized downloading fonts and typesetting for network and TV devices ” and （ii）“HTTP Standardization of encapsulation protocols ”.

The previous award is related to MPEG Open font format for （ISO/IEC 14496-22） of . Font is a key component of any written communication . Words carry meaning , But it is the font that makes the text readable , in other words , The typeface gives the written word an image . In the past 20 In the year ,MPEG The standardization of Open Font Format technology has greatly affected the capabilities of various consumer electronic devices , For digital TV 、 Streaming media environment and network have brought advanced font technology . It has also inspired many open source projects , High quality font rendering and advanced text support can be adopted on a large scale , Enable high quality font support for OEM、 Simple and cost-effective for service providers and content authors , To deploy new features and applications that support all languages and writing systems in the world .

The latter award is related to MPEG Dynamic adaptation HTTP flow （ namely MPEG DASH,ISO/IEC 23009） of .MPEG DASH Standards are the only commercially deployed HTTP International standard technology for media streaming , It is widely used in many products .2012 year ,MPEG And 3GPP co-developed DASH The first edition of the standard , Since then, four more versions have been developed , Revise the core specification by adding new features and extended functionality . Besides ,MPEG Six other standards have also been developed , As ISO/IEC 23009 The addition of “ part ”, send MPEG DASH Standard and reference software and conformance testing tools 、 Guidelines and enhancements to other deployment scenarios are effectively utilized .MPEG DASH By providing a 3GPP、ATSC、DVB and HbbTV And other standards widely adopted by different alliances and different fields , It has greatly changed the streaming media industry . The success of this standard is due to its excellent technology , Substantial industry involvement in its development , Solved the market demand , And in ISO/IEC JTC 1/SC 29 MPEG System standard development practices and leadership , Work with all sectors of the industry .

“ Thanks to the National Academy of Television Arts and Sciences （NATAS） Yes ISO/IEC JTC 1 Recognition of the outstanding contributions of Chinese media coding experts . Their work has greatly expanded MPEG The benefits of Standardization ”.Philip C. Wennblom,ISO/IEC JTC 1 The chairman of the said .

“ stay 2022 Won two Emmy Awards for technology and engineering , That's right SC 29 in MPEG Great recognition of the work of the system team .MPEG Open font formats and DASH The standard has been widely adopted in the industry , It has become the basis of interoperability between TV and network-based multimedia services .”Gary J. Sullivan,ISO/IEC JTC 1/SC 29 The chairman said .

“MPEG The system has been working tirelessly , Work with key industry players and other standards development organizations , Respond to industry needs . This year, ,MPEG Open font formats and DASH The standard has won two Emmy Awards for technology and engineering , It shows the great success of this effort and the recognition of the industry .”MPEG System group convener Youngkwon Lim said .

This is a MPEG The fifth and sixth Emmy Awards for technology and engineering （ Following 1996 year MPEG-1 and MPEG-2 And JPEG、2008 year AVC、2013 year MPEG-2 TS Flow and 2021 year ISO After the basic media file format ）, It's also MPEG The seventh and eighth overall Emmy Awards （ Include 2008 year AVC High Profile and 2017 year HEVC Golden time project Emmy Award ）.

MPEG The audio code is MPEG-I Immersive audio options 6DoF technology

before ,MPEG On 2021 year 4 In June, a report on 6DoF Proposal solicitation for immersive audio technology （CfP）, For augmented virtual reality （VR） And augmented reality （AR） Experience . In the 137 Time MPEG The meeting , To submit to the CfP The materials were reviewed , And the technology is selected . In all parts of the world 12 Three subjective tests were conducted at three test points VR/AR Hearing test , Yes 14 Submissions were evaluated , As the basis of choice .

stay CfP The technology selected in allows users to own VR or AR Experience , Where users can use 6 A degree of freedom （6DoF） Browse freely and interact with the virtual environment , Thus, unconstrained spatial motion and user rotation can be realized . Rendered audio signals can use audio objects 、 Channels and Higher Order Ambisonics（HOA） In a variety of formats .VR/AR Environmental Science （ scene ） Encoded in a bitstream ,MPEG-I Immersive audio technology renders binaural audio signals to the user's headphones according to the transmitted and decoded scenes , And achieve rich audio effects , Such as directivity 、 location 、 Range 、 Occlusion 、 Diffraction and Doppler shift of sound source , And the complex modeling of acoustic environment . Support a wide range of users and VR/AR Environmental interaction .

It is expected that the standard will be in 2023 year 1 March enters the first formal stage of its approval process , The committee draft （CD）,2023 year 4 International standard draft in October （DIS）, as well as 2023 year 10 International standards for the month （IS）.

MPEG The requirements group issued a proposal solicitation activity on the synchronization of encoder and packer

In the 137 Time MPEG The meeting ,MPEG Demand group （WG 03） A proposal solicitation order was issued （CfP）, Solicit technologies to realize encoder and packer synchronization and related distributed media asset storage .

The encoder and packer synchronization framework will be based on the existing MPEG standard , Such as general media application format （CMAF）、MPEG Dynamic adaptation HTTP flow （DASH）, And potentially other standards and codes , Define the preferred way to generate content from distributed sources . This will enable redundant live and （ video on demand ）VoD Content generation settings can generate synchronized content , It has strong stability against the failure and input loss of one or more components in the setting . Besides , We also solicited solutions for large-scale storage media assets . The asset storage solution will implement live to video and video to live use cases , And use synchronous encoder / Packer frame .

This CfP Ask companies and other organizations for advice . Ask for in 2022 year 4 month 10 Day ahead registration ,2022 year 4 month 17 Recently submitted . stay 2022 year 4 The... Was held in September 138 Time MPEG The meeting , Will respond to CfP For evaluation .

Please contact companies and organizations that have developed encoder synchronization and media storage technologies MPEG System group convener Youngkwon Lim Doctor （[email protected]） or MPEG Requirements group convener Igor Curcio Doctor （[email protected]）, Use this information as a reference to the CfP Response .

MPEG The system group drives MPEG-I The scenario description is in the final stage

In the 137 Time MPEG The meeting ,MPEG System group （WG 03） It's done MPEG-I Scene description criteria , This is to achieve immersive 3D A key technology of user experience , As the final draft international standard （FDIS） Advance to the final approval stage . The specification describes by referencing and locating different media assets in the scenario 3D The composition of the scene . The information provided in the scene description is then used by the application to render the corresponding 3D scene . To meet the needs of immersive applications , This specification is Khronos glTF 2.0 Developed MPEG Expand , This is a scenario description solution widely used in the industry .glTF 2.0 It provides a solid and effective benchmark for exchangeable and interoperable scenario description , It can realize the realistic rendering of immersive content , For example, by using physics based rendering （PBR）. However ,glTF 2.0 Mainly designed for static scenarios and assets , It can not fully meet the dynamic and rich 3D Requirements and needs of the scene .

Based on this analysis ,MPEG Yes Khronos Of glTF 2.0 It has been extended , To integrate real-time media , That is, it supports dynamic visual objects 、 Audio 、 Regular scene update and media access related functions . The standard also defines an architecture , By specifying the relevant API To access the media referenced by the scenario description , Decouple media rendering from media access . This work has been working with Khronos and 3GPP Coordinate . Further details can be found in http://mpeg-sd.org/.

MPEG The system group pushes the media smart contract into the final stage

In the last few years ,MPEG A set of standardized resource description framework is developed （RDF） Noumenon and XML Pattern , For the compilation of intellectual property rights related to music and media （IP） Information .ISO/IEC 21000-19 Media value chain ontology （MVCO） By capturing user roles and their effects on specific IP The permissible behavior of an entity , To be fair 、 Timely and transparent market transactions provide rights tracking .ISO/IEC 21000-21 Media contract body （MCO） It promotes the development of narrative contract and intellectual property management 、 Payment and notification related digital contract conversion . About the latter ,XML The pattern has been developed as ISO/IEC 21000-20 Contract expression language （CEL）.

In the 137 Time MPEG The meeting ,MPEG System group （WG 03） It's done ISO/IEC 21000-23 Development of media smart contract , Advance the standard to the final draft international standard （FDIS） Stage , This is the final approval milestone for the development standard . This standard specifies that the above RDF Noumenon and XML The model can be transformed into the existing distributed ledger technology （DLT） Means of executing smart contracts in the environment （ Such as application programming interface ）. This important standard will greatly help the music and media industry and its stakeholders achieve effective interoperability , So that in different DLT Exchange of validated contract data between environments . such , It will increase trust among stakeholders , In order to share high-value data in the ecosystem （ Such as music copyright ）. Another important feature of the standard is , It provides the possibility of binding the terms of a smart contract to the corresponding terms of a human readable contract through persistent links . such , The signature is in accordance with ISO/IEC 21000-23 Each party of a standard smart contract can accurately know what its terms express .

MPEG The system group has further strengthened ISOBMFF standard

In the 137 Time MPEG The meeting ,MPEG System group （WG 03） In its recent Emmy Award criteria ISO/IEC 14496-12 ISO Basic media file format （ISOBMFF） The first formal milestone has been reached in the approval process of the amendment , This amendment includes improved brand documentation and other improvements .ISO/IEC 14496-12:2021 Amendment 1 Committee amendment draft （CDAM） This includes enhancement by using an external elementary stream to provide an inter prediction reference for the decoding of the elementary stream as the mainstream storage ISOBMFF Random access to media stored in , Support for so-called extension dependent random access points （EDRAPs）. Besides , The amendment provides a way , To list the... That represent a version of the media presentation （ One or more ） A collection of media components , These components can be selected by the user for synchronous decoding and presentation . The amendment also clarifies the use of major brands and compatible brands in the document type box , To define ISOBMFF The new brand identity provides a guide . The amendment is expected to come into force in 2023 As the final draft amendment at the beginning of the year （FDAM） Reach its final approval milestone .

MPEG The video coding group has completed LCEVC Consistency and reference software

In the 137 Time MPEG The meeting ,MPEG Video coding group （WG 04） Completed low complexity enhanced video coding （LCEVC） Consistency and reference software standards （ISO/IEC 23094-3） Formulation , And promote it to the final draft of international standards （FDIS） Stage , For final approval and publication .

This standardization will help LCEVC In the process of developing the product, the implementer checks the correct function of its implementation . These tests will also help LCEVC Users and potential users of the product , Provide a way to validate these product declarations of conformance . therefore , This testing will help the community achieve interoperability between encoder and decoder products , And will encourage LCEVC Adoption and use of .

LCEVC Added an enhanced data flow , By building on existing and future video coding formats , It can significantly improve the resolution of the reconstructed video 、 Bit depth and visual quality , And it has effective compression efficiency with limited coding and decoding complexity . It is designed to work with existing video workflows （ Such as CDN、 Metadata management and DRM/CA） And streaming media / Media format （ Such as DASH and CMAF） compatible , To facilitate the rapid deployment of enhanced video services .LCEVC It can be used to provide higher video quality with limited bandwidth , Especially when the available bit rate is relatively low for high-resolution video coding and transmission , Or when the complexity of decoding is a challenge .

MPEG The video encoding group sends MPEG Committee draft for consistency and reference software for immersive video

In the 137 Time MPEG The meeting ,MPEG Video coding group （WG 04） To promote the MPEG Immersive video （MIV） Consistency and reference software standards （ISO/IEC 23090-23） Enter the committee draft （CD） Stage , This is the first formal milestone in its approval process . This document specifies how to conduct conformance testing , And for ISO/IEC 23090-12 MPEG Immersive video provides reference encoder and decoder software . The draft includes 18 A verified and confirmed consistent bit stream based on MPEG Immersive video test model （TMIV）12.0 Version of encoding and decoding reference software . The test model 、 Objective indicators and other tools can be found in https://gitlab.com/mpeg-i-visual.

MIV Is developed to support the compression of immersive video content , One of the real or virtual 3D scenes is captured by multiple real or virtual cameras . The standard can store and distribute immersive video content on existing and future networks , With 6 A degree of freedom （6DoF） The position and direction of the viewing angle .MIV Is a flexible depth multi view video （MVD） standard , Use powerful hardware support for common video formats to encode volume video . Views can use isometric 、 Perspective or orthographic projection . By trimming and wrapping the view ,MIV Efficient video coding can be used （HEVC） Realize about 15 to 30Mb/s Bit rate of , And the equivalent of HEVC 5.2 Level pixel rate . Except for MVD Of MIV Outside the main configuration file , also MIV Geometry missing Profile , It is suitable for depth estimation based on cloud and decoder , as well as MIV Extended profile , It can be applied to multi plane images （MPI） Encoding .MIV The standard is designed for video coding based on visual volume and point cloud coding based on video （ISO/IEC 23090-5） A standard set of extensions and profile restrictions , Its consistent bit stream covers all specified configuration files .

In addition to conformance testing ,MIV The verification test of is also in progress ,MIV The transmission of is through V3C Data transmission standard （ISO/IEC 23090-10） Stipulated .MPEG Demand group （WG 02） Will release MIV Final use cases and requirements for the second edition , This will be MIV Evolution of standards ,MPEG Liaison and communication group （AG 3） About will be published MIV White paper for （ As discussed further below ）.

JVET Made VVC and VSEI Second edition of , And finally determined VVC Reference software

In the 137 Time MPEG The meeting ,MPEG Joint video coding team and ITU-T SG 16（WG 05;JVET） Completed universal video coding （VVC,ISO/IEC 23090-3 | ITU-T H.266） And multifunctional supplementary enhancement information for encoding video bitstreams （VSEI,ISO/IEC 23002-7 | ITU-T H.274） Development of the second edition , Upgrade them to the final draft international standard （FDIS） state , For final approval and publication . new VVC The version defines support for larger bit depths （ the height is 16 The bit ） Configuration file and level of , Including some low-level coding tool modifications , To obtain improved compression efficiency of high bit depth video at high bit rate .VSEI The first 2 Version added SEI Information , For scalability 、 Multiview 、 Show adaptability 、 Improved flow access and other use cases provide additional support . Besides , Also released VVC Committee draft of the next amendment （CDAM）, To begin the formal approval process , send VVC And green metadata （ISO/IEC 23001-11） And video decoding interface （ISO/IEC 23090-13） Standards , And for particularly high-capacity applications （ Such as professional 、 Some uses in scientific and medical application scenarios ） Add a new unconstrained level . Last ,VVC（ISO/IEC 23090-16） The reference package for has also been completed , To obtain the FDIS state . Refer to the software pair VVC Device developers are very helpful , Help them test whether their implementation conforms to the video coding specification .

JVET Push AVC The tenth edition became the final draft of the international standard

In the 137 Time MPEG The meeting ,MPEG Joint video coding team and ITU-T SG 16 (WG 05;JVET) The 10th edition of advanced video coding has been finalized （ISO/IEC 14496-10 | ITU-T H.264）, As the final draft international standard （FDIS） Release , For final approval and publication . In addition to various text improvements , The standard also stipulates a new SEI Information , Used to describe the shutter interval applied during video capture . This may be variable in the camera , Conveying this information is of great value to the post-processing of analyzing and decoding video .

JVET take HEVC Extended to 16K And above

In the 137 Time MPEG The meeting ,MPEG Joint video coding team and ITU-T SG 16 (WG 05;JVET) Started efficient video coding （HEVC,ISO/IEC 23008-2 | ITU-T H.265） The ratification process of the new second amendment , The Committee issued a draft amendment （CDAM）, New levels and hierarchies are defined , Support extremely high bit rates and up to 16K Video resolution , An unconstrained level is defined . This will enable HEVC In new application fields , Including major 、 Scientific and medical video .

MPEG The gene coding Group assesses responses to new advanced genomic features and technologies

High throughput DNA The widespread use of sequencing technology has achieved what is known as “ pre ” New medical methods , And many other emerging applications , Such as monitoring the evolution of outbreaks in agriculture and food industry and pathogen monitoring .DNA Sequencing technology produces an extremely large amount of heterogeneous data , Include raw sequence readings 、 The results of the analysis 、 Annotations and related metadata , This data is stored in different repositories around the world , The use of data needs to be realized through standardized and interoperable formats . These genomic data need to be structured and compressed with high performance , To reduce the amount of storage , Increase transmission speed , And improve the browsing and searching performance of these large datasets , This is the requirement of various applications and use cases . current MPEG-G Standard series （ISO/IEC 23092） It solves the representation of genome sequencing data 、 Indexes 、 Compression and transmission problems , Support annotation data and search capability .ISO/IEC 23092（MPEG-G） The standard series provides a file and transfer format 、 compression technique 、 Metadata Specification 、 Protection support and standards API, For accessing genomic data and annotation data in a locally compressed format .

In the 134 Time MPEG The meeting ,MPEG Gene coding group （WG 06） A proposal solicitation notice was issued （CfP）, To collect improvements ISO/IEC 23092 Current compression of the standard series 、 New technology delivery for transport and indexing capabilities .

Now? , In the 137 Time MPEG The meeting ,MPEG Gene coding group （WG 08） Evaluated the submitted pair of CfP The response of the , It solves the representation and use of graphic genome reference , And support interfaces with existing clinical data exchange standards . The preliminary assessment results show that , Incorporate local representations of graphical genomic references MPEG-G Standards are desirable , It will provide new information for expanding the use of genome sequencing data 、 Advanced and effective presentation capabilities . About supporting existing clinical data exchange standards （HL7 and FHIR） The interface of , The reply received indicates that , This extension is useful for MPEG-G Effective integration of data into clinical workflow is desirable , It also shows the standardized path to achieve this integration .

MPEG white paper

In the 137 Time MPEG The meeting ,MPEG Liaison and communication group （AG 03） The following three were approved MPEG white paper .

Neural network coding （NNC）

Artificial neural networks have been used for a wide range of tasks in almost all technical fields , Such as medical application 、 transport 、 network optimization 、 Big data analysis 、 monitor 、 voice 、 Audio 、 Image and video classification 、 Image and video compression, etc . Their recent success is based on the fact that processing is bigger than it used to be 、 More complex neural networks （ Deep neural network ,DNN） The feasibility of , And the availability of large-scale training data sets . Another factor in the exponential growth in the use of such neural networks is the emergence of new use cases , Such as joint learning of continuous communication between many devices . therefore , This requires the highest neural network compression capability , To minimize the overall traffic , And reduce the size of the network when it is used for reasoning . therefore , Neural network coding （NNC） The standard of is already in ISO/IEC 15938-17 “ Neural network compression for Multimedia description and analysis ” In the definition of .

Low complexity enhanced video coding （LCEVC）

This white paper is for ISO/IEC 23094-2 Low complexity enhanced video coding （LCEVC） It is summarized . The encoding format is designed to be used in combination with existing video encoding formats , Using encoder driven upsampling and specific tools “ residual ” Encoding , That is, the difference between the original video and the predicted interpretation .LCEVC Can improve compression efficiency , And by using a small number of specialized enhancement tools , Reduce the overall computational complexity of coding at a given resolution and bit depth . This white paper outlines LCEVC The architecture of 、 Overview of coding tools and compression efficiency .

MPEG Immersive video

from MPEG To make the MPEG Immersive video （MIV） The standard has been completed , Its final draft international standard （FDIS） On 2021 year 10 Published in , For final approval and publication .MIV The goal of the standard is for the immersive six degrees of freedom （6DoF） Volume vision scene provides efficient coding . And three degrees of freedom （3DoF） Different forms of expression , Immersive 6DoF Presentation provides a larger viewing space , The viewer can move freely in translation and rotation .6DoF Video can also realize the perception of motion parallax , That is, the relative position of the scene geometry changes with the posture of the viewer . without MIV Of 6DoF function ,3DoF The absence of motion parallax in video does not conform to the working principle of human visual system , Often cause visual discomfort . This white paper is important for new MIV The standard is briefly introduced .