Open source optimized VVC encoder in general scenarios

source ：SPIE 2021 author ：Adam Wieckowski, Christian Stoffers, Benjamin Bross, Detlev Marpe Content arrangement ： Zhao Yan stay HEVC Seven years after the publication of the standard , Universal video coding standard H.266/VVC On 2020 year 7 Finalized monthly . Compared with the same video quality HEVC, VVC An appointment can be provided 50% The code rate is saved . To provide ratio VVC Reference software VTM Faster running speed , Optimize software VVenC Open source soon .VVenC Multithreading is also supported 、 rate control 、 Additional functions such as subjective quality optimization , And for high resolution Random Access The mode is optimized . This paper deals with VVenC Test the performance under various configurations and video content . Experiments show that ,VVenC It can be approximated with lower computational effort VTM Performance of , And provides additional R-D ( Bit rate - The distortion ) Work point .

Catalog

• Introduce
• VVenC
  • Supported by VVC characteristic
  • Increased optimization and configuration space
  • Derivation of preset parameter settings
  • Low delay (Low Delay, LD) and Full frame (All Intra, AI) Mode support
  • Screen content encoding tool (Screen Content Coding, SCC) Support
  • High dynamic range video (High Dynamic Range, HDR) Support
• Experimental content
• experimental result
  • Coding performance at different video resolutions
  • Coding performance in different time domain modes
  • Coding performance under different video content
• summary
• reference

Introduce

After testing ,VVC Coding standard ratio HEVC Yes 50% The code rate is saved , And widely supports various video content and use cases . And HEVC The difference is , The first edition VVC The standard already has screen content encoding 、 High level semantic division and other functions , And support various special video formats ：10 bit High dynamic range of bit depth (High Dynamic Range, HDR)、 Optional chroma sampling format ( Include 4:4:4)、 Resolution scaling and reference frame resampling . at present ,VVC The second edition of the standard is being prepared , This version of the standard will support 10 bit Video coding with upper depth and higher resolution .

VTM yes VVC The reference test model , Algorithm evaluation test used in standard setting .VTM Not optimized for practical applications , Instead, the tedious algorithm module is improved to obtain the maximum coding gain as much as possible . therefore ,VTM The optimization degree and vectorization degree of each component in the encoder are uneven , Lack of multithreading support and efficient rate estimation , And only basic subjective quality optimization is supported . stay Random Access General test configuration ,VTM The running time of is about HM (HEVC Standard reference software ) Of 8 times , This is a VVC The increase of self computing complexity 、 The increase of the number of modules leads to the increase of the solution space and the more flexible partition mechanism .

stay VVC Soon after the standard was finalized , A new open source encoder VVenc Open source release , It aims to provide... In real application scenarios VVC coded ： With shorter running time VTM Approximate performance , And provide additional work points （fast, faster, medium, slow, slower）, It can achieve higher running speed . Besides ,VVenC Support the coding function in real application scenarios , Such as multi-threaded acceleration 、 Single stage ／ Two stage code control and subjective quality optimization . This paper deals with VVenC Test the coding performance under different configurations , And evaluate the high-resolution video of different video content and non general test content .

VVenC

Following 2020 Published in 0.1 After version ,VVenC 1.0 Version on 2021 Issued in May of . Unless otherwise specified below , All test contents are from VVenC 1.0 edition . For the project C++ To write , Two independent encoder running programs are provided .

Supported by VVC characteristic

VVenC The encoder supports Main10 The configuration file is used for processing slice (/tile) code , as well as 4:0:0 and 4:4:4 Chroma down sampling format , But it still does not support Main10-4:4:4 The configuration file .VVC Main10 Except adaptive transformation (ACT) and Two way weighted prediction (BCW) Other coding tools are VVenC 1.0 Chinese support , And both of these will follow 1.1、1.2 The version introduces . Not YUV The format content will be converted to YUV Format , therefore ACT No VVenC Current R & D priorities . although VVenC The encoder cannot change the resolution by resampling the reference frame , However, the bit stream generated can be divided and recombined （ Only minor changes are required ）, Video bitstream with adaptive resolution change .

Increased optimization and configuration space

VVenC It's from VTM Improved from the basic version of , Transplanted a lot VTM Internal structure and application interface . On this basis ,VVenC take VTM The performance of the core coding tool is improved and the bottom layer is optimized . therefore ,VVenC Many of the tools in are related to VTM Very similar in , But better performance .

As mentioned earlier ,VTM The search space for coding patterns in is very large , Less early stop mechanism is conservatively adopted .VVenC To optimize the pattern search algorithm , Simple use cases are only allowed in several preset configurations (fast, faster, medium, slow, slower) Make a selection , Only in professional applications will the search variables of each coding tool be accurately selected . The previous practice was ： If the switch ID of the corresponding coding tool is set to '0', Indicates that it is disabled ; If the identification bit is a value greater than zero, it means that . For this multi rate setting , The larger the value of the identification bit , It indicates that the faster coding speed configuration is selected . Besides ,VVenC Some independent acceleration tools have also been introduced , Control them separately .

And VTM comparison ,VVenC Pair partition 、 Motion estimation and pattern search of some intra coding are optimized , Select speed in optimized mode . Besides , Integer based optimization RDOQ Also in the faster and fast Configuration is introduced .

Derivation of preset parameter settings

VVenC The adjustable configuration parameters and their ranges are shown in the table 1 Shown .

surface １. VVenC Parameter selection in

The derivation process of the value range of these parameters is from the Pareto set of the configuration space (Pareto Set) The approximate beginning of , By disabling most coding tools , Open almost all acceleration tools , A search starting point can be approximately determined . Then in each iteration , Test a subsequent set of candidate work points , The configuration vector with the best performance will be selected as the new approximation point of Pareto set . This process will be repeated many times , A set of points is determined as the optimal Pareto approximation . This process does not yield the real best , Because the interaction between configuration options is not considered , Nor does it traverse the entire search space .

Approximate Pareto optimality set , There are five operating points selected as preset configuration points ：fast, faster, medium, slow, slower, The slowest configuration (slower) Provide with VTM Same compression performance . And the fastest configuration (faster) And HM-16 comparison , Only a small percentage of the running time can achieve at least 10% Of BD-rate gain . The other three operating points have achieved a good trade-off in terms of speed and compression performance . The Pareto set approximation is JVET CTC A series of test sequences (class A1, A2, B) According to experience , This article will VVenC Performance on other sequences and configurations .

Low delay (Low Delay, LD) and Full frame (All Intra, AI) Mode support

The optimal parameters in the previous section are derived in random access (Random Access, RA) The best effect can be achieved in mode . according to JVET-CTC Regulations ,RA The random access interval of the mode is about one second . except RA Pattern ,JVET CTC It also stipulates that All Intra (AI) Mode and two Low Delay (LD) Pattern ( Only... Are allowed separately P The frame or B Inter prediction of frames ). In order to inherit the conditions derived above , The internal configuration of the encoder needs to be compatible with it .

The tests in this article use the simplest approach ：AI In mode , Only use IDR (Instantaneous Decoding Refresh) The frame or CRA (Clean Random Access) frame , All interframe coding tools are off ; about LD Pattern , This article only tests Low Delay-B To configure , And continue to use VTM Low Delay-B in GOP=8 Set up , And all two-way equidistant prediction models (e.g. DMVR, BDOF) Wait until the module is disabled .

Screen content encoding tool (Screen Content Coding, SCC) Support

VVenC Support VVC Main10 All screen content codes in the configuration (SCC) Tools . stay JVET CTC Specification , All of these SCC Whether the tool is enabled or not is determined by the test sequence . and VVenC As a practical encoder , The end user does not need to know the setting of optimal coding parameters .VVenC Use the built-in screen content detection tool , And according to the corresponding content classification results, the on or off of each coding tool is determined . therefore , there SCC The tool breaks the semantic description of previous search algorithms ： about TS (Transform Skip, Transform skip ), Chroma TS, IBC (Intra Block Copy, Intra block copy ) And BDPCM (Block Differential Pulse Code Modulation, Block differential pulse code modulation ) And other tools , The value of the flag bit is '0' Means that the tool is disabled ,'1' Means that the tool is always opened ,'2' The representative shall decide whether to enable... According to the result of frame content classification . therefore IBC A separate parameter is required to select the speed . Besides ,MCTF (Motion Compensated Temporal Filter, Motion compensated time domain filtering ) It will also be disabled under the screen content . See table for specific settings 1.

High dynamic range video (High Dynamic Range, HDR) Support

VVenC The encoder also supports HDR Video coding , The parameters involved include color format 、QP Offset, etc , Here they are unified into one 'Hdr' Parameters , Its value is one of the following values ：{'off', 'pq', 'pq_2020', 'hlg', 'hlg_2020'}, The corresponding input signals are respectively SDR, PQ, HLG Equiform , And support BT.709 and BT.2020 The distinction of color space . This article tests , Separately '--Hdr=pq' and '--Hdr=hlg' Set for JVET CTC H1 class and H2 class In the video . And VTM The difference is , Set up Hdr Parameter will cause the encoder to generate HDR SEI Metadata .

Experimental content

The encoder tested in this paper is VVenC 1.0.0, Single threaded HM-16.23 and VTM-12.0 Encoder is for comparison , among HM Is the performance benchmark (anchor). The experimental equipment is 32 Nuclear Intel Xeon [email protected] CPU, Disable hyper threading technology .

Each encoder is mounted on four fixed QP value (22, 27, 32, 37) Code under , With PSNR As a measure of distortion ,BD-rate The performance comparison is shown in the figure 1,2,3 and surface 2 Shown , The running time of the horizontal axis is in logarithmic coordinates .

surface 2. chart 1-3 Data collation

experimental result

Coding performance at different video resolutions

Here the RA Patterns of JVET CTC Pass test sequence for testing ,VTM and 　VVenC The encoder adopts GOP=32,HM The encoder adopts GOP=16 Set up , but I Frame period and VTM alignment . except VVenC Of faster Configuration , All encoders use MCTP (Motion Compensated Temporal Filter, Motion compensated time domain filtering ) Tools .

The experimental results are shown in the figure 1 And table 2 Shown , It can be seen that slower Set up VVenC The encoder is approximately the same as VTM Performance is quite , But the running time is only about half of the latter . These five working point settings approximately uniformly complete the transition from slow to fast coding speed .

Comparisons across sequence categories can reveal ,VTM and VVenＣ Have the same trend ： As the video resolution increases , The compression efficiency is also gradually improved , This is because VVC The standard is specially optimized for high-resolution video . Besides , although VVenC The coding performance is approximately balanced in each category , but class A1 and A2 The video sequence in medium Set lower curve crossing , namely faster and slower The configurations can be A1 and A2 The sequence of provides better performance tradeoffs . This explanation A1 The sequence property of the class is more responsive to a simpler Toolset , Affine motion estimation can be A2 The sequence in brings better performance .

chart 1. RA Each sequence in mode class Through test performance comparison of (HD4K yes class A1, A2, B Set )

Coding performance in different time domain modes

chart 2 And table 2 Shown in LD and AI The performance comparison of three encoders in the test mode . except class E stay LD-B Performance in mode , Other items are compared with RA Results in mode ( chart １) Very similar .class E The sequence in is conference video , There is little motion information in the video content , stay LD-B The configuration shows the best performance , This may be VVenC Default in SCC Caused by the encoding mechanism ： although class E The video in is not screen content , But these relatively static videos are recognized as screen content by the encoder , And use SCC Code the tool , Some performance gains have been achieved .

It can be seen that ,VVenC stay LD The encoding speed under the configuration can reach VTM Twice as many , The coding efficiency is equal to or better than the latter . stay AI In mode ,VVenC slower Configuration compared to VTM There are about 1.5 Times the acceleration ratio , This may be because VVenC For some intra coding tools (e.g. ISP, LFNST) Selective opening is carried out . Besides ,VVenC stay class A2 Performance on... Compared to VTM There are about 2% The loss of .

chart 2. LD and AI Each sequence in mode class Through test performance comparison of

Coding performance under different video content

VVenC The five coding configurations of are for JVET high resolution SDR Sequence (class A1, A2, B) optimized . chart 3 The performance of the encoder under other test sequences is shown in , The sequence source is HHI-Berlin Test set . It can be seen that ,VTM and VVenC Compare with HM The performance improvement of the has decreased . except faster Configuration , Other models are best at 8k Video processing , The second is 4k and 2k.

chart 3 in a) and b) The figures show the pairs of HDR Video and screen content encoding performance . And VTM The difference is , To ensure that end users can rebuild correctly HDR video ,VVenC Appropriate... Needs to be generated and transmitted SEI Information . Besides ,VVenC There are two ways to process screen content ：VVenC auto SCC and VVenC SCC, The former uses an automatic screen content detection mechanism , The latter explicitly opens SCC Coding tools . There may be misjudgment in screen content detection , therefore auto SCC Slight performance loss . As a benchmark HM The encoder does not have a screen content encoding tool , So in this part of the test ,VTM and VVenC The performance advantage of is particularly obvious .

chart 3. RA Comparison of coding performance of different video content in mode

summary

This article focuses on open source VVC Encoder VVenC Performance tests were carried out . Experiments show that , In addition to screen content coding ,VVenC Can provide an approximation at a faster operating speed VTM The compression efficiency of ： stay slower Under configuration ,VVenC It can provide at least twice the acceleration ratio , The acceleration effect of other configurations is stronger . By testing various video types , Also shows VVenC It is a mature universal VVC Encoder , Can provide good coding performance across content and use cases . Besides , Experimental proof ：1) Automatic screen content detection is very effective ;2) A simplified HDR Parameters are not only easy to use , It also maintains compression performance .

reference

[1] ITU-T and ISO/IEC JTC 1, “Versatile video coding,” Rec. ITU-T H.266 and ISO/IEC 23090-3 (VVC), August, (2020). Google Scholar [2] ITU-T and ISO/IEC JTC 1, “High Efficiency Video Coding,” Rec. ITU-T H.265 and ISO/IEC 23008-2 (HEVC), April, (2013). Google Scholar [3] V. Baroncini and M. Wien, “VVC Verification Test Report for UHD SDR Video Content,” doc. JVET-T2020 of ITU-T/ISO/IEC Joint Video Experts Team (JVET), 20th meeting, October, (2020). Google Scholar [4] V. Baroncini and M. Wien, “VVC verification test report for HD SDR and 360° video content,” doc. JVET-V2020 T2020 of ITU-T/ISO/IEC Joint Video Experts Team (JVET), 22th meeting, April, (2021). Google Scholar [5] B. Bross, “Overview of the Versatile Video Coding (VVC) Standard and its Applications,” in IEEE Transactions on Circuits and Systems for Video Technology, (2021). Google Scholar [6] F. Bossen, X. Li, K. Sühring, K. Sharman, and V. Seregin, “JVET AHG report: Test model software development (AHG3),” doc. JVET-W0003 of ITU-T/ISO/IEC Joint Video Experts Team (JVET), 23rd meeting, July, (2021). Google Scholar [7] F. Bossen, J. Boyce, X. Li, V. Seregin, and K. Sühring, “VTM common test conditions and software reference configurations for SDR video,” doc. JVET-T2010 of ITU-T/ISO/IEC Joint Video Experts Team (JVET), 20th meeting, October, (2020). Google Scholar [8] A. Wieckowski, “VVenC: An Open and Optimized VVC Encoder Implementation,” in 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), (2021). Google Scholar [9] J. Brandenburg, “Towards Fast and Efficient VVC Encoding,” in 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP), (2020). Google Scholar [10] J. Brandenburg, “Pareto-optimized coding configurations for VVenC, a fast and efficient VVC encoder,” in 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), Google Scholar [11] R. Skupin, “Open GOP Resolution Switching in HTTP Adaptive Streaming with VVC,” in 2021 Picture Coding Symposium (PCS), (2021). Google Scholar [12] P. Czyzżak and A. Jaszkiewicz, “Pareto simulated annealing—a metaheuristic technique for multiple-objective combinatorial optimization,” Journal of Multi-Criteria Decision Analysis, 7 34 –47 (1998). https://doi.org/10.1002/(SICI)1099-1360(199801)7:1<34::AID-MCDA161>3.0.CO;2-6 Google Scholar [email protected] [13] G. Bjøntegaard, “Calculation of average PSNR differences between RD-curves,” Technical Report VCEG-M33, ITU-T SG16/Q6, Austin, Texas, USA, (2001). Google Scholar [14] ITU-T HSTP-VID-WPOM and ISO/IEC TR 23002-8, “Working practices using objective metrics for evaluation of video coding efficiency experiments,” (2021). Google Scholar [15] B.Bross, H. Kirchhoffer, C.Bartnik, M. Palkow, and D. Marpe, “AHG4 Multiformat Berlin Test Sequences,” doc. JVET-Q0791, January, (2020). Google Scholar [16] A. Segall, E. François, W. Husak, S. Iwamura, D. Rusanovskyy, “JVET common test conditions and evaluation procedures for HDR/WCG video,” JVET-V2021 of ITU-T/ISO/IEC Joint Video Experts Team (JVET), 22nd meeting, Apr., (2021). Google Scholar

Finally, the video of the speech is attached ：

http://mpvideo.qpic.cn/0bc3cmaa4aaa34ajss7gfzqvae6dbyjqadqa.f10002.mp4?dis_k=57fb536211e2779a8c10ed9f253e9f3a&dis_t=1645150731&vid=wxv_2229856777368633350&format_id=10002&support_redirect=0&mmversion=false