Tencent Youtu won the champion of iccv2021 LVIs challenge workshop and the best innovation award of the project

In recent days, , stay ICCV2021 Host LVIS Challenge Workshop In the game , Tencent Youtu laboratory won the championship , At the same time, he was awarded the best innovation award of the project .LVIS Challenge 2021 It is an instance segmentation task for large-scale long tail data , As this ICCV One of the heavyweight competitions of , It has attracted many well-known enterprises and universities at home and abroad to participate in . The core technical scheme of this competition will also be applied to industry AI In the quality inspection scenario , Further improve the accuracy of defect detection and segmentation , Support the industrial landing with the most core technologies .

chart 1. The final list of the competition , Tencent Youtu ranks first

LVIS Is included 1k+ Large scale long tailed data sets of classes , Compared with the common instance segmentation data set ,LVIS With finer dimensions and more categories , So its distribution is closer to the natural scene . According to statistics , The number of instances of the tail category only accounts for about... Of the total number of instances 0.41%, This poses a great challenge to the existing instance segmentation algorithms . in addition , Different from the previous games , This time LVIS The competition adopted Boundary AP replace Mask AP As an evaluation indicator , It puts forward higher requirements for segmentation accuracy .

chart 2. LVIS Introduction to the competition

In response to the above challenges , Tencent Youtu team proposed balanced distribution , Optimize the instance segmentation method of edges , The results are obtained on the test set 48.1%AP Result . It is worth mentioning that , This time Workshop In the meeting ,Ross Girshick Point out the APr And APf The results are very similar ！

chart 3. Workshop The result of the conference contest was announced Apr And APf near

  The technical details are as follows

Tencent Youtu team will Hybrid Task Cascade（HTC） The instance segmentation algorithm is used as baseline, It adopts the method of Swin-Transformer As the basic backbone network , meanwhile , be based on CBNetV2, Compound links two identical Swin-Transformer The Internet , As the ultimate backbone network to enhance performance .

chart 4. Strong baseline

For the long tail problem , Tencent Youtu proposed the distribution balance module , Including data balance and loss balance processing , So as to enhance the attention to the tail rare category instances in the process of network training . among , Data balancing methods include RFS, Balanced Copy-Paste and Balanced Mosaic, Increase the probability of tail category data , Give consideration to image-level and instance-level Data balance . meanwhile , Youtu adopts Seesaw Loss, In training, the excessive negative sample gradients on the tail categories are dynamically suppressed , And add the punishment for misclassification samples .

In order to better optimize the segmentation effect , Tencent Youtu proposed a fine segmentation module , contain Mask Scoring and RefineMask Method . be based on Mask Scoring Method , The classification confidence and instance segmentation score are decoupled , Using the new network branch learning example to predict the quality , Thus, the problem of mismatch between classification confidence and segmentation quality is avoided . Optimize the accuracy of edge segmentation , Tencent Youtu adopts RefineMask Method , Fusion of multi-stage fine-grained up sampling semantic features , So as to produce high-quality segmentation results . Consider the balance between time and accuracy , Eutu lab will only pipeline Last of Mask head Replace with Refinemask head. thus it can be seen , Tencent Youtu's method still has room for improvement .

besides , Observation of training process based on Tencent Youtu , It creatively adopts the training strategy of head and tail performance balance , It not only improves the overall AP result , The performance gap between the tail and the head categories is even greater . Final , The Youtu team takes 48.1%AP No. 1 .

chart 5. Distribution balance module

chart 6. Fine segmentation module

As Tencent's top Artificial Intelligence Laboratory , UTRA lab focuses on computer vision , Focus on face recognition 、 Image recognition 、OCR And other fields , In the process of promoting industrial digital upgrading , Always insist on basic research 、 The development strategy of industry landing and walking on two legs , Deep integration with Tencent cloud and smart industry , Tap the pain points of customers , Reduce cost and increase efficiency for the industry . future , Tencent Youtu laboratory will also continue to cultivate deeply CV technology , And will continue to explore more application scenarios and application spaces , Let more users enjoy the dividend brought by technology .