Centripetalnet: more reasonable corner matching, improved cornernet | CVPR 2020 in many aspects

CentripetalNet The core of this method is the new corner matching method , Learn an extra centripetal offset value , A corner whose offset is small enough is a match , be relative to embedding How vectors match , This method is more robust , Better explanation . In addition, the cross star deformation convolution proposed in this paper also fits the scene of corner target detection very well , Enhance corner features undefined 

source ： Xiaofei's algorithm Engineering Notes official account

The paper : CentripetalNet: Pursuing High-quality Keypoint Pairs for Object Detection

• Address of thesis ：https://arxiv.org/abs/2003.09119
• Paper code ：https://github.com/KiveeDong/CentripetalNet

Introduction

---

  CornerNet Opens up a new way of target detection , Target location by detecting corners , On the corner matching , Added extra embedding vector , The corner with smaller vector distance is called matching . And the paper argues that , This method is not only difficult to train , And only by the surface of the object , Lack of target location information . For similar objects ,embedding It's hard for vectors to express themselves in a specific way , Pictured 1 Shown , Similar objects can cause frame errors .

  So , The paper proposes that CentripetalNet, The core is to propose a new corner matching method , Learn an extra centripetal offset value , A corner whose offset is small enough is a match . be relative to embedding vector , This method is more robust , Better explanation . in addition , The paper also proposes the cross star deformation convolution , For corner prediction scenarios , In feature extraction, it can accurately sample the features of key positions . Finally, the instance segmentation branch is added , Can expand the network to the instance segmentation task .

CentripetalNet

---

  Pictured 2 Shown ,CentripetalNet There are four modules , Respectively ：

• Corner prediction module (Corner Prediction Module)： Used to generate candidate corners , This part is related to CornerNet equally .
• Centripetal offset module (Centripetal Shift Module)： Predict the centripetal offset of the corner , According to the migration results, the similar corners are grouped .
• Cross Star deformation convolution (Cross-star Deformable Convolution)： Convolution for corner scenes , The feature of corner position can be enhanced efficiently .
• Instance split Branch (Instance Mask Head)： similar MaskRCNN Add instance segmentation Branch , It can improve the performance of object detection and increase the ability of instance segmentation .

Centripetal Shift Module

---

Centripetal Shift

  about $bbox^i=(tlx^i,tly^i,brx^i,bry^i)$, The geometric center is $(ctx^i, cty^i)=(\frac{tlx^i+brx^i}{2}, \frac{tly^i+bry^i}{2})$, Define the centripetal offset of the upper left corner and the lower right corner as ：

  $log$ The function is used to reduce the numerical range of the centripetal offset , Make training easier . During the training , Due to the GT Corners need to be combined with corner offset to calculate centripetal offset , More complicated , Pictured a Shown , So only for GT Corner use smooth L1 The training of centripetal deviation was carried out ：

Corner Matching

  Corners that belong to the same group should have centers close enough , So after getting the centripetal offset and the corner offset , According to the center point corresponding to the corner, we can judge whether the two corners are corresponding . First, we will satisfy the geometric relationship $tlx < brx \wedge tly < bry$ The corners of the image are combined into a prediction box , The confidence of each prediction box is the mean of the corner confidence . next , Pictured c Shown , Define the central area of each prediction box ：

  $R_{central}$ The corner of is calculated as ：

  $0 < \mu \le 1$ It is the proportion of the side length of the prediction box corresponding to the central area , Calculate the center of the upper left corner according to the centripetal offset $(tl{ctx}, tl{cty})$ And the center of the lower right corner $(br{ctx}, br{cty})$, The calculation satisfies the central region relationship $(tl^j{ctx}, tl^j{cty})\in R^j{central} \wedge (br^j{ctx}, br^j{cty})\in R^j{central}$ The weight of the prediction box ：

  From formula 5 It can be seen that , The closer the center point corresponding to the corner is , The higher the weight of the prediction box , For the prediction box that does not satisfy the geometric relationship of the center point , The weight is directly set to 0, Last , Use weights to weight the confidence .

Cross-star Deformable Convolution

---

  In order to let the corner perceive the location information of the target ,coner pooling Use max and sum To carry out the horizontal and vertical transmission of target information , There is a cross star phenomenon in the output feature map , Pictured 4a Shown , The boundary of the cross contains rich contextual information . In order to further extract the features of the cross star boundary , It doesn't just need a bigger sense , It also needs to adapt to its special geometry , So the paper puts forward the cross star deformation convolution .

  But not all boundary features are useful , For the upper left corner , Because the upper left boundary feature of the cross is outside the target , So it's relatively useless for the upper left corner , So the paper uses bias guidance (guiding shift) To display the pilot offset value (offset field) Learning from , The offset guidance is shown in the figure b Shown . The offset value is obtained through three convolution layers , The first two convolutions transform corner pooling Output , Supervised learning through the following loss function ：

  $\delta$ Guide for offset , Defined as ：

  The third layer of convolution maps the feature to the final offset value , It contains the context information and geometry information of the target .

  Different sampling methods are visualized in this paper , It can be seen that the effect of the cross star deformation convolution proposed in this paper is in line with the expectation , The sampling points corresponding to the upper left corner are the lower right boundary of the cross .

Instance Mask Head

---

  In order to get the result of instance segmentation , Paper selection soft-NMS The previous test results are used as candidate boxes , Using a fully convolutional network mask forecast . In order to ensure that the detection module can provide effective candidate box , First pair CentripetalNet A few pre training rounds , And then take top-k Candidate box for RoIAlign Get the feature , Feature extraction using four consecutive convolution layers , Finally, the deconvolution layer is used for up sampling , Cross entropy loss is performed on each candidate box during training ：

Experiment

---

  The complete loss function is ：

  $L{det}$ and $L{off}$ Follow CornerNet The definition is the same , To predict box loss and corner offset loss ,$\alpha$ Set to 0.005.

  Target detection performance comparison .

  Instance segmentation performance comparison .

  CornerNet/CenterNet/CentripetalNet Visual comparison .

Conclusion

---

  CentripetalNet The core of this method is the new corner matching method , Learn an extra centripetal offset value , A corner whose offset is small enough is a match , be relative to embedding How vectors match , This method is more robust , Better explanation . In addition, the cross star deformation convolution proposed in this paper also fits the scene of corner target detection very well , Enhance corner features .

If this article helps you , Please give me a compliment or watch it ～undefined More on this WeChat official account 【 Xiaofei's algorithm Engineering Notes 】