1. Pedestrian recognition based on incremental occlusion generation and confrontation suppression

[5]. Cairong Zhao, Xinbi Lv, Shuguang Dou, Shanshan Zhang, Jun Wu, Liang Wang.
Incremental Generative Occlusion Adversarial Suppression Network for Person ReID（ Pedestrian recognition based on incremental occlusion generation and confrontation suppression ）.
IEEE Transactions on Image Processing, 2021. [pdf][code]

---

Pedestrian recognition based on incremental occlusion generation and confrontation suppression

Zhaocairong   ,  LV Xinbi   ,  Dou Shuguang   ,  Zhang Shanshan   ,  Wu Jun   ,  Wang Liang   

   Tongji University ,   Nanjing university of science and technology ,   Fudan University ,   Institute of automation, Chinese Academy of Sciences

TIP 2021

writer ： Zhaocairong , LV Xinbi , Dou Shuguang

Recommended director ： Lin Zechen

Original title ：Incremental Generative Occlusion Adversarial Suppression Network for Person ReID

Original address ：https://ieeexplore.ieee.org/abstract/document/9397375

Code link ：https://github.com/Vill-Lab/IGOAS

◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆

Abstract

In an occluded scene , Pedestrian images contain occlusion and less discriminative pedestrian information . Previous work designed complex modules to capture implicit information （ Including key points of human posture , Mask graph and spatial information ） To achieve effective alignment . A small amount of research work focuses on data enhancement , Only bring limited performance improvement . In order to solve the occlusion problem , We propose a new method to generate occlusion and combat suppression （Incremental Generative Occlusion Adversarial Suppression ,IGOAS） Method . Blocking data Occluded-DukeMTMC On , Our approach is Rank-1 and mAP The indicators have reached 60.1% and 49.4%.

background

In order to solve the occlusion problem , Existing methods constantly try to design complex modules to capture implicit information （ Such as attitude key points and mask diagram ）. This is to force the network to focus on the distinguishing features of non occluded body areas and further realize the matching under spatial dislocation .

Using data enhancement has its drawbacks

Generally speaking , Data enhancement does not require any additional parameter learning and can effectively improve the model's response to data changes （ Including occlusion ） Robustness . However , The random generation strategy of training samples due to difficulties , Its performance improvement is limited .
Besides , The traditional data enhancement method based on single sample cuts a single image randomly [3]、 Random erase [4] Wait for the operation , Sometimes it will make the training data more complex and diverse .

Inspired by the learning strategies and confrontational thoughts from easy to difficult , We propose a novel incremental generation occlusion antagonism suppression （IGOAS） Network to solve this challenging problem .
say concretely , We first propose an incremental occlusion generation （IGO） modular . It is different from the traditional data enhancement method based on single sample ,IGO A method from easy to difficult is used to generate occlusion data instead of random , This makes the network more robust to occlusion . secondly , We propose a global and antagonistic suppression (G&A) frame , Make the model ignore the generated occlusion area , This creates a confrontational process . The schematic diagram of this method is shown in Figure 1 Shown .

IGOAS Network flow chart

Fig. 2. The flowchart of the proposed IGOAS network：
1、Specifically, in the training phase, the IGO blockconverts the raw input into occluded data,
2、and then the raw data and the occluded data are entered into the respective branch of the frame for feature extraction.

• In global branch, we retain the ResNet-50 baselineto extract steady global features of the raw data.
• In adversarial suppression branch, the OSM and a global max pooling operation are employing to force this branch to suppress the occlusion’s response and strengthen discriminative feature representation on non-occluded regions of the pedestrian.

3、Finally, we get a more robust pedestrian feature descriptor by concatenating two branches’ features. And in the test phase, the incremental occlusion block won’t be performed.

Global and confrontation suppression framework

G&A The framework consists of a backbone network 、 A global branch and an antagonistic inhibition Branch .
We use Global branch Come on Learn the overall characteristics of robustness .
and Fight against suppressed branches Aimed at By suppressing the response of the generated occluded area to zero , So as to pay more attention to the prospect information .
We design a occlusion suppression module （OSM） To achieve this goal .

chart 2. OSM Structure .   Represents element level multiplication

OSM The structure of is shown in the figure 2 Shown . step ：

1. Input characteristics    It is first sent to the attention module to obtain refined features   . In this paper , We use CBAM[8] As OSM The attention module .
2. then , Through binary mask and    Perform element level operation multiplication to get   . The binary mask is obtained by scaling the artificially designed image occlusion mask .
3. Last ,  Monitor through mask loss  （ Lose pair through mask    To supervise ）, In this way, the model can learn to ignore the background area in the process of back propagation （ chart 2 in     The black part of ）.

Mask loss can be expressed as ：

  among    Represents mean square error . More specifically . The mask loss function makes the feature in the region corresponding to the feature in the occluded region as zero as possible . Because the location of occlusion is known and random , It can be used as the monitoring information of the attention module to learn to suppress the generated occlusion response .

 Fig. 5. Comparison of (a) single-based random occlusion block, (b) batch-based incremental occlusion block. In (a), each data in the batch suffers from a variable-size and variable-position occlusion. In (b), all data in the batch suffer from occlusions with a uniform size and position. But as the number of iterations increases, it allows to generate variable-size, variable-position, and easy-to-hard occlusions.

Incremental generation occlusion module

In order to strengthen the network response to the recognition of blocked pedestrians , We propose a data enhancement method based on batch samples - Incremental generation occlusion module . We randomly generate occlusion data from easy to difficult to simulate occluded images , Learn more difficult occlusion gradually instead of learning the most difficult occlusion directly , Make the network more robust to occlusion . The details of the algorithm are as follows ：

And other data enhancement methods Batch DropBlock[11], Slow-Drop Block[14] and Batch Random Erasing Block The comparison is shown in the figure below ：

chart 3. IGO Compared with three data enhancement methods

experimental result

We are in two occluded pedestrian data sets -Occluded-DukeMTMC and Occluded-REID And two complete pedestrian data sets -Market-1501 and DukeMTMC-reID Evaluate the proposed algorithm . In order to prove the effectiveness of our proposed method , We propose a IGOAS The basic version . On the one hand, we directly use the random erasure module based on batch samples to generate occlusion , On the other hand , We use CBAM To replace OSM Location in the network . We call this method of combination BRE+CBAM.

surface 1.  stay Occluded-DukeMTMC Performance on

surface 2.  stay Occluded-ReID Performance on

surface 3.  Performance on two complete pedestrian data sets

Experimental results show that , Our method is compared with the most advanced occlusion method HONet and MHSA Can get better results . In the complete pedestrian data set ,IGOAS It also shows competitive performance . It is worth noting that , Most of the above methods use additional information and complex network structure . Our method does not use any additional information , Only by generating occlusion and then suppressing the generated occlusion to solve the occlusion problem in pedestrian recognition .