New hybrid architecture iformer! Flexible migration of convolution and maximum pooling to transformer

Recent research shows that ,Transformer Strong ability to build remote dependencies , However, it performs poorly in capturing high-frequency information that transmits local information . To solve this problem , The author puts forward a new general Inception Transformer, abbreviation iFormer, It can effectively learn the comprehensive features of high-frequency and low-frequency information in visual data .

To be specific , The author designed a Inception mixer, To transplant the advantages of convolution and maximum pooling , Capture high frequency information to Transformer. Unlike the recent hybrid framework ,Inception mixer Higher efficiency is achieved through the channel splitting mechanism , Using parallel convolution / The maximum pooling path and the self attention path are used as high frequency and low frequency mixers , At the same time, it can flexibly model the identification information scattered in a wide frequency range .

Consider that the bottom layer plays a greater role in capturing high-frequency details , The top layer plays a greater role in modeling low-frequency global information , The author further introduces a frequency gradient structure , That is, gradually reduce the size fed to the high frequency mixer , Increase the size of the feed to the low frequency mixer , It can effectively balance high frequency and low frequency components between different layers .

In a series of visual tasks, the author studied iFormer Benchmarking , And shows its application in image classification 、COCO Detection and ADE20K Amazing performance in segmentation . for example ,iFormer-S stay ImageNet-1K Up to 83.4% Of Top-1 precision , Than DeiT-S Higher than 3.6%, Even bigger than Swin-B（83.3%） Slightly better , Only 1/4 Parameters and 1/3 Of FLOPs.

Paper title ：

Inception Transformer

Address of thesis ：

https://arxiv.org/abs/2205.12956

Code address ：

https://github.com/sail-sg/iFormer