IJCAI 2022 | greatly improve the effect of zero sample learning method with one line of code. Nanjing Institute of Technology & Oxford proposed the plug and play classifier module

Zero sample learning （Zero-Shot Learning） Focus on categorizing categories that have not appeared during training , The zero sample learning based on semantic description is realized from the visible class by pre-defined high-order semantic information of each class （seen class） No class found （unseen class） The transfer of knowledge . The traditional zero sample learning only needs to recognize the unknown classes in the testing stage , And generalized zero sample learning （GZSL） You need to identify both visible and unseen classes , The evaluation index is the harmonic average of the average accuracy of the visible class and the average accuracy of the unseen class .

A general zero sample learning strategy is to use visible class samples and semantic training to generate conditional models from semantic space to visual sample space , With the help of the semantics of unseen classes, the pseudo samples of unseen classes are generated , Finally, the classification network is trained with visible class samples and unseen class pseudo samples .

However , To learn two modes （ Semantic modality and visual modality ） A good mapping between often requires a large number of samples （ reference CLIP）, This cannot be achieved in the traditional zero sample learning environment . therefore , The distribution of visual samples generated by using the semantics of unseen classes usually deviates from the distribution of real samples （bias）, That means two things ：1. The accuracy of this method is limited .2. When the average number of pseudo samples generated by each class is equal to the average number of samples generated by each class of the visible class , There is a large difference between the accuracy of the non observed class and that of the visible class , The following table 1 Shown .

We found that even if we only learn the mapping from semantics to category Center , It is also possible to copy a single sample point that has not been mapped to a class semantics for many times and then participate in the classifier training, which is close to the effect of using the generated model . This means that the pseudo sample features generated by the generation model are homogeneous to the classifier （homogeneity） Of .

Previous methods usually cater to this problem by generating a large number of unseen class artifacts GZSL Evaluation indicators （ Although a large number of samples are not helpful to distinguish between classes ）. However, this resampling （re-sampling） The strategy of learning in the long tail （Long-tail Learning） The domain is proved to cause the classifier to over fit some features , Here is the pseudo unseen feature offset from the real sample . This situation is not conducive to the identification of real samples of visible and unseen classes . that , Can we discard this resampling strategy , Instead, it will generate the offset and homogeneity of the unseen pseudo samples （ Or the categories of visible and unseen classes are not balanced ） As an inductive bias （inductive bias） Implant classifier learning ？

Based on this , We propose a plug and play classifier module , Only one line of code needs to be modified to improve the effect of the generative zero sample learning method . Each invisible class only needs to generate 10 Pseudo samples , You can reach SOTA level . Compared with other generative zero sample methods , The new method has great advantages in computational complexity . The research members are from Nanjing University of technology and Oxford University .

• The paper : https://arxiv.org/abs/2204.11822
• Code : https://github.com/cdb342/IJCAI-2022-ZLA

This paper is guided by the goal of consistent training and testing , The variational lower bound of the generalized zero sample learning evaluation index is derived . The classifier based on this model can avoid using the re - adoption strategy , Prevent the classifier from over fitting the generated pseudo samples, which will adversely affect the recognition of real samples . The proposed method can make the embedded classifier effective on the generative method framework , It reduces the dependence of the classifier on the quality of the generated pseudo samples .