Beyond natural motion: exploring the discontinuity of video interpolation

Paper title ： Beyond Natural Motion: Exploring Discontinuity for Video Frame Interpolation 

Thesis link ：https://arxiv.org/abs/2202.07291

Video interpolation is the task of synthesizing intermediate frames given two consecutive frames . Most of the previous studies have focused on the appropriate frame distortion operation and the thinning module of distorted frames . These studies are conducted on natural video with only continuous motion . However , Many practical videos contain many discontinuous actions , For example, chat window 、 watermark 、GUI Element or subtitle . We propose three techniques to extend the concept of transition between two consecutive frames , To solve these problems . The first is a new architecture that can separate continuous and discontinuous motion regions . We also propose a new data enhancement strategy , It is called graphic text blending (FTM), So that our model can learn more general scenarios . Last , We propose a loss function , Monitor the discontinuous motion area through data enhancement . We collected a special data set consisting of some mobile games and chat videos . We show that , Our method significantly improves the quality of video interpolation on special data sets . Besides , Our model is superior to the latest method for natural video data sets containing only continuous motion , for example DAVIS and UCF101.