Multimodal emotion recognition_ Research on emotion recognition based on multimodal fusion

Hello everyone , I meet you again , I'm your friend, Quan Jun .

Abstract ：

Emotion is the important information that people transmit in the process of communication , The change of emotional state affects people's perception and decision-making . Emotion recognition is an important research field of pattern recognition , It introduces the emotional dimension into human-computer interaction . Modes of emotional expression include facial expressions 、 voice 、 posture 、 Physiological signals 、 Words etc. , Emotion recognition is essentially a multi-modal fusion problem . A multi-modal fusion algorithm for emotion recognition is proposed , Extract facial expression and speech features from facial image sequences and speech signals , Based on Hidden Markov model and multi-layer perceptron, an emotion classifier integrating expression and speech modality is designed . Establish an active appearance model of facial expression image , Realize the location and tracking of facial feature points ; According to the displacement of facial feature points , Calculate facial animation parameters as expression features . Time domain analysis of speech signal 、 And frequency domain analysis , Extract the short-time average energy of each frame 、 Pitch frequency and formant are used as speech features . Using the extracted facial expression and speech features , use Viterbi The algorithm trains hidden Markov models of various expressions and speech emotions ; Using the conditional probabilities of eigenvectors with respect to each hidden Markov model , Back propagation learning algorithm is used to train multilayer perceptron . Experimental results show that , The application of emotion recognition algorithm combining expression and speech in recognition samples 、 sad 、 anger 、 It has a high accuracy in emotional states such as disgust . The proposed multimodal recognition algorithm makes good use of the emotional information in video and audio , Compared with the recognition result using only speech mode, it has a great improvement , Compared with the recognition result of expression mode, it also has some improvement , It is an emotion recognition algorithm that can be used .

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/151800.html Link to the original text ：https://javaforall.cn