Search

Facial Expression Recognition Algorithm Integrating Semantic Features and Attention Mechanisms

Author(s)

Xue Tian1, Shuili Zhang1,2,*, Rui Huo1

Affiliation(s)

1College of Physics and Electronic Information, Yanan University, Yan'an, Shaanxi, China 2Shaanxi Key Laboratory of Intelligent Processing for Big Energy Data, Yan'an, Shaanxi, China *Corresponding Author.

Abstract

Due to the insufficient feature values extracted by traditional CNN networks, existing facial expression recognition algorithms struggle to effectively cope with the diversity and complexity of human emotional expressions, as well as the influence of lighting conditions and environmental factors, this paper proposes a facial expression recognition algorithm that integrates semantic features and attention mechanisms. By embedding attention mechanisms into the U-Net network, more prominent facial expression features are extracted, and the improved U-Net structure network is integrated in each layer of Resnet34 to extract richer feature values. Utilizing each layer of ResNet34 to process features, combined with U-Net and ECA -Net to generate weights, and outputting the weights and feature values through a residual network, more significant facial expression features can be extracted, enhancing the robustness and generalization of the model. Experimental evidence shows that the algorithm achieves a recognition rate of 78.1% on the FER2013 facial expression dataset, indicating superior accuracy.

Keywords

U-Net; Resnet34; Attention Mechanism; FER2013 Dataset

References

[1]He J. "Research on the Different Meanings of Facial Expressions under Various Cultural Backgrounds". Proceedings of the International Conference on Interdisciplinary Humanities and Communication Studies (ICIHCS 2022) (part5). Ed. University California Davis; 2022, 300-304. [2]Luo Sishi, Li Maojun, Chen Man. Facial expression recognition network with multi-scale fusion attention mechanism. Computer Engineering and Applications, 2023, 59 (01):199-206. [3]Buciu I,Pitas I.ICA and Gabor representation for facial expression recognition//Proceedings 2003 International Conference on Image Processing (Cat. No. 03CH37429). IEEE, 2003, 2: II-855. [4]Zhang B, Liu G, Xie G. Facial expression recognition using LBP and LPQ based on Gabor wavelet transform //2016 2nd IEEE international conference on computer and communications (ICCC). IEEE, 2016: 365-369. [5]Shan C, Gong S, Mcowan P W. Facial Expression Recognition Based on Local Binarypatterns: A Comprehensive Study. Image and Vision Computing, 2009, 27(6):803. [6]Zhu Y, Li X, Wu G. Face expression recognition based on equable principal component analysis and linear regression classification//2016 3rd International Conference on Systems and Informatics (ICSAI). IEEE, 2016: 876-880. [7]Shi Y, Lv Z, Bi N and Zhang C. An improved SIFT algorithm for robust emotion recognition under various face poses and illuminations. Neural Computing and Applications, 2020, 32: 9267-9281. [8]Wang K, Peng X, Yang J, Meng D and Qiao Y. Region attention networks for pose and occlusion robust facial expression recognition. IEEE Transactions on Image Processing, 2020, 29: 4057-4069. [9]Nie H. Face expression classification using squeeze-excitation based VGG16 network//2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE). IEEE, 2022: 482-485. [10]Chen Y, Wang J, Chen S, Shi Z and Cai J. Facial motion prior networks for facial expression recognition//2019 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2019: 1-4. [11]Wang Q, Wu B, Zhu P, Li P, Zuo W and Hu Q. ECA-Net: Efficient channel attention for deep convolutional neural networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020, 11531-11539. [12]Hu J, Shen L, Sun G. Squeeze-and-excitation networks//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132-7141. [13]Li J,Jin K,Zhou D, Kubota N and Ju Z. "Attention mechanism-based CNN for facial expression recognition." Neurocomputing 411 (2020): 340-350. [14]Wang Z, Zeng F, Liu S, Zeng B."OAENet: Oriented attention ensemble for accurate facial expression recognition." Pattern Recognition 112 (2021): 107694. [15]Zhang Dongyu, Zhao Lei. Facial Expression Recognition with Improved ResNet Integrating Attention Mechanism. Computer Technology and Development, 2023, 33(05):130-137. [16]He K, Zhang X, Ren S and Sun J. Deep residual learning for image recognition //Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778. [17]Miao K, Zhao N, Lv Q, He X, Xu M, et al. Prediction of benign and malignant ovarian tumors using Resnet34 on ultrasound images. Journal of Obstetrics and Gynaecology Research, 2023, 49(12): 2910-2917. [18]Zhang P, Jiang M, Li Y, Ling X,Wang Z,et al. An efficient ECG denoising method by fusing ECA-Net and CycleGAN. Mathematical Biosciences and Engineering, 2023, 20(7): 13415-13433. [19]Ronneberger O, Fischer P, Brox T .U-Net: Convolutional Networks for Biomedical Image Segmentation. CoRR, 2015, abs/1505.04597. [20]Pham L, Vu T H, Tran T A. Facial expression recognition using residual masking network//2020 25Th international conference on pattern recognition (ICPR). IEEE, 2021: 4513-4519. [21]Khaireddin Y, Chen Z. Facial emotion recognition: State of the art performance on FER2013. arxiv preprint arxiv:2105.03588, 2021. [22]Woo S, Park J, Lee J Y, Kweon I S. Cbam: Convolutional block attention module//Proceedings of the European conference on computer vision (ECCV). 2018: 3-19. [23]Santoso B E, Kusuma G P. Facial emotion recognition on FER2013 using VGGSPINALNET. Journal of Theoretical and Applied Information Technology, 2022, 100(7): 2088-2102. [24]Pecoraro R, Basile V, Bono V. Local multi-head channel self-attention for facial expression recognition. Information, 2022, 13(9): 419. [25]Boudouri Y, Bohi A. EmoNeXt: an Adapted ConvNeXt for Facial Emotion Recognition//2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2023: 1-6.