Face and Facial Expressions Recognition System for Blind People Using ResNet50 Architecture and CNN

Corresponding Author ORCID Identifer

Kok-Why Nghttps://orcid.org/0000-0003-4516-4634

Section

Regular Issue

Abstract

Many blind individuals have difficulties in recognizing people’s facial expression which may impact their social interaction. With the recognition, the blind individuals can accurately interpret and respond to the emotions. There is a lack in the existing application with the combination of face and facial expressions recognition. The blind individuals have to rely on multiple applications to accomplish the same task, making it difficult and time-consuming for them to use. The paper aims to recognize faces and facial expressions for blind individuals and provides feedback in real-time. Three face detection algorithms of Haar Cascade Classifier, Dlib, and RetinaFace are compared. Dlib is chosen to process with Histogram of Oriented Gradients (HOG) and Support Vector Machine (SVM). It loads the pre-trained model, computes the HOG features, slide the window scanning at different scales, classify the windows using the SVM classifier, generate bounding boxes, and applying non-maximum suppression. ResNet50 architecture is employed to recognize face and Convolutional Neural Networks (CNN) is applied to recognize facial expression. The training accuracy is 70% and validation accuracy is 60%.

Cite This Article

Lee, J.-R., Ng, K.-W., & Yoong, Y.-J. (2023). Face and Facial Expressions Recognition System for Blind People Using ResNet50 Architecture and CNN. Journal of Informatics and Web Engineering, 2(2), 284–298. https://doi.org/10.33093/jiwe.2023.2.2.20