This study aims to design and implement an emotion recognition AI system by combining Convolutional Neural Networks (CNN) with speech recognition technology. The system processes multimodal data and classifies emotions using the FER-2013 image dataset and the RAVDESS audio dataset. CNNs analyze image data, while Long Short-Term Memory (LSTM) models process audio data to predict emotional states. The results of the two models are integrated to develop a system that classifies emotions with greater precision.

This approach enhances the accuracy of emotion recognition technology and suggests its applicability in diverse real-world scenarios, such as human-computer interaction, mental health monitoring, and customer service. The study's findings propose data augmentation and feature expansion to improve recognition accuracy and provide a foundation for the advancement of multimodal emotion recognition technologies in the future.

Keywords

Emotion recognition, CNN, Voice recognition, RAVDESS and LSTM