EMOJIS TO EMOTIONS: A HOLISTIC MULTIMODAL APPROACH FOR MENTAL HEALTH MONITORING ON SOCIAL MEDIA PLATFORMS

Abstract

The widespread use of social media has transformed how individuals express emotions, with emojis emerging as a powerful non-verbal communication tool alongside text. However, most existing mental health monitoring systems rely primarily on textual analysis, overlooking the emotional depth conveyed through emojis and user behavior patterns. This project proposes a holistic multimodal approach that integrates textual sentiment, emoji interpretation, and behavioral context to improve the detection of mental health conditions such as stress, anxiety, and depression. The system collects social media data and performs preprocessing to extract meaningful features from text and emojis. Advanced Natural Language Processing (NLP) techniques and deep learning models are used to analyze text, while emojis are mapped to emotional representations to enhance sentiment understanding. Behavioral features such as posting frequency and temporal activity patterns are also incorporated to capture psychological trends over time. These multimodal features are fused using a deep learning framework to classify users into different mental health categories. Experimental results indicate that the proposed approach significantly outperforms traditional text-only models by providing higher accuracy and better contextual understanding of user emotions. The system enables early detection and continuous monitoring, making it a valuable tool for mental health awareness and intervention. This research highlights the importance of leveraging multimodal data for more effective and reliable mental health analysis in digital environments.