MULTIMODAL MISINFORMATION AND DEEPFAKE DETECTION SYSTEM

Abstract

The exponential growth of social media platforms has significantly accelerated the spread of misinformation across diverse content formats, including textual narratives, images, memes, and videos. Conventional fake news detection approaches remain insufficient as they are predominantly designed for textual data, failing to address the complexities introduced by multimodal content and sophisticated deepfake media. This work proposes a comprehensive artificial intelligence-driven system for detecting misinformation and identifying deepfake content within a unified architecture. The system incorporates Natural Language Processing (NLP), Computer Vision (CV), Convolutional Neural Networks (CNN), and Retrieval-Augmented Generation (RAG) to deliver accurate, interpretable, and context-aware outcomes. Textual inputs are processed using advanced NLP techniques to extract key claims, while Optical Character Recognition (OCR) retrieves embedded textual information from images and memes for multimodal fusion. Video inputs employ frame-based CNN analysis, with frame-level predictions aggregated into a final classification. The system produces conclusive output categorizing content as True, False, or Misleading, accompanied by a confidence score and evidence-backed explanation, enhancing user trust and suitability for real-world applications