AN INTELLIGENT DJANGO-BASED FRAMEWORK FOR PREDICTIVE DATA CORRUPTION DETECTION

Abstract

In the era of big data, maintaining data integrity is crucial, as even minor corruptions can significantly distort analytical insights across domains such as e-commerce, social network analysis, and cybersecurity. Traditional manual inspection techniques and legacy anomaly detection models like LOF, Isolation Forest, and One-Class SVM often fall short in identifying subtle corruptions, especially in high-dimensional or evolving datasets. Furthermore, these approaches lack predictive capabilities and adaptability to unseen data, making them inadequate for real-time applications. Addressing these limitations, this research presents a novel full-stack Django-based framework that integrates a custom graph-theory-driven model—PAACDA (Proximity-based Adamic-Adar Corruption Detection Algorithm)—with a supervised Random Forest classifier. PAACDA employs the Adamic-Adar similarity index to calculate a proximity-based anomaly score for each data point, flagging outliers whose similarity exceeds a dynamic threshold based on local deviations (mean/4). To enhance predictive accuracy, PAACDA-generated features are used to train a Random Forest classifier, creating a hybrid model that achieves good detection accuracy and generalizes well across unseen datasets. This system is deployed via Django, with an interactive web interface displaying performance metrics and visualizations, enabling real-time anomaly detection and prediction. The proposed solution demonstrates superior performance, scalability, and adaptability, making it highly effective for dynamic, data-intensive environments.