Auditing Algorithmic Fairness: A Data-Centric Evaluation Framework

Abstract

Machine learning systems in sensitive domains can amplify biases from historical data, leading to unfair predictions. This paper presents a Fair ML Analysis System using the COMPAS recidivism dataset to evaluate and compare multiple ML models while measuring demographic fairness. Five classifiers (Logistic Regression, Random Forest, Decision Tree, Gradient Boosting, SVM) are trained and evaluated using both accuracy and Disparate Impact (DI) fairness metrics. The system implements a complete pipeline including data preprocessing, feature engineering, model training, and fairness evaluation. A Django web dashboard enables interactive analysis and real-time prediction. Results reveal significant accuracy-fairness trade-offs: Random Forest achieves highest accuracy (78.4%) but lowest DI (0.62), while Logistic Regression achieves better fairness (DI: 0.81) with moderate accuracy (72.1%), highlighting the critical need for fairness-aware model selection.