REAL TIME BANK TRANSACTION FRAUD DETECTION USING KAFKA AND MACHINE LEARNING

Abstract

The rapid expansion of digital banking and online financial services has significantly increased the volume and velocity of transaction data, making fraud detection a critical challenge for modern financial systems. Traditional fraud detection mechanisms are often batch-based, reactive, and unable to process high-frequency transaction streams in real time, resulting in delayed responses and increased financial risks. To address these limitations, this project proposes a Real-Time Bank Transaction Fraud Detection System using Apache Kafka and Machine Learning (ML), which combines real-time data streaming with intelligent predictive analytics. The proposed system utilizes Apache Kafka, a distributed streaming platform, to handle continuous transaction data flow through a producer-consumer architecture. The producer publishes transaction records to Kafka topics, while the consumer continuously retrieves the streaming data and forwards it to the ML model for analysis. The machine learning component, implemented using algorithms such as Random Forest, is trained on historical transaction data to identify patterns of normal and fraudulent behavior. Data preprocessing techniques are applied to convert non-numeric attributes into numerical format and normalize features, ensuring compatibility with ML models. Once deployed, the system performs real-time fraud detection by analyzing incoming transaction streams and classifying them as either legitimate or fraudulent. The integration with a Flask-based web interface allows users to generate Kafka streams, consume data, and visualize prediction results dynamically. Experimental results demonstrate the system’s ability to process streaming data efficiently and provide instant predictions, making it scalable and suitable for real-world banking applications. Overall, the proposed approach enhances fraud detection accuracy, reduces response time, and improves financial security by enabling proactive and real-time decision-making.