Latent Acoustic Representation Learning via Self-Supervision with Interpretable Ensemble Inference for Urban Sound Intelligence

Abstract

Rapid urbanization has significantly increased the demand for intelligent systems that can ensure efficient traffic management and improved public safety. Traditional methods for detecting road incidents, traffic congestion, and crime-related activities rely heavily on manual surveillance, basic sensors, or human intervention, which often suffer from limited scalability, delayed responses, and inconsistent accuracy. These limitations make them inadequate for dynamic urban environments requiring real-time decision-making. To address these challenges, this study proposes an automated acoustic event recognition framework for urban traffic monitoring and safety analysis. The system leverages environmental audio signals and classifies them into multiple event categories such as vehicle collisions, honking, congestion noise, and suspicious or crime-related sounds. At its core, the Transformer Encoder for Representations of Audio (TERA) is employed to extract rich temporal and spectral features from raw audio data, transforming them into high-dimensional representations. These features are then processed using supervised machine learning models including Categorical Boosting (CB), Histogram-Based Gradient Boosting (HGB), Extra Trees (ET), and a novel Tree-based Generalized Additive Model (TGAM). The integration of advanced feature extraction with robust classifiers enables accurate modeling of complex acoustic patterns. Experimental results demonstrate that the TGAM model achieves superior performance, with macro-level precision, recall, and F1-scores exceeding 90% on balanced datasets. The system also includes visualization tools such as confusion matrices, ROC curves, and waveform plots for interpretability. Implemented as a role-based desktop application using Tkinter, it supports secure authentication via TinyDB and SHA-256 encryption, ensuring reliable and user-friendly operation.