Efficient Log File Classification Using Template Mining and Machine Learning with Event ID Optimization

Abstract

Log files generated from largescale systems are critical for monitoring performance and detecting anomalies, but their massive size makes analysis challenging. This paper proposes an efficient log classification approach using template mining by converting log messages into compact event IDs, reducing data size and improving processing speed. Machine learning algorithms including Random Forest, Decision Tree, XGBoost, and LightGBM are applied, with SMOTE used to address class imbalance. Experimental results demonstrate that the proposed method achieves high classification accuracy (up to 99%) while minimizing training time and computational cost.