Lung Cancer Detection: A Comparative Approach Using Regressor Methods

Abstract

Early and accurate detection of lung cancer is critical due to its high mortality rate and the potential for improved patient outcomes through timely intervention. Traditional diagnostic methods, relying primarily on imaging and clinical assessment, often face challenges such as delayed detection, subjective interpretation, and limited predictive accuracy. This investigation employs a comprehensive lung cancer risk dataset, preprocessed through normalization using StandardScaler and MinMaxScaler, along with feature selection via Recursive Feature Elimination with Support Vector Machines. Class imbalance was addressed using Synthetic Minority Oversampling Technique (SMOTE). Multiple regression and classification algorithms, including Logistic Regression, K-Nearest Neighbors, Gaussian and Multinomial Naive Bayes, Decision Tree, Support Vector Classifier, Random Forest, XGBoost, Gradient Boosting, and Multi-Layer Perceptron, were trained and optimized through RandomizedSearchCV with K-Fold cross-validation. A Voting Classifier combining Random Forest, Decision Tree, and Extra Trees was implemented, and model interpretability was enhanced using explainable AI techniques such as LIME and SHAP, with deployment facilitated via the Flask framework. Evaluation using accuracy, precision, recall, F1-score, and confusion matrices demonstrated that the Voting Classifier outperformed all individual models, achieving 98.1% across all metrics. The integration of ensemble learning with explainable AI provides enhanced diagnostic reliability, transparency, and interpretability, significantly improving lung cancer detection accuracy over conventional approaches.