Abstract:
This study is conducted to evaluate six representative machine learning models—Support Vector Machine (SVM), Decision Tree (DT), Naïve Bayes (NB), Random Forest (RF), XGBoost, and Multilayer Perceptron (MLP)—for hail identification using X-band dual-polarization radar observations and groundbased hail reports collected in the Chengdu region during 2022—2024. Eleven hail-related features are extracted to construct a labeled dataset. Using Accuracy (ACC), False Alarm Ratio (FAR), Probability of Detection (POD), F1 Score, Critical Success Index (CSI), area under the ROC curve (AUC), Equitable Threat Score (ETS), Heidke Skill Score (HSS), and Bias, and incorporating five-fold cross-validation, the models are systematically evaluated for discriminative capability, operational suitability, and cross-fold stability; moreover, practical detection performance and false-alarm risk are examined through case-based analyses. Feature-importance analyses indicate that dual-polarization parameters consistently make substantial contributions across models. The main findings are as follows: (1) XGBoost delivers the best overall performance, leads across core metrics such as ACC, POD, FAR, F1, CSI, AUC, ETS, and HSS, shows minimal variability across folds, and—in case-based tests—achieves perfect separation between hail and non-hail samples, indicating stable and reliable detection capability. (2) MLP ranks second, exhibits uniformly strong scores on the core metrics, and presents well-balanced performance with good generalization. (3) SVM is slightly inferior to MLP; its more conservative decision policy confers an advantage in controlling false alarms. (4) RF and DT demonstrate appreciable hail identification capability; however, elevated FAR suggests limited generalization, and overforecasting is observed in case-based tests. (5) NB exhibits the weakest overall capability, and its discriminative accuracy is insufficient for operational needs. Building on the strengths of each model, a business-oriented, flexibly configurable multi-model ensemble is proposed to provide a reliable technical pathway for hail identification and warning across diverse scenarios.