Last-mile delivery accounts for up to 53% of total supply chain costs, yet on-time delivery performance remains volatile due to weather disruptions, traffic congestion, and warehouse overloads. Proactive delay prediction offers logistics operators a principled path to early intervention. However, existing machine learning (ML) studies suffer from three persistent gaps: inconsistent benchmarking methodology, neglect of class imbalance (delays are inherently rare events), and opaque models that provide no actionable operational insights. This paper presents a systematic benchmark of three ML models—Logistic Regression (LR), Random Forest (RF), and XGBoost—evaluated on a synthetic 50,000-shipment last-mile delivery dataset with 12 operational features. Hyperparameters are tuned via randomized search with 5-fold cross-validation (CV), and robustness is assessed across five independent random seeds. We apply the Synthetic Minority Oversampling Technique (SMOTE), class-weight strategies, and scale_pos_weight to address imbalance, and use SHapley Additive exPlanations (SHAP) to interpret the best-performing model. After tuning, XGBoost achieves the highest test-set F1-score (0.516), Precision-Recall Area Under the Curve (PR-AUC) of 0.503, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC) of 0.681, closely followed by LR (F1 = 0.514, AUC-ROC = 0.680). Multi-seed experiments confirm stable performance (XGBoost F1 = 0.513 ± 0.008; LR F1 = 0.513 ± 0.004). SHAP analysis identifies warehouse load factor, weather condition, and traffic index as the top three delay drivers, with rankings perfectly stable across three independent seeds. These findings deliver a reproducible, interpretable framework that logistics practitioners can deploy using routinely available Transportation Management System (TMS) data for proactive delay mitigation.