Track: Business Analytics

Abstract

This study utilized Machine Learning (ML) models to analyze and predict customer churns in the context of web-based collaboration platform. Secondary data was used in the study, containing 109,740 observations and 12 predictors. This was divided into test dataset and out-of-time (OOT) dataset where the prior was used for model fitting and the latter is to test performance stability in unseen data. Furthermore, Synthetic Minority Over-sampling Technique (SMOTE) was performed to resolve the data imbalance, hence, preventing bias and distortion in the models’ performance. These 3 ML models were assessed based on Accuracy, ROC-AUC, Precision, Recall and F1- Score. Given the business context’s applicability, F1-score and Accuracy were used as bases for performance, leading to the selection of Decision Tree Classifier as the ML model in this study, with Accuracy of 92.1% and F1-Score of 63%. Furthermore, hyper-parameter tuning was performed on Decision Tree Classifier to prevent overfitting. To reinforce the model selected, Survival Analysis was implemented, specifically, Kaplan-Meier (KM) Estimator and Cox Proportional Hazard (CPH) were utilized to analyze the rate and timeframe of disengagement to the platform, revealing that beyond 72 months, it was projected to retain only 60% of its user base. Hence, these multidimensional results and insights derived from both Decision Tree Classifier and Survival Analysis were anchored in the formulation of customer retention strategy, proactively target customers who are predicted to churn.