Track: Data Analytics and Big Data

Abstract

Models are abstractions of reality expressed in mathematical terms. Abstraction simplifies the underlying process and phenomenon being investigated. Model risk arises when the abstraction process is inappropriate as a result of data used or model selected. Model risk poses a problem for any organisation that relies on models to perform tasks and making decisions. The presence of model risk has resulted in the development of model validation techniques to detect inappropriate models. Model validation techniques, such as k -fold cross-validation, has been widely adopted for statistical models and machine learning models. With the advances of newer models such as AI, deep learning and esoteric machine learning techniques, there is a need to develop the appropriate model validation techniques to manage this risk. In this paper, we discuss the philosophical issues for model validation and model limitations. Through this discussion, we hope to provide possible solutions to tackle model risk as well as general principles to tackle model risk in the situation where there are no precedents.