Machine Learning Behind The Scenes: An Exploratory Study in Fintech

Abstract—Artificial Intelligence has become increasingly important for organizations. Pioneers in the Artificial Intelligence industry are asking how to better develop and maintain Artificial Intelligence software. This paper focuses on Machine Learning, the branch of Artificial Intelligence that deals with the automatic generation of knowledge models based on sample data. This study aims to understand the evolution of the processes by which Machine Learning applications are developed and how state-of-the-art lifecycle models fit the current needs of the fintech industry. We conducted an exploratory case study at \ING, a global bank with a strong European base. We interviewed 17 people with different roles and from different departments within the organization. We analyze existing lifecycle models in the literature and refine them by adding stages for data collection, a feasibility study, documentation, model monitoring, and a sub-stage of model risk assessment within model evaluation. The results indicate that the existing lifecycle models CRISP-DM and TDSP largely reflect the current development processes of Machine Learning applications, but there are crucial steps missing from the fintech perspective, including a feasibility study, documentation, model evaluation, and model monitoring. Our work shows that the real challenges of applying Machine Learning go much beyond sophisticated learning algorithms -- more focus is needed on the entire lifecycle.