FROM DATA TO DECISIONS: THE IMPORTANCE OF MONITORING ML SYSTEMS IN INDUSTRIAL SETTINGS

Adrian-Ioan ARGESANU, Gheorghe-Daniel ANDREESCU

Abstract


Machine learning (ML) systems have been extensively used in various industrial applications, including process optimization, predictive maintenance, quality control, and decision support. The success of ML systems in these domains depends on their ability to make accurate predictions, decisions, and recommendations. With all ML models being inherently perishable, precautionary measures need to be put in place to ensure that degrading performance is detected before it yields negative outcomes. In this paper, we explore the importance of monitoring ML systems, and the various areas that need to be surveilled. We present a novel implementation, leveraging a bespoke combination of components to achieve complete visibility over the state of the ML solution at any given moment, discussing employed methods and how these provide insights for Process & Pipeline Services’ use-case of detecting mechanically induced stress cracking in pipelines. This paper adds to the limited literature on ML system monitoring.

Full Text:

PDF

References


E. Breck, S. Cai, E. Nielsen, M. Salib, D. Sculley, The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, Proceedings of IEEE Big Data, 2017.

A. -I. Argesanu, G. -D. Andreescu, A Platform to Manage the End-to-End Lifecycle of Batch-Prediction Machine Learning Models, 2021 IEEE 15th International Symposium on Applied Computational Intelligence and Informatics (SACI), 2021, pp. 329-334.

C. Sridharan, Distributed Systems Observability, O'Reilly Media, Inc., 2018, ISBN 9781492033424.

S. Shankar, A. Parameswara, Towards Observability for Machine Learning Pipelines, 2021, https://arxiv.org/abs/2108.13557.

Y. Wu, E.D. Yinjun, S.B. Davidson. DeltaGrad: Rapid retraining of machine learning models, International Conference on Machine Learning, 2020.

E.D. Nascimento, I., Ahmed, E. Oliveira, M.P. Palheta, I. Steinmacher, T.U. Conte, Understanding Development Process of Machine Learning Systems: Challenges and Solutions, 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 1-6, 2019.

S. Shankar, R. Garcia, J.M. Hellerstein, A.G. Parameswaran. Operationalizing machine learning: An interview study, arXiv preprint arXiv:2209.09125 (2022).

Crack capabilities overview, Process & Pipeline Services, accessed June 2023, https://www.bakerhughes.com/sites/bakerhughes/files/2020-07/19004_BH_PPS_ILI_US_BRO_1912%20%28CRACK%20CAPAB%29.pdf

G. Nguyen, S. Dlugolinsky, M. Bobák, et al. Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey, Artif Intell Rev 52, 77–124 (2019), https://doi.org/10.1007/s10462-018-09679-z

P. Baier, S. Dragiev, Challenges in Live Monitoring of Machine Learning Systems, The Upper-Rhine Artificial Intelligence Symposium UR-AI 2021, ARTIFICIAL INTELLIGENCE - APPLICATION IN LIFE SCIENCES AND BEYOND, 2021.


Refbacks

  • There are currently no refbacks.


JOURNAL INDEXED IN :