Hidden Technical Debt in Machine Learning Systems

By David Sculley, Gary Holt, D. Golovin, Eugene Davydov, Todd Phillips, D. Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, Dan Dennison. 2015

1,552 citations recorded by the source corpus.

Subfields: ML Systems & Hardware, Software Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

In Software Engineering

Citations in the corpus

Generative Models for Effective ML on Private, Decentralized Datasets (2019, background): “As Sculley et al. (2015) puts it, “ML is required in exactly those cases when the desired behavior cannot be effectively expressed in software logic without dependency on external data.””
Model-Reuse Attacks on Deep Learning Systems (2018, background): “On the upside, this “plug-and-play” paradigm significantly simplifies and expedites the development cycles of ML systems [53].”
Anchors: High-Precision Model-Agnostic Explanations (2018, background): “2008), propagate feedback loops (Sculley et al. 2015), or fail to notice data leaks (Kaufman, Rosset, and Perlich 2011).”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, background): “ater impact on industry and on the fairness of ML systems in practice. 2 BACKGROUND AND RELATED WORK The design, prototyping, and maintenance of machine learning systems raises many unique challenges [30, 67, 85, 95] not commonly faced with other kinds of intelligent systems or computing systems mor”
Efficient Feature Transformations for Discriminative and Generative Continual Learning (2021, background): “Pragmatically, this typically means that data collection must be completed before training a neural network, which can be problematic in settings like reinforcement learning [34] or the real world [44], which is constantly evolving.”
Clipper: A Low-Latency Online Prediction Serving System (2016, background): “The resulting choice of model is typically static and therefore susceptible to changes in model performance due to factors such as feature corruption or concept drift [52].”
SPGISpeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition (2021, background): “[4, 5, 6, 7, 8, 9]), and improvements to one model’s performance may degrade end-to-end performance as a whole due to distribution shift [10].”
TFX: A TensorFlow-Based Production-Scale Machine Learning Platform (2017, background): “However, putting together several disjoint components to do the job can result in signiﬁcant technical debt in forms of hard-to-maintain glue code, hidden dependencies, feedback loops, etc. [19].”
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems (2016, background): “…contribution of crowdsourcing to machine learning has been mostly limited to the creation of ofﬂine data sets for learning ( e.g. , (Lin et al. 2014; Sheng, Provost, and Ipeirotis 2008)), with recent interest in active crowd participation to the development of machine learning algorithms (Cheng…”
Debugging Tests for Model Explanations (2020, background): “Diagnosing and ﬁxing model errors–model debugging–remains a longstanding machine learning challenge [12, 14–17, 55, 73].”
Related Pins at Pinterest: The Evolution of a Real-World Recommender System (2017, methodology): “Because engagement logs are used for training, we introduced a direct feedback loop , as described in [22]: the model that is currently deployed dramatically impacts the training examples produced for future models.”
Unsolved Problems in ML Safety (2021, background): “They are fragile, rarely correctly handle all test cases, and cannot become error-free with short code patches [157].”
Towards A Rigorous Science of Interpretable Machine Learning (2017, background): “…criteria such as safety [Otte, 2013, Amodei et al., 2016, Varshney and Alemzadeh, 2016], nondiscrimination [Bostrom and Yudkowsky, 2014, Ruggieri et al., 2010, Hardt et al., 2016], avoiding technical debt [Sculley et al., 2015], or providing the right to explanation [Goodman and Flaxman, 2016].”
Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development (2021, neutral): “[101], data work is largely erased, despite its importance to machine learning practices.”
The Scientific Method in the Science of Machine Learning (2019, background): “Many of these concerns, both social and technical, spring from the lack of understanding of the root causes of performance in ML systems (Rahimi & Recht, 2017b; Lipton & Steinhardt, 2018; Sculley et al., 2018b; 2015; Woods, 2018).”
SLEAP: A deep learning system for multi-animal pose tracking (2022, methodology): “SLEAP was built using industry-standard best practices in both software engineering and machine learning system design 27 .”
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI (2021, background): “Although the behaviour of AI systems is critically determined by data, even more so than code [111]; many of our practitioner strategies mirrored best practices in software engineering [38, 83].”
Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021, background): “loop in which our models become even more confdent in their own incorrect decisions [43].”
Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making (2018, background): “The notion of a visibility debt has received some attention from engineers of both machine learning and traditional software engineering [56, 67].”