FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply

By Lingjiao Chen, Matei A. Zaharia, James Y. Zou. 2020

50 citations recorded by the source corpus.

Prediction APIs offered for a fee are a fast-growing industry and an important part of machine learning as a service. While many such services are available, the heterogeneity in their price and performance makes it challenging for users to decide which API or combination of APIs to use for their own data and budget. We take a first step towards addressing this challenge by proposing FrugalML, a principled framework that jointly learns the strength and weakness of each API on different data, and performs an efficient optimization to automatically identify the best sequential strategy to adaptively use the available APIs within a budget constraint. Our theoretical analysis shows that natural sparsity in the formulation can be leveraged to make FrugalML efficient. We conduct systematic experiments using ML APIs from Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including facial emotion recognition, sentiment analysis and speech recognition. Across various tasks, FrugalML can achieve up to 90% cost reduction while matching the accuracy of the best single API, or up to 5% better accuracy while matching the best API's cost.

Subfields: Artificial Intelligence, Efficiency & Compression, ML Systems & Hardware

Citations in the corpus

ALICE: Active Learning with Contrastive Natural Language Explanations (2020, neutral): “potential for reducing computation runtime (Chen et al., 2020; Mailthody et al., 2019).”
FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (2023, unknown): “Recent work Chen et al. (2020; 2022) builds a customized cascade for cost reduction, with a focus on classiﬁcation ML APIs.”
RouteLLM: Learning to Route LLMs with Preference Data (2024, unknown): “A closely related study, Hybrid-LLM [14], shares similarities with our framework but differs in three key ways: it uses synthetic preference labels derived via BARTScore [26], relies on a single BERT-based router architecture, and limits evaluation to in-domain generalization.”