NGBoost: Natural Gradient Boosting for Probabilistic Prediction
By Tony Duan, Anand Avati, Daisy Yi Ding, S. Basu, Andrew Y. Ng, Alejandro Schuler. International Conference on Machine Learning, 2019
474 citations recorded by the source corpus.
We present Natural Gradient Boosting (NGBoost), an algorithm for generic probabilistic prediction via gradient boosting. Typical regression models return a point estimate, conditional on covariates, but probabilistic regression models output a full probability distribution over the outcome space, conditional on the covariates. This allows for predictive uncertainty estimation -- crucial in applications like healthcare and weather forecasting. NGBoost generalizes gradient boosting to probabilistic regression by treating the parameters of the conditional distribution as targets for a multiparameter boosting algorithm. Furthermore, we show how the Natural Gradient is required to correct the training dynamics of our multiparameter boosting approach. NGBoost can be used with any base learner, any family of distributions with continuous parameters, and any scoring rule. NGBoost matches or exceeds the performance of existing methods for probabilistic prediction while offering additional benefits in flexibility, scalability, and usability. An open-source implementation is available at this http URL.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/3b432eea984904c926e2d6cc4dc2b70753499ca5
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Arber Zela
- Julien N. Siems
- Margret Keuper
- Jovita Lukasik
- Lucas Zimmer
- Frank Hutter
- Quoc V. Le
- Holger H. Hoos
Citations in the corpus
- NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search (2020, methodology): “, 2017) and NGBoost (Duan et al., 2020), recently used for predictorbased NAS (Luo et al.”