Methods for comparing uncertainty quantifications for material property predictions

By Kevin K Tran, Willie Neiswanger, Junwoong Yoon, Qingyang Zhang, Eric P. Xing, Zachary W. Ulissi. 2019

208 citations recorded by the source corpus.

Data science and informatics tools have been proliferating recently within the computational materials science and catalysis fields. This proliferation has spurned the creation of various frameworks for automated materials screening, discovery, and design. Underpinning these frameworks are surrogate models with uncertainty estimates on their predictions. These uncertainty estimates are instrumental for determining which materials to screen next, but the computational catalysis field does not yet have a standard procedure for judging the quality of such uncertainty estimates. Here we present a suite of figures and performance metrics derived from the machine learning community that can be used to judge the quality of such uncertainty estimates. This suite probes the accuracy, calibration, and sharpness of a model quantitatively. We then show a case study where we judge various methods for predicting density-functional-theory-calculated adsorption energies. Of the methods studied here, we find that the best performer is a model where a convolutional neural network is used to supply features to a Gaussian process regressor, which then makes predictions of adsorption energies along with corresponding uncertainty estimates.

Subfields: Computational Theory and Mathematics, Materials Chemistry

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

Citations in the corpus

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, methodology): “The experimental setup is similar to the previous section, but we compute a standard measure of calibration: root mean squared calibration error (RMSCE) on the test set (Kuleshov, Fenner, and Ermon 2018; Tran et al. 2020).”