Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm

By Alex Dunn, Qi Wang, Alex M. Ganose, Daniel Dopp, Anubhav Jain. npj Computational Materials, 2020

457 citations recorded by the source corpus.

We present a benchmark test suite and an automated machine learning procedure for evaluating supervised machine learning (ML) models for predicting properties of inorganic bulk materials. The test suite, Matbench, is a set of 13 ML tasks that range in size from 312 to 132k samples and contain data from 10 density functional theory-derived and experimental sources. Tasks include predicting optical, thermal, electronic, thermodynamic, tensile, and elastic properties given a material’s composition and/or crystal structure. The reference algorithm, Automatminer, is a highly-extensible, fully automated ML pipeline for predicting materials properties from materials primitives (such as composition and crystal structure) without user intervention or hyperparameter tuning. We test Automatminer on the Matbench test suite and compare its predictive power with state-of-the-art crystal graph neural networks and a traditional descriptor-based Random Forest model. We find Automatminer achieves the best performance on 8 of 13 tasks in the benchmark. We also show our test suite is capable of exposing predictive advantages of each algorithm—namely, that crystal graph methods appear to outperform traditional machine learning methods given ~10 4 or greater data points. We encourage evaluating materials ML algorithms on the Matbench benchmark and comparing them against the latest version of Automatminer.

Subfields: Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/5e49a142f0c1bcfd80c11af68b31f78e04134d49

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

Citations in the corpus

Leveraging large language models for predictive chemistry (2024, unknown): “The yellow line we obtained using the Automatminer 25 , which uses as input the chemical composition.”
A universal graph deep learning interatomic potential for the periodic table (2022, methodology): “33 ).”