MatSciBERT: A materials domain language model for text mining and information extraction

By Tanishq Gupta, Mohd Zaki, N. Krishnan, Mausam. npj Computational Materials, 2021

320 citations recorded by the source corpus.

A large amount of materials science knowledge is generated and stored as text published in peer-reviewed scientific literature. While recent developments in natural language processing, such as Bidirectional Encoder Representations from Transformers (BERT) models, provide promising information extraction tools, these models may yield suboptimal results when applied on materials domain since they are not trained in materials science specific notations and jargons. Here, we present a materials-aware language model, namely, MatSciBERT, trained on a large corpus of peer-reviewed materials science publications. We show that MatSciBERT outperforms SciBERT, a language model trained on science corpus, and establish state-of-the-art results on three downstream tasks, named entity recognition, relation classification, and abstract classification. We make the pre-trained weights of MatSciBERT publicly accessible for accelerated materials discovery and information extraction from materials science texts.

Subfields: Artificial Intelligence, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/b146be9e80c66a6e062a1525693311fac65ae19e

Citations in the corpus

• ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text Embeddings (2025, unknown): “Some models, such as MatSciBERT [19] and ChemicalBERT [20], extend the pre-training of general-purpose encoders like BERT on chemical corpora.”
• ChemTEB: Chemical Text Embedding Benchmark, an Overview of Embedding Models Performance & Efficiency on a Specific Domain (2024, unknown): “To the best of our knowledge, the only existing embedding models specifically adapted to the chemical domain are MatSciBERT [44], and ChemicalBERT (from Recobo 4 ).”