Unsupervised word embeddings capture latent knowledge from materials science literature

By V. Tshitoyan, John Dagdelen, Leigh Weston, Alex Dunn, Ziqin Rong, Olga Kononova, Kristin A. Persson, Gerbrand Ceder, Anubhav Jain. 2019

1,028 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Renewable Energy, Sustainability and the Environment, Representation & Self-Supervised Learning, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Renewable Energy, Sustainability and the Environment

In Representation & Self-Supervised Learning

In AI for Physical Sciences

Citations in the corpus

CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, methodology): “In NLP, they were also shown to produce better language models, and are being widely used both for language representation and for classification in nearly every subfield (Tshitoyan et al. 2019; Gao et al. 2019; Lee et al. 2020; Feder et al. 2020).”
Strategies for Pre-training Graph Neural Networks (2019, background): “Investigating what pre-trained models have learned would also be useful to aid scientiﬁc discovery (Tshitoyan et al., 2019).”
Emerging materials intelligence ecosystems propelled by machine learning (2020, background): “Moreover, it was demonstrated 34 that a wealth of materials science knowledge is also captured by the latent space of word embeddings, including the underlying structure of the periodic table and different structure–property relationships (such as melting temperatures, electronegativities and…”
Cross-Loss Influence Functions to Explain Deep Network Representations (2020, background): “Deep learning has become a ubiquitous tool in domains from speech-recognition (Bengio and Heigold, 2014) to knowledge-discovery (Tshitoyan et al., 2019), owing to large datasets, increasing computation power, and the development of powerful unsupervised and semi-supervised learning techniques.”
Structured information extraction from scientific text with large language models (2024, methodology): “Datasets were prepared from our database of more than 8 million research paper abstracts 63 .”
Empowering Biomedical Discovery with AI Agents (2024, positive): “[76 ] demonstrated that using latent knowledge from published materials science literature can recommend novel materials.”
A Survey of Deep Learning for Scientific Discovery (2020, background): “It has already found interesting uses in aiding with the automatic analysis of scientiﬁc texts, such as drug name recognition systems [131], biomedical named entity recognition [73], identifying important concepts in materials science [230] and even detecting chemical-protein interactions [37].”
Leveraging large language models for predictive chemistry (2024, positive): “The fact that we get good results independent of the chemical representation illustrates that these language models are very apt at extracting correlations from any text 15 .”
ScholarBERT: Bigger is Not Always Better (2022, background): “While NLP techniques have long played an integral part in some domains (such as biomedicine (Yandell and Ma-joros, 2002) and materials science (Tshitoyan et al., 2019)), resulting in an abundance of datasets, they are just starting to be adopted in other domains (such as social sciences (Duede et…”