Translation between Molecules and Natural Language

By Carl N. Edwards, Tuan Manh Lai, Kevin Ros, Garrett Honke, Heng Ji. Conference on Empirical Methods in Natural Language Processing, 2022

301 citations recorded by the source corpus.

We present MolT5 - a self-supervised learning framework for pretraining models on a vast amount of unlabeled natural language text and molecule strings. MolT5 allows for new, useful, and challenging analogs of traditional vision-language tasks, such as molecule captioning and text-based de novo molecule generation (altogether: translation between molecules and language), which we explore for the first time. Since MolT5 pretrains models on single-modal data, it helps overcome the chemistry domain shortcoming of data scarcity. Furthermore, we consider several metrics, including a new cross-modal embedding-based metric, to evaluate the tasks of molecule captioning and text-based molecule generation. Our results show that MolT5-based models are able to generate outputs, both molecules and captions, which in many cases are high quality.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/3b9b1aba877ecd3f7e508cbc78a41b623349902b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Heng Ji
2. Jiawei Han
3. Philippe Schwaller
4. Meng Jiang
5. Bowen Jin
6. Chi Han
7. Gang Liu
8. Cynthia Rudin

In Computational Theory and Mathematics

1. Philippe Schwaller
2. Heng Ji
3. Jiawei Han
4. Cynthia Rudin
5. Tong Wang
6. Jianyong Wang
7. George Karypis
8. Fadi A. Thabtah

In Materials Chemistry

1. Philippe Schwaller
2. Heng Ji
3. Meng Jiang

Citations in the corpus

• Leveraging large language models for predictive chemistry (2024, unknown): “One of the fascinating advances in machine learning has been the development of large language models (LLMs), so-called foundation models 1–6 .”
• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, unknown): “A.5 Text Graph Year Method Task Code 2016 Li et al. [165] Text-to-KG Generation - 2019 KG-BERT [166] Text-to-KG Generation https://github.com/yao8839836/kg-bert 2020 Malaviya et al. [167] Text-to-KG Generation https://github.com/allenai/commonsense-kg-completion 2019 Petroni et al. citepetroni2019la”
• Augmenting large language models with chemistry tools (2023, background): “Although recent approaches 103,104 tackle this issue, further research is still needed towards human-level molecular captioning tools.”
• Large Language Models on Graphs: A Comprehensive Survey (2023, background): “Specifically, MolT5 [123] is developed based on the T5 [29], suitable for these two tasks.”