ChemTEB: Chemical Text Embedding Benchmark, an Overview of Embedding Models Performance & Efficiency on a Specific Domain

By Ali Shiraee Kasmaee, Mohammad Khodadad, Mohammad Arshi Saloot, Stephen Dokas, H. Mahyar, Soheila Samiee. 2024

Subfields: Representation & Self-Supervised Learning, Evaluation & Benchmarks, Computational Chemistry & Drug Discovery

Citations in the corpus

• Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation (2025, unknown): “Embedding models determine how chemical terminology, entities, and semantic relations are encoded and compared during retrieval [11, 12].”
• ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text Embeddings (2025, unknown): “ChemTEB [9] is a chemistry-focused adaptation of MTEB containing two retrieval tasks mainly sourced from encyclopedic data, aiming to evaluate the generalizability of embedding models to chemical domains.”
• Towards Domain Specification of Embedding Models in Medicine (2025, unknown): “The Chemical Text Embedding Benchmark (ChemTEB) [27] demonstrates how specialized suites can capture domain-specific challenges through tasks like chemical document classification, retrieval, and bitext mining, inspiring our approach to medical text embedding evaluation.”
• LLMs Are Also Effective Embedding Models: An In-depth Overview (2024, unknown): “Additionally, through distillation from the output signals of a cross-encoder, a relatively small-scale model like Gecko (1.5B) can achieve performance comparable to that of larger models (7B).”