Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings

By L. Hallee, Rohan Kapur, Arjun Patel, Jason P. Gleghorn, Bohdan B. Khomtchouk. 2024

Subfields: Representation & Self-Supervised Learning, Model Architectures

Citations in the corpus

• Training Sparse Mixture Of Experts Text Embedding Models (2025, unknown): “While Hallee et al. (2024) explored domain-specific MoE embeddings and Li & Zhou (2024) investigated using MoE language model states as embeddings, our work is the first to develop a general-purpose MoE architecture specifically for text embeddings.”