Fantastic (small) Retrievers and How to Train Them: mxbai-edge-colbert-v0 Tech Report

By Rikiya Takehi, Benjamin Clavi'e, Sean Lee, Aamir Shakir. 2025

Subfields: Information Retrieval & Search, Efficiency & Compression

Citations in the corpus

• ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models (2026, unknown): “Even mxbai-edge-colbert-v0 [6], a recent study that details a recipe to train very strong ColBERT models from scratch (that is, MLM-pretained Ettin [7] models) performed all the early stages in a single-vector setting and only used multi-vector in the latest stage, knowledge distillation.”
• DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search (2026, unknown): “Following prior work (Ren et al., 2021; Santhanam et al., 2022; Jha et al 2024; Takehi et al., 2025), we distill a strong cross-encoder into our models by applying a KL divergence loss between student and teacher scores.”