Seq vs Seq: An Open Suite of Paired Encoders and Decoders

By Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn J. Lawrie, Benjamin Van Durme. 2025

Subfields: Large Language Models, Model Architectures

Citations in the corpus

• HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions (2026, unknown): “9), and the English-only cross-encoder cross-encoder/ettin-reranker-400m-v1 (Ettin; Weller et al., 2025) reaches macro 70 .”
• ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models (2026, unknown): “Even mxbai-edge-colbert-v0 [6], a recent study that details a recipe to train very strong ColBERT models from scratch (that is, MLM-pretained Ettin [7] models) performed all the early stages in a single-vector setting and only used multi-vector in the latest stage, knowledge distillation.”
• Fantastic (small) Retrievers and How to Train Them: mxbai-edge-colbert-v0 Tech Report (2025, unknown): “We use the Ettin [35] encoder models as starting models, which are a replication of the ModernBERT training recipe across various model sizes [33].”
• ModernVBERT: Towards Smaller Visual Document Retrievers (2025, unknown): “(2025); when possible, training encoder models from scratch remain better for retrieval tasks.”
• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning (2025, unknown): “Rather than simply using a lower masking ratio at the end of training as shown by Boizard et al. (2025); Weller et al. (2025), we progressively lower the mask rate at each stage.”
• Better Language Models Exhibit Higher Visual Alignment (2024, unknown): “The additional parameters, combined with ReLU activations between layers, enhance ShareLock ’s ability to capture complex, non-linear cross-modal correspondences.”
• A Model of the Language Process (2026, unknown): “More recent encoder variants, such as the Etin400M Model (Weller et al., 2025), provide contemporary encoder base models with improved performance and efficient scaling.”