The Claude 3 Model Family: Opus, Sonnet, Haiku

Citations in the corpus

• From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline (2024, methodology): “We construct “ground truth” labels for 200 randomly sampled queries by collecting majority votes from GPT-4o [31], Claude-3-Opus [2], and Gemini-1.”
• LiveBench: A Challenging, Contamination-Free LLM Benchmark (2024, unknown): “5-turbo-0125 [6, 42]; four Anthropic models: claude-3-5-sonnet-20240620 , claude-3-opus-20240229 , claude-3-sonnet-20240229 , claude-3-haiku-20240307 [2]; two Mistral models: mistral-large-2402 , mistral-small -2402 [30]; and two Google models: gemini-1.”
• LVBench: An Extreme Long Video Understanding Benchmark (2024, unknown): “The rapid development of Multi-modal Large Language Models (MLLMs) has led to significant advances in vision-language understanding [26, 37, 38], driven by break-throughs in both large language models [3, 8, 25] and visual encoders [30, 35, 46].”
• A-MEM: Agentic Memory for LLM Agents (2025, unknown): “Besides reporting experiment results with four additional metrics (ROUGE-L, ROUGE-2, METEOR, and SBERT Similarity), we also present experimental outcomes using different foundation models including DeepSeek-R1-32B [11], Claude 3.0 Haiku [2], and Claude 3.5 Haiku [3] in Appendix A.3.”
• VLMEvalKit: An Open-Source ToolKit for Evaluating Large Multi-Modality Models (2024, unknown): “With the rapid development of Large Language Models (LLMs) [3, 36, 43, 44], Large Multi-Modality Models (LMMs) [37,42] have also experienced significant advancements.”
• WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (2024, background): “With the recent state-of-the-art LMs (e.g., GPT-4, Gemini, Claude [1, 33, 3]) and their capabilities, researchers have begun using these models as judges [41] to assist in moderation.”
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024, unknown): “5-Sonnet [8], have demonstrated exceptional performance, their closed nature limits transparency OpenAIo1/MMMU78.”
• DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence (2024, unknown): “• General language models that we compare include Llama3 70B (Meta, 2024), GPT-4 (OpenAI, 2023), Claude 3 Opus (Anthropic, 2024), and Gemini 1.5 Pro (Reid et al., 2024).”
• InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models (2025, unknown): “5 Sonnet [3], Gemini-2.”
• Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (2024, unknown): “The rapid development of MLLMs in recent years [66] has highlighted their impressive perception and cognitive capabilities across various multimodal benchmarks [16, 43, 67].”
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, unknown): “Part of results are collected from the OpenCompass leaderboard [20] and other papers [2, 30, 32, 148, 176].”
• SnapKV: LLM Knows What You are Looking for Before Generation (2024, background): “Many inspiring works have successfully expanded LLMs to handle longer contexts, overcoming the difficulties in context maintenance and attention mechanism scalability, such as GPT-4 [1] and Command-R [2] with context length 128K, Claude-3 [3] with 200K, and Gemini-Pro-1.”
• WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild (2024, methodology): “To identify challenging tasks that can distinguish the performance of different LLMs, we used GPT-4-Turbo [26], Claude-3-Sonnet, and Opus [2] to analyze the required background knowledge and reasoning capabilities for each task.”