A Comprehensive Overview of Large Language Models

By Humza Naveed, Asad Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, Ajmal Saeed Mian. ACM Transactions on Intelligent Systems and Technology, 2023

1,803 citations recorded by the source corpus.

Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multimodal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to provide not only a systematic survey but also a quick, comprehensive reference for the researchers and practitioners to draw insights from extensive, informative summaries of the existing works to advance the LLM research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ca31b8584b6c022ef15ddfe994fe361e002b7729

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model (2024, background): “Over the last few years, the ability of LLMs has increased significantly (Naveed et al., 2023).”
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (2024, neutral): “While Large Language Models (LLMs) [3, 6, 9, 36, 61, 67, 68, 77, 81, 82, 88, 105] have advanced linguistic intelligence, visual-spatial intelligence remains under-explored, despite its relevance to robotics [7, 8, 21, 64], autonomous driving [79], and AR/VR [12, 27, 55].”
Small Language Models are the Future of Agentic AI (2025, neutral): “…scale then enables them to demonstrate better performance across a wide array of specialized natural language tasks, including text generation, translation, and reasoning, outperforming small models trained both (a) in the same general fashion and (b) from scratch specifically for these tasks [54].”
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges (2024, methodology): “The deep neural network (DNN) framework is utilized in LLMs to enhance its performance which is similar to human-like understanding [3], [82].”
The application of large language models in medicine: A scoping review (2024, neutral): “101 Unlike MLS, LLMs, through their deep neural network architecture, can automatically identify and leverage complex patterns and relationships within language data, without the need for pre-selecting variables or explicit feature engineering.”
RUMAD: Reinforcement-Unifying Multi-Agent Debate (2026, unknown): “Multi-agent debate (MAD) frameworks harness the collective intelligence of multiple large language models (LLMs) [1, 5, 6, 9, 13, 21, 25] to achieve superior reasoning and decision-making beyond what any single model can attain.”
GraphAgents: Knowledge Graph-Guided Agentic AI for Cross-Domain Materials Design (2026, unknown): “The growing performance of large language models (LLMs) [6–11] presents a compelling strategy to reason over the massive corpus of scientific knowledge [12, 13] in ways that extend beyond the limitations of human-driven analysis.”
Knowledge Enhanced Efficient Robotic Decision Making with Intuitive-Deliberate LLMs (2025, unknown): “While LLMs are good at processing text data and general knowledge [17], navigation demands understanding of 3D spaces, physical constraints, and spatial relationships between objects.”
Understanding generative AI output with embedding models (2025, unknown): “From the embedding vectors, a final task-specific “head” makes predictions using these embeddings ( 19 – 21 ).”
KnowThyself: An Agentic Assistant for LLM Interpretability (2025, unknown): “Large language models (LLMs) have attracted significant attention for their impressive capabilities in language understanding, reasoning, and problem solving [1].”
Understanding Generative AI Content with Embedding Models (2024, methodology): “After this mapping, the data is passed through a final statistical learning layer or task-specific “head,” commonly a linear classifier or token predictor, which makes predictions based on the outputs the FE [10–12].”