Challenges and Applications of Large Language Models

By Jean Kaddour, J. Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, R. McHardy. 2023

504 citations recorded by the source corpus.

Large Language Models (LLMs) went from non-existent to ubiquitous in the machine learning discourse within a few years. Due to the fast pace of the field, it is difficult to identify the remaining challenges and already fruitful application areas. In this paper, we aim to establish a systematic set of open problems and application successes so that ML researchers can comprehend the field's current state more quickly and become productive.

Subfields: Artificial Intelligence, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

Citations in the corpus

Are We Done with MMLU? (2024, neutral): “The advent of transformer-based Large Language Models (LLMs) [1, 2, 3, 4, 5, 6, 7, 8] marked a significant advancement in generative models, enabling interaction with computing devices through natural language.”
StruQ: Defending Against Prompt Injection with Structured Queries (2024, unknown): “A general-purpose LLM can be used for a specific task with zero-shot prompting [24], where the input to the LLM is formed by concate-nating a prompt (containing the application developer’s task specification) with the user input [25].”
Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review (2023, background): “In real application, the prompt is the input of the model, and its engineering can result in significant output difference [12].”
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (2023, background): “Large language models (LLMs) have demonstrated remarkable capabilities across various tasks (Park et al., 2023; Kaddour et al., 2023; Song et al.; Li et al., 2023a; Wang et al., 2023a; Chen et al., 2023; Zheng et al., 2023; Wang et al., 2023c), However, even the most advanced LLMs face challenges…”
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents (2024, background): “As has been observed in other domains such as reinforcement learning [17, 40, 8], the current output of foundation models is stochastic and highly sensitive to their inputs [20, 34, 30, 43], motivating the need to test them under varying conditions.”
Security and Privacy Challenges of Large Language Models: A Survey (2024, unknown): “Several security and privacy risks and vulnerabilities are prevalent in LLMs, e.g., misinfor-mation [44], trustworthiness [45], hallucinations [46], [47], and resource consumption [48].”
Large Language Models: A Survey (2024, unknown): “It is worth noting that LLM challenges are discussed in details in a work by Kaddour et al. [207].”
Making Large Language Models Better Reasoners with Alignment (2023, unknown): “…Yuan et al., 2023a; Luo et al., 2023; Mukherjee et al., 2023), because LLMs still lack reasoning skills (Wang et al., 2023b;d; Zheng et al., 2023; Kaddour et al., 2023; Zhao et al., 2023a) that are essential for them to serve as the brain of artificial general intelligence agents (Wang et al.,…”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “…2023), previously generated context (Liu et al., 2022), or factual knowledge (Min et al., 2023; Muhlgay et al., 2023; Li et al., 2023a)—this phenomenon is commonly referred to as hallucination, which significantly undermines the reliability of LLMs in real-world scenarios (Kaddour et al., 2023).”
AutoSurvey: Large Language Models Can Automatically Write Surveys (2024, unknown): “While several advanced large models, including GPT-4 and Claude 3, support inputs exceeding 100k tokens, their output is still limited to fewer than 8k tokens (the output length of GPT-4 is 8k, and the output length of Claude 3 is 4k).”