Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond

By Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, Xia Hu. ACM Transactions on Knowledge Discovery from Data, 2023

1,077 citations recorded by the source corpus.

This article presents a comprehensive and practical guide for practitioners and end-users working with Large Language Models (LLMs) in their downstream Natural Language Processing (NLP) tasks. We provide discussions and insights into the usage of LLMs from the perspectives of models, data, and downstream tasks. First, we offer an introduction and brief summary of current language models. Then, we discuss the influence of pre-training data, training data, and test data. Most importantly, we provide a detailed discussion about the use and non-use cases of large language models for various natural language processing tasks, such as knowledge-intensive tasks, traditional natural language understanding tasks, generation tasks, emergent abilities, and considerations for specific tasks. We present various use cases and non-use cases to illustrate the practical applications and limitations of LLMs in real-world scenarios. We also try to understand the importance of data and the specific challenges associated with each NLP task. Furthermore, we explore the impact of spurious biases on LLMs and delve into other essential considerations, such as efficiency, cost, and latency, to ensure a comprehensive understanding of deploying LLMs in practice. This comprehensive guide aims to provide researchers and practitioners with valuable insights and best practices for working with LLMs, thereby enabling the successful implementation of these models in a wide range of NLP tasks. A curated list of practical guide resources of LLMs, regularly updated, can be found at https://github.com/Mooler0410/LLMsPracticalGuide. An LLMs evolutionary tree, editable yet regularly updated, can be found at llmtree.ai.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/131c6f328c11706de2c43cd16e0b7c5d5e610b6a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Instruction Tuning for Large Language Models: A Survey (2023, neutral): “Section 5 reviews multi-modality techniques and datasets for IT, including images, speech, and video.”
Knowledge Graph Prompting for Multi-Document Question Answering (2023, methodology): “2023) followed by a prompting function to extract pertinent knowledge for downstream tasks (Yang et al. 2023).”
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, background): “…and challenges on ChatGPT and other LLMs for fundamental applications in seeking information and knowledge discovery in biomedicine and health, although there are several previous survey papers on general LLMs [10, 11] and use of ChatGPT for different specific health applications [12-14].”
Sentiment Analysis in the Era of Large Language Models: A Reality Check (2023, background): “They can directly perform tasks in zero-shot or few-shot in-context learning manner and achieve strong performance without the need for any in-domain supervised training (Bang et al., 2023; Ye et al., 2023; Zhong et al., 2023; Yang et al., 2023).”
Explainability for Large Language Models: A Survey (2023, background): “In contrast, LLMs that have been pre-trained on extensive datasets can generalize to a variety of downstream tasks through fine-tuning (Yang et al., 2023).”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Larger models tend to have more parameters, potentially indicating increased capabilities but also greater computational demands.”
H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models (2023, neutral): “…adopted by plenty of natural language processing (NLP) applications with prevailing successes (Yang et al., 2019b; Liu et al., 2019b; Talmor et al., 2018; Jaiswal et al., 2021; Yang et al., 2019a; Wang et al., 2018; Ding et al., 2019; Chowdhery et al., 2022; Wei et al., 2022b; Yang et al., 2023).”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “Large language models (LLMs) pre-trained on large-scale corpus have shown great potential in various NLP tasks [13].”
A survey on large language model based autonomous agents (2023, background): “On the other hand, [166] primarily focus on the applications of LLMs in various downstream tasks and the challenges associated with their deployment.”
Large Language Models for Software Engineering: Survey and Open Problems (2023, methodology): “Many authors have highlighted, both scientifically and anecdotally, that hallucination is a pervasive problem for LLMs [1] and also that it poses specific problems for LLM-based SE [2].”
Large Language Models for Software Engineering: A Systematic Literature Review (2023, background): “Simultaneously, researchers have embarked on a series of research activities regarding LLMrelated works, where a number of literature reviews or survey papers have been produced [31, 68, 69, 362, 395].”
The application of large language models in medicine: A scoping review (2024, neutral): “55 The expectation is that LLMs will alleviate the burdenofadministrativetasksandpaperworkinresearchplanning,offeringsigniﬁcanttimesavingsformedicalstaff.”