FinBERT: Financial Sentiment Analysis with Pre-trained Language Models

By Dogu Araci. arXiv.org, 2019

1,024 citations recorded by the source corpus.

Financial sentiment analysis is a challenging task due to the specialized language and lack of labeled data in that domain. General-purpose models are not effective enough because of the specialized language used in a financial context. We hypothesize that pre-trained language models can help with this problem because they require fewer labeled examples and they can be further trained on domain-specific corpora. We introduce FinBERT, a language model based on BERT, to tackle NLP tasks in the financial domain. Our results show improvement in every measured metric on current state-of-the-art results for two financial sentiment analysis datasets. We find that even with a smaller training set and fine-tuning only a part of the model, FinBERT outperforms state-of-the-art machine learning methods.

Subfields: Artificial Intelligence, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/7102bb3fe73bd057ff161d9db5214a267c1ef312

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

BloombergGPT: A Large Language Model for Finance (2023, methodology): “Financial NLP tasks (Shah et al., 2022) include sentiment analysis (Araci, 2019), named entity recognition (Salinas Alvarado et al., 2015), news classification (Sinha and Khandait, 2020), and question answering (Chen et al., 2021b, 2022).”
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models (2023, methodology): “As aligned Large Language Models (LLMs) have been widely used to support decision-making in both professional and social domains (Araci, 2019; Luo et al., 2022; Tinn et al., 2023), they have been equipped with safety features that can prevent them from generating harmful or objection-able responses…”
The Voice of Monetary Policy (2021, methodology): “As an alternative to our approach, we use FinBERT (Araci, 2019), a pre-trained BERT model which was fine-tuned for sentiment analysis of financial texts (rather than general texts).”
FinQA: A Dataset of Numerical Reasoning over Financial Data (2021, background): “The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing (2021, background): “• many domains like Medical and Legal are labeled data starved which limits the application of AI models in these domains.”
OpenAGI: When LLM Meets Domain Experts (2023, background): “• Language-related Models: Sentiment Analysis classifies the sentiment polarity of a given sentence [1]; Text Summarization creates a text summary that represents the most important or relevant information within the original text content [14]; Machine Translation converts a sentence from a source l”
FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining (2020, background): “Model Accuracy F1 LPS [Malo et al. , 2014] 0.71 0.71 HSC [Krishnamoorthy, 2018] 0.71 0.76 ULMFit [Raaci, 2019] 0.83 0.79 FB-SA [Raaci, 2019] 0.86 0.84 FinBERT BASE (ours) 0.91 0.89 FinBERT LARGE (ours) 0.94 0.93 Financial Question Answering (QA)”
Continual Learning of Large Language Models: A Comprehensive Survey (2024, neutral): “1 summarizes the existing studies on CPT, and here are some key observations we make about CPT.”
FinGPT: Open-Source Financial Large Language Models (2023, neutral): “We compare FinGPT against standard financial NLP base-lines: • FinBERT [Araci, 2019]; • BloombergGPT [Wu et al. , 2023]; • ChatGPT (zero-shot) [OpenAI, 2023]; • Llama3.”
Benchmarking Embedding Models for ESG Data (2026, unknown): “Similarly, the financial domain has benefited from domain-specific pre-training, with [27] and [28] demonstrating significant performance improvements on financial text understanding tasks.”