iBet uBet web content aggregator. Adding the entire web to your favor.

Link to original content: https://doi.org/10.1145/3604237.3626866

Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models | Proceedings of the Fourth ACM International Conference on AI in Finance

research-article

Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models

Authors:

Muhammad Ali Babar,

Xiao-Yang LiuAuthors Info & Claims

ICAIF '23: Proceedings of the Fourth ACM International Conference on AI in Finance

Pages 349 - 356

https://doi.org/10.1145/3604237.3626866

Published: 25 November 2023 Publication History

Abstract

Financial sentiment analysis is critical for valuation and investment decision-making. Traditional NLP models, however, are limited by their parameter size and the scope of their training datasets, which hampers their generalization capabilities and effectiveness in this field. Recently, Large Language Models (LLMs) pre-trained on extensive corpora have demonstrated superior performance across various NLP tasks due to their commendable zero-shot abilities. Yet, directly applying LLMs to financial sentiment analysis presents challenges: The discrepancy between the pre-training objective of LLMs and predicting the sentiment label can compromise their predictive performance. Furthermore, the succinct nature of financial news, often devoid of sufficient context, can significantly diminish the reliability of LLMs’ sentiment analysis. To address these challenges, we introduce a retrieval-augmented LLMs framework for financial sentiment analysis. This framework includes an instruction-tuned LLMs module, which ensures LLMs behave as predictors of sentiment labels, and a retrieval-augmentation module which retrieves additional context from reliable external sources. Benchmarked against traditional models and LLMs like ChatGPT and LLaMA, our approach achieves 15% to 48% performance gain in accuracy and F1 score.

References

[1]

Dogu Araci. 2019. FinBERT: Financial sentiment analysis with pre-trained language models. In arXiv preprint arXiv:1908.10063.

[2]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.

[3]

Deng Cai, Yan Wang, Lemao Liu, and Shuming Shi. 2022. Recent advances in retrieval-augmented text generation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3417–3419.

Digital Library

[4]

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality.

[5]

Min-Yuh Day and Chia-Chou Lee. 2016. Deep learning for financial sentiment analysis on finance news providers. In IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 1127–1134.

[6]

Gartner Glossary. 2023. Definition of Sentiment Analysis - Finance Glossary - Gartner.

[7]

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459–9474.

[8]

Shangqing Liu, Yu Chen, Xiaofei Xie, Jingkai Siow, and Yang Liu. 2020. Retrieval-augmented generation for code summarization via hybrid gnn. arXiv preprint arXiv:2006.05405 (2020).

[9]

Ilya Loshchilov and Frank Hutter. 2017. Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101 (2017).

[10]

Renze Lou, Kai Zhang, and Wenpeng Yin. 2023. Is prompt all you need? no. A comprehensive and broader view of instruction learning. arXiv preprint arXiv:2303.10475 (2023).

[11]

Neural Magic. 2022. Twitter Financial News Sentiment. http://precog.iiitd.edu.in/people/anupama.

[12]

Macedo Maia, Siegfried Handschuh, Andre Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra. Balahur. 2018. WWW ’18: Companion Proceedings of the The Web Conference 2018. In International World Wide Web Conferences Steering Committee (Lyon, France). Republic and Canton of Geneva, CHE.

[13]

Pekka Malo, Ankur Sinha, Pekka Korhonen, Jyrki Wallenius, and Pyry Takala. 2014. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology 65, 4 (2014), 782–796.

Digital Library

[14]

Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. 2020. Generation-augmented retrieval for open-domain question answering. arXiv preprint arXiv:2009.08553 (2020).

[15]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.

[16]

Md Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Retrieval augmented code generation and summarization. arXiv preprint arXiv:2108.11601 (2021).

[17]

Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters. In Association for Computing Machinery (Virtual Event, CA, USA) (KDD ’20). New York, NY, USA, 3505–3506.

[18]

Vipula Rawte, Aparna Gupta, and Mohammed J Zaki. 2020. A comparative analysis of temporal long text similarity: Application to financial documents. In Workshop on Mining Data for Financial Applications. Springer, 77–91.

[19]

Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, 2021. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 (2021).

[20]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015).

[21]

Sahar Sohangir, Dingding Wang, Anna Pomeranets, and Taghi M Khoshgoftaar. 2018. Big Data: Deep Learning for financial sentiment analysis. Journal of Big Data 5, 1 (2018), 1–25.

[22]

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca.

[23]

Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, 2022. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).

[24]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).

[25]

M.K. Vijaymeena1 and K. Kavitha. 2016. A Survey on Similarity Measures in Text Mining. Machine Learning and Applications (2016).

[26]

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560 (2022).

[27]

Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2022. Finetuned Language Models are Zero-Shot Learners. In International Conference on Learning Representations.

[28]

Paul J Werbos. 1988. Generalization of backpropagation with application to a recurrent gas market model. Neural networks 1, 4 (1988), 339–356.

[29]

Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. BloombergGPT: A large language model for finance. arXiv preprint arXiv:2303.17564 (2023).

[30]

Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. FinGPT: Open-Source Financial Large Language Models. arXiv preprint arXiv:2306.06031 (2023).

[31]

Yi Yang, Mark Christopher Siy Uy, and Allen Huang. 2020. Finbert: A pretrained language model for financial communications. arXiv preprint arXiv:2006.08097 (2020).

[32]

Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414 (2022).

[33]

Boyu Zhang, Hongyang Yang, and Xiao-Yang Liu. 2023. Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models. arXiv preprint arXiv:2306.12659 (2023).

[34]

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A Survey of Large Language Models. arxiv:2303.18223 [cs.CL]

Cited By

Tasan MOzkan YOzgur AOzpinar A(2024)Data Plateau: A Unified Analytics Platform with Intuitive Interfaces for Real-Time and ML-Driven InsightsOrclever Proceedings of Research and Development10.56038/oprd.v4i1.4574:1(73-89)Online publication date: 31-May-2024
https://doi.org/10.56038/oprd.v4i1.457
Mathebula MModupe AMarivate V(2024)Fine-Tuning Retrieval-Augmented Generation with an Auto-Regressive Language Model for Sentiment Analysis in Financial ReviewsApplied Sciences10.3390/app14231078214:23(10782)Online publication date: 21-Nov-2024
https://doi.org/10.3390/app142310782
Iaroshev IPillai RVaglietti LHanne T(2024)Evaluating Retrieval-Augmented Generation Models for Financial Report Question and AnsweringApplied Sciences10.3390/app1420931814:20(9318)Online publication date: 12-Oct-2024
https://doi.org/10.3390/app14209318
Show More Cited By

Index Terms

Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

LLMs to the Moon? Reddit Market Sentiment Analysis with Large Language Models
WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023

Market sentiment analysis on social media content requires knowledge of both financial markets and social media jargon, which makes it a challenging task for human raters. The resulting lack of high-quality labeled data stands in the way of conventional ...
Semi-supervised probabilistic sentiment analysis: merging labeled sentences with unlabeled reviews to identify sentiment
ASIST '13: Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information Boundaries

Document level sentiment analysis, the task of determining whether the sentiment expressed in a document is positive or negative, is commonly performed by supervised methods. As with all supervised tasks, obtaining training data for these methods can be ...
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICAIF '23: Proceedings of the Fourth ACM International Conference on AI in Finance

November 2023

697 pages

ISBN:9798400702402

DOI:10.1145/3604237

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICAIF '23

ICAIF '23: 4th ACM International Conference on AI in Finance

November 27 - 29, 2023

NY, Brooklyn, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,603
Total Downloads

Downloads (Last 12 months)1,576
Downloads (Last 6 weeks)163

Reflects downloads up to 09 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tasan MOzkan YOzgur AOzpinar A(2024)Data Plateau: A Unified Analytics Platform with Intuitive Interfaces for Real-Time and ML-Driven InsightsOrclever Proceedings of Research and Development10.56038/oprd.v4i1.4574:1(73-89)Online publication date: 31-May-2024
https://doi.org/10.56038/oprd.v4i1.457
Mathebula MModupe AMarivate V(2024)Fine-Tuning Retrieval-Augmented Generation with an Auto-Regressive Language Model for Sentiment Analysis in Financial ReviewsApplied Sciences10.3390/app14231078214:23(10782)Online publication date: 21-Nov-2024
https://doi.org/10.3390/app142310782
Iaroshev IPillai RVaglietti LHanne T(2024)Evaluating Retrieval-Augmented Generation Models for Financial Report Question and AnsweringApplied Sciences10.3390/app1420931814:20(9318)Online publication date: 12-Oct-2024
https://doi.org/10.3390/app14209318
Sharma HUd Din FOgunleye B(2024)Electric Vehicle Sentiment Analysis Using Large Language ModelsAnalytics10.3390/analytics30400233:4(425-438)Online publication date: 1-Nov-2024
https://doi.org/10.3390/analytics3040023
Papasotiriou KSood SReynolds SBalch T(2024)AI in Investment Analysis: LLMs for Equity Stock RatingsProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698694(419-427)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698694
Fatemi SHu Y(2024)FinVision: A Multi-Agent Framework for Stock Market PredictionProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698688(582-590)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698688
Gu JYe JWang GYin W(2024)Adaptive and Explainable Margin Trading via Large Language Models on Portfolio ManagementProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698681(248-256)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698681
Tian FByadgi AKim DZha DWhite MXiao KLiu X(2024)Customized FinGPT Search Agents Using Foundation ModelsProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698637(469-477)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698637
Lin SWang KLiu X(2024)Analyzing Cascading Outbreak of GameStop Event: A Practical Approach Using Network Analysis and Large Language ModelsProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698636(428-436)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698636
Han SKang HJin BLiu XYang S(2024)XBRL Agent: Leveraging Large Language Models for Financial Report AnalysisProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698614(856-864)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698614
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents