Exposing Product Bias in LLM Investment Recommendation

• URL: http://arxiv.org/abs/2503.08750v1
• Date: Tue, 11 Mar 2025 13:10:00 GMT
• Title: Exposing Product Bias in LLM Investment Recommendation
• Authors: Yuhan Zhi, Xiaoyu Zhang, Longtian Wang, Shumin Jiang, Shiqing Ma, Xiaohong Guan, Chao Shen,
• Abstract summary: Large language models (LLMs), as a new generation of recommendation engines, possess powerful summarization and data analysis capabilities.<n>In this paper, we reveal a novel product bias in LLM investment recommendation, where LLMs exhibit systematic preferences for specific products.<n>Such preferences can subtly influence user investment decisions, potentially leading to inflated valuations of products and financial bubbles.
• Score: 35.192944979712394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs), as a new generation of recommendation engines, possess powerful summarization and data analysis capabilities, surpassing traditional recommendation systems in both scope and performance. One promising application is investment recommendation. In this paper, we reveal a novel product bias in LLM investment recommendation, where LLMs exhibit systematic preferences for specific products. Such preferences can subtly influence user investment decisions, potentially leading to inflated valuations of products and financial bubbles, posing risks to both individual investors and market stability. To comprehensively study the product bias, we develop an automated pipeline to create a dataset of 567,000 samples across five asset classes (stocks, mutual funds, cryptocurrencies, savings, and portfolios). With this dataset, we present the bf first study on product bias in LLM investment recommendations. Our findings reveal that LLMs exhibit clear product preferences, such as certain stocks (e.g., `AAPL' from Apple and `MSFT' from Microsoft). Notably, this bias persists even after applying debiasing techniques. We urge AI researchers to take heed of the product bias in LLM investment recommendations and its implications, ensuring fairness and security in the digital space and market.

Related papers

• Your AI, Not Your View: The Bias of LLMs in Investment Analysis [55.328782443604986]
  Large Language Models (LLMs) face frequent knowledge conflicts due to discrepancies between pre-trained parametric knowledge and real-time market data.<n>This paper offers the first quantitative analysis of confirmation bias in LLM-based investment analysis.<n>We observe a consistent preference for large-cap stocks and contrarian strategies across most models.
  arXiv  Detail & Related papers  (2025-07-28T16:09:38Z)
• DeepRec: Towards a Deep Dive Into the Item Space with Large Language Model Based Recommendation [83.21140655248624]
  Large language models (LLMs) have been introduced into recommender systems (RSs)<n>We propose DeepRec, a novel LLM-based RS that enables autonomous multi-turn interactions between LLMs and TRMs for deep exploration of the item space.<n> Experiments on public datasets demonstrate that DeepRec significantly outperforms both traditional and LLM-based baselines.
  arXiv  Detail & Related papers  (2025-05-22T15:49:38Z)
• Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking [12.837781884216227]
  Large Language Models (LLMs) have demonstrated notable capabilities across financial tasks.<n>Their real-world effectiveness in managing complex fund investment remains inadequately assessed.<n>We introduce DeepFund, a live fund benchmark tool designed to rigorously evaluate LLM in real-time market conditions.
  arXiv  Detail & Related papers  (2025-05-16T10:00:56Z)
• DeepFund: Will LLM be Professional at Fund Investment? A Live Arena Perspective [10.932591941137698]
  This paper introduces DeepFund, a comprehensive platform for evaluating Large Language Models (LLMs) in a simulated live environment. Our approach implements a multi agent framework where LLMs serve as both analysts and managers, creating a realistic simulation of investment decision making. We provide a web interface that visualizes model performance across different market conditions and investment parameters, enabling detailed comparative analysis.
  arXiv  Detail & Related papers  (2025-03-24T03:32:13Z)
• A Framework for Using LLMs for Repository Mining Studies in Empirical Software Engineering [12.504438766461027]
  Large Language Models (LLMs) have transformed Software Engineering (SE) by providing innovative methods for analyzing software repositories.<n>Our research packages a framework, coined Prompt Refinement and Insights for Mining Empirical Software repositories (PRIMES)<n>Our findings indicate that standardizing prompt engineering and using PRIMES can enhance the reliability and accuracy of studies utilizing LLMs.
  arXiv  Detail & Related papers  (2024-11-15T06:08:57Z)
• AI in Investment Analysis: LLMs for Equity Stock Ratings [0.2916558661202724]
  This paper explores the application of Large Language Models (LLMs) to generate multi-horizon stock ratings. Our study addresses these issues by leveraging LLMs to improve the accuracy and consistency of stock ratings. Our results show that our benchmark method outperforms traditional stock rating methods when assessed by forward returns.
  arXiv  Detail & Related papers  (2024-10-30T15:06:57Z)
• AAPM: Large Language Model Agent-based Asset Pricing Models [4.326886488307076]
  We propose a novel asset pricing approach, which fuses qualitative discretionary investment analysis from LLM agents and quantitative manual financial economic factors. The experimental results show that our approach outperforms machine learning-based asset pricing baselines in portfolio optimization and asset pricing errors.
  arXiv  Detail & Related papers  (2024-09-25T18:27:35Z)
• When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments [55.19252983108372]
  We have developed a multi-agent AI system called StockAgent, driven by LLMs. The StockAgent allows users to evaluate the impact of different external factors on investor trading. It avoids the test set leakage issue present in existing trading simulation systems based on AI Agents.
  arXiv  Detail & Related papers  (2024-07-15T06:49:30Z)
• How Ethical Should AI Be? How AI Alignment Shapes the Risk Preferences of LLMs [0.0]
  This study examines the risk preferences of Large Language Models (LLMs) and how aligning them with human ethical standards affects their economic decision-making. We find that aligning LLMs with human values, focusing on harmlessness, helpfulness, and honesty, shifts them towards risk aversion.
  arXiv  Detail & Related papers  (2024-06-03T10:05:25Z)
• The Economic Implications of Large Language Model Selection on Earnings and Return on Investment: A Decision Theoretic Model [0.0]
  We use a decision-theoretic approach to compare the financial impact of different language models. The study reveals how the superior accuracy of more expensive models can, under certain conditions, justify a greater investment. This article provides a framework for companies looking to optimize their technology choices.
  arXiv  Detail & Related papers  (2024-05-27T20:08:41Z)
• Scaling Down to Scale Up: A Cost-Benefit Analysis of Replacing OpenAI's LLM with Open Source SLMs in Production [3.41402911469979]
  Many companies use large language models (LLMs) offered as a service, like OpenAI's GPT-4, to create AI-enabled product experiences. At the same time, a flurry of open-source small language models (SLMs) has been made available for commercial use. This paper presents a systematic evaluation methodology and a characterization of modern open-source SLMs.
  arXiv  Detail & Related papers  (2023-12-20T19:27:59Z)
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity [61.54815512469125]
  This survey addresses the crucial issue of factuality in Large Language Models (LLMs) As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital.
  arXiv  Detail & Related papers  (2023-10-11T14:18:03Z)
• LLMRec: Benchmarking Large Language Models on Recommendation Task [54.48899723591296]
  The application of Large Language Models (LLMs) in the recommendation domain has not been thoroughly investigated. We benchmark several popular off-the-shelf LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation.
  arXiv  Detail & Related papers  (2023-08-23T16:32:54Z)
• A Survey on Large Language Models for Recommendation [77.91673633328148]
  Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) This survey presents a taxonomy that categorizes these models into two major paradigms, respectively Discriminative LLM for Recommendation (DLLM4Rec) and Generative LLM for Recommendation (GLLM4Rec)
  arXiv  Detail & Related papers  (2023-05-31T13:51:26Z)
• On Learning to Summarize with Large Language Models as References [101.79795027550959]
  Large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets. We study an LLM-as-reference learning setting for smaller text summarization models to investigate whether their performance can be substantially improved.
  arXiv  Detail & Related papers  (2023-05-23T16:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.