Explainable assessment of financial experts' credibility by classifying social media forecasts and checking the predictions with actual market data

• URL: http://arxiv.org/abs/2406.11924v1
• Date: Mon, 17 Jun 2024 08:08:03 GMT
• Title: Explainable assessment of financial experts' credibility by classifying social media forecasts and checking the predictions with actual market data
• Authors: Silvia García-Méndez, Francisco de Arriba-Pérez, Jaime González-Gonzáleza, Francisco J. González-Castaño,
• Abstract summary: We propose a credibility assessment solution for financial creators in social media that combines Natural Language Processing and Machine Learning. The reputation of the contributors is assessed by automatically classifying their forecasts on asset values by type and verifying these predictions with actual market data. The system provides natural language explanations of its decisions based on a model-agnostic analysis of relevant features.
• Score: 6.817247544942709
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Social media include diverse interaction metrics related to user popularity, the most evident example being the number of user followers. The latter has raised concerns about the credibility of the posts by the most popular creators. However, most existing approaches to assess credibility in social media strictly consider this problem a binary classification, often based on a priori information, without checking if actual real-world facts back the users' comments. In addition, they do not provide automatic explanations of their predictions to foster their trustworthiness. In this work, we propose a credibility assessment solution for financial creators in social media that combines Natural Language Processing and Machine Learning. The reputation of the contributors is assessed by automatically classifying their forecasts on asset values by type and verifying these predictions with actual market data to approximate their probability of success. The outcome of this verification is a continuous credibility score instead of a binary result, an entirely novel contribution by this work. Moreover, social media metrics (i.e., user context) are exploited by calculating their correlation with the credibility rankings, providing insights on the interest of the end-users in financial posts and their forecasts (i.e., drop or rise). Finally, the system provides natural language explanations of its decisions based on a model-agnostic analysis of relevant features.

Related papers

• MisinfoEval: Generative AI in the Era of "Alternative Facts" [50.069577397751175]
  We introduce a framework for generating and evaluating large language model (LLM) based misinformation interventions. We present (1) an experiment with a simulated social media environment to measure effectiveness of misinformation interventions, and (2) a second experiment with personalized explanations tailored to the demographics and beliefs of users. Our findings confirm that LLM-based interventions are highly effective at correcting user behavior.
  arXiv  Detail & Related papers  (2024-10-13T18:16:50Z)
• Performative Prediction on Games and Mechanism Design [69.7933059664256]
  We study a collective risk dilemma where agents decide whether to trust predictions based on past accuracy. As predictions shape collective outcomes, social welfare arises naturally as a metric of concern. We show how to achieve better trade-offs and use them for mechanism design.
  arXiv  Detail & Related papers  (2024-08-09T16:03:44Z)
• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Exposing and Explaining Fake News On-the-Fly [4.278181795494584]
  This work contributes with an explainable and online classification method to recognize fake news in real-time. The proposed method combines both unsupervised and supervised Machine Learning approaches with online created lexica. The performance of the proposed solution has been validated with real data sets from Twitter and the results attain 80 % accuracy and macro F-measure.
  arXiv  Detail & Related papers  (2024-05-03T14:49:04Z)
• Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting [74.68371461260946]
  SocialSense is a framework that induces a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics. Our method surpasses existing state-of-the-art in experimental evaluations for both zero-shot and supervised settings.
  arXiv  Detail & Related papers  (2023-10-20T06:17:02Z)
• Measuring the Effect of Influential Messages on Varying Personas [67.1149173905004]
  We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona might have upon seeing a news message. The proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response. This enables more accurate and comprehensive inference on the mental state of the persona.
  arXiv  Detail & Related papers  (2023-05-25T21:01:00Z)
• DeepTrust: A Reliable Financial Knowledge Retrieval Framework For Explaining Extreme Pricing Anomalies [0.0]
  We introduce DeepTrust, a reliable financial knowledge retrieval framework on Twitter to explain extreme price moves at speed. Our proposed framework consists of three modules, specialized for anomaly detection, information retrieval and reliability assessment. The framework is evaluated on two self-annotated financial anomalies, i.e., Twitter and Facebook stock price on 29 and 30 April 2021.
  arXiv  Detail & Related papers  (2022-03-11T06:29:22Z)
• Personalized multi-faceted trust modeling to determine trust links in social media and its potential for misinformation management [61.88858330222619]
  We present an approach for predicting trust links between peers in social media. We propose a data-driven multi-faceted trust modeling which incorporates many distinct features for a comprehensive analysis. Illustrated in a trust-aware item recommendation task, we evaluate the proposed framework in the context of a large Yelp dataset.
  arXiv  Detail & Related papers  (2021-11-11T19:40:51Z)
• SOK: Seeing and Believing: Evaluating the Trustworthiness of Twitter Users [4.609388510200741]
  Currently, there is no automated way of determining which news or users are credible and which are not. In this work, we created a model which analysed the behaviour of50,000 politicians on Twitter. We classified the political Twitter users as either trusted or untrusted using random forest, multilayer perceptron, and support vector machine.
  arXiv  Detail & Related papers  (2021-07-16T17:39:32Z)
• Information Credibility in the Social Web: Contexts, Approaches, and Open Issues [2.2133187119466116]
  Credibility, also referred as believability, is a quality perceived by individuals, who are not always able to discern, with their own cognitive capacities, genuine information from fake one. Several approaches have been proposed to automatically assess credibility in social media.
  arXiv  Detail & Related papers  (2020-01-26T15:42:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.