Forecasting Cryptocurrency Returns from Sentiment Signals: An Analysis of BERT Classifiers and Weak Supervision

• URL: http://arxiv.org/abs/2204.05781v3
• Date: Sun, 19 Mar 2023 21:32:59 GMT
• Title: Forecasting Cryptocurrency Returns from Sentiment Signals: An Analysis of BERT Classifiers and Weak Supervision
• Authors: Duygu Ider, Stefan Lessmann
• Abstract summary: We introduce weak learning, a recently proposed NLP approach to address the problem that text data is unlabeled. We confirm that finetuning using weak labels enhances the predictive value of text-based features and raises forecast accuracy in the context of predicting cryptocurrency returns. More fundamentally, the modeling paradigm we present, weak labeling domain-specific text and finetuning pretrained NLP models, is universally applicable in (financial) forecasting.
• Score: 6.624726878647541
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Anticipating price developments in financial markets is a topic of continued interest in forecasting. Funneled by advancements in deep learning and natural language processing (NLP) together with the availability of vast amounts of textual data in form of news articles, social media postings, etc., an increasing number of studies incorporate text-based predictors in forecasting models. We contribute to this literature by introducing weak learning, a recently proposed NLP approach to address the problem that text data is unlabeled. Without a dependent variable, it is not possible to finetune pretrained NLP models on a custom corpus. We confirm that finetuning using weak labels enhances the predictive value of text-based features and raises forecast accuracy in the context of predicting cryptocurrency returns. More fundamentally, the modeling paradigm we present, weak labeling domain-specific text and finetuning pretrained NLP models, is universally applicable in (financial) forecasting and unlocks new ways to leverage text data.

Related papers

• F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
  We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm. By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases. Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
  arXiv  Detail & Related papers  (2024-06-23T21:28:50Z)
• Natural Language Processing and Multimodal Stock Price Prediction [0.8702432681310401]
  This paper utilizes stock percentage change as training data, in contrast to the traditional use of raw currency values. The choice of percentage change aims to provide models with context regarding the significance of price fluctuations. The study employs specialized BERT natural language processing models to predict stock price trends.
  arXiv  Detail & Related papers  (2024-01-03T01:21:30Z)
• Corporate Bankruptcy Prediction with Domain-Adapted BERT [7.931904787652709]
  This study performs BERT-based analysis, which is a representative contextualized language model, on corporate disclosure data to predict impending bankruptcies. We achieve the accuracy rate of 91.56% and demonstrate that the domain adaptation procedure brings a significant improvement in prediction accuracy.
  arXiv  Detail & Related papers  (2023-12-06T00:05:25Z)
• Forecasting Cryptocurrency Prices Using Deep Learning: Integrating Financial, Blockchain, and Text Data [3.8443430569753025]
  We analyse the influence of public sentiment on cryptocurrency valuations using advanced deep learning NLP methods. We compare the performance of various ML models, both with and without NLP data integration. We discover that pre-trained models, such as Twitter-RoBERTa and BART MNLI, are highly effective in capturing market sentiment.
  arXiv  Detail & Related papers  (2023-11-23T16:14:44Z)
• Measuring Consistency in Text-based Financial Forecasting Models [10.339586273664725]
  FinTrust is an evaluation tool that assesses logical consistency in financial text. We show that the consistency of state-of-the-art NLP models for financial forecasting is poor. Our analysis of the performance degradation caused by meaning-preserving alternations suggests that current text-based methods are not suitable for robustly predicting market information.
  arXiv  Detail & Related papers  (2023-05-15T10:32:26Z)
• In and Out-of-Domain Text Adversarial Robustness via Label Smoothing [64.66809713499576]
  We study the adversarial robustness provided by various label smoothing strategies in foundational models for diverse NLP tasks. Our experiments show that label smoothing significantly improves adversarial robustness in pre-trained models like BERT, against various popular attacks. We also analyze the relationship between prediction confidence and robustness, showing that label smoothing reduces over-confident errors on adversarial examples.
  arXiv  Detail & Related papers  (2022-12-20T14:06:50Z)
• NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-task Financial Forecasting [17.691653056521904]
  This paper describes a numeric-oriented hierarchical transformer model to predict stock returns and financial risk using multi-modal aligned earnings calls data. We present the results of a comprehensive evaluation of Num HTML against several state-of-the-art baselines using a real-world publicly available dataset.
  arXiv  Detail & Related papers  (2022-01-05T10:17:02Z)
• Automatically Identifying Semantic Bias in Crowdsourced Natural Language Inference Datasets [78.6856732729301]
  We introduce a model-driven, unsupervised technique to find "bias clusters" in a learned embedding space of hypotheses in NLI datasets. interventions and additional rounds of labeling can be performed to ameliorate the semantic bias of the hypothesis distribution of a dataset.
  arXiv  Detail & Related papers  (2021-12-16T22:49:01Z)
• Artificial Text Detection via Examining the Topology of Attention Maps [58.46367297712477]
  We propose three novel types of interpretable topological features for this task based on Topological Data Analysis (TDA) We empirically show that the features derived from the BERT model outperform count- and neural-based baselines up to 10% on three common datasets. The probing analysis of the features reveals their sensitivity to the surface and syntactic properties.
  arXiv  Detail & Related papers  (2021-09-10T12:13:45Z)
• NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models [54.184609286094044]
  We propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional OOD detection evaluation metrics by 55% on average compared to the original fine-tuned models.
  arXiv  Detail & Related papers  (2021-08-29T06:58:28Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.