Augmented Bilinear Network for Incremental Multi-Stock Time-Series
Classification
- URL: http://arxiv.org/abs/2207.11577v1
- Date: Sat, 23 Jul 2022 18:54:10 GMT
- Title: Augmented Bilinear Network for Incremental Multi-Stock Time-Series
Classification
- Authors: Mostafa Shabani, Dat Thanh Tran, Juho Kanniainen, Alexandros Iosifidis
- Abstract summary: We propose a method to efficiently retain the knowledge available in a neural network pre-trained on a set of securities.
In our method, the prior knowledge encoded in a pre-trained neural network is maintained by keeping existing connections fixed.
This knowledge is adjusted for the new securities by a set of augmented connections, which are optimized using the new data.
- Score: 83.23129279407271
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep Learning models have become dominant in tackling financial time-series
analysis problems, overturning conventional machine learning and statistical
methods. Most often, a model trained for one market or security cannot be
directly applied to another market or security due to differences inherent in
the market conditions. In addition, as the market evolves through time, it is
necessary to update the existing models or train new ones when new data is made
available. This scenario, which is inherent in most financial forecasting
applications, naturally raises the following research question: How to
efficiently adapt a pre-trained model to a new set of data while retaining
performance on the old data, especially when the old data is not accessible? In
this paper, we propose a method to efficiently retain the knowledge available
in a neural network pre-trained on a set of securities and adapt it to achieve
high performance in new ones. In our method, the prior knowledge encoded in a
pre-trained neural network is maintained by keeping existing connections fixed,
and this knowledge is adjusted for the new securities by a set of augmented
connections, which are optimized using the new data. The auxiliary connections
are constrained to be of low rank. This not only allows us to rapidly optimize
for the new task but also reduces the storage and run-time complexity during
the deployment phase. The efficiency of our approach is empirically validated
in the stock mid-price movement prediction problem using a large-scale limit
order book dataset. Experimental results show that our approach enhances
prediction performance as well as reduces the overall number of network
parameters.
Related papers
- Dreaming Learning [41.94295877935867]
Introducing new information to a machine learning system can interfere with previously stored data.
We propose a training algorithm inspired by Stuart Kauffman's notion of the Adjacent Possible.
It predisposes the neural network to smoothly accept and integrate data sequences with different statistical characteristics than expected.
arXiv Detail & Related papers (2024-10-23T09:17:31Z) - Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learning [22.13331870720021]
We propose a beyond prompt learning approach to the RFCL task, called Continual Adapter (C-ADA)
C-ADA flexibly extends specific weights in CAL to learn new knowledge for each task and freezes old weights to preserve prior knowledge.
Our approach achieves significantly improved performance and training speed, outperforming the current state-of-the-art (SOTA) method.
arXiv Detail & Related papers (2024-07-14T17:40:40Z) - Adaptive Memory Replay for Continual Learning [29.333341368722653]
Updating Foundation Models as new data becomes available can lead to catastrophic forgetting'
We introduce a framework of adaptive memory replay for continual learning, where sampling of past data is phrased as a multi-armed bandit problem.
We demonstrate the effectiveness of our approach, which maintains high performance while reducing forgetting by up to 10% at no training efficiency cost.
arXiv Detail & Related papers (2024-04-18T22:01:56Z) - A Survey on Knowledge Editing of Neural Networks [43.813073385305806]
Even the largest artificial neural networks make mistakes, and once-correct predictions can become invalid as the world progresses in time.
Knowledge editing is emerging as a novel area of research that aims to enable reliable, data-efficient, and fast changes to a pre-trained target model.
We first introduce the problem of editing neural networks, formalize it in a common framework and differentiate it from more notorious branches of research such as continuous learning.
arXiv Detail & Related papers (2023-10-30T16:29:47Z) - Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information.
We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting.
Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
arXiv Detail & Related papers (2023-09-15T17:10:51Z) - DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecasting [23.64541466875515]
We propose an end-to-end framework with two adapters, which can effectively adapt the data and the model to mitigate the effects of distribution shifts.
Our key insight is to automatically learn how to adapt stock data into a locally stationary distribution in favor of profitable updates.
Experiments on real-world stock datasets demonstrate that DoubleAdapt achieves state-of-the-art predictive performance and shows considerable efficiency.
arXiv Detail & Related papers (2023-06-16T14:18:32Z) - Contextual Squeeze-and-Excitation for Efficient Few-Shot Image
Classification [57.36281142038042]
We present a new adaptive block called Contextual Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new task to significantly improve performance.
We also present a new training protocol based on Coordinate-Descent called UpperCaSE that exploits meta-trained CaSE blocks and fine-tuning routines for efficient adaptation.
arXiv Detail & Related papers (2022-06-20T15:25:08Z) - Hyperparameter-free Continuous Learning for Domain Classification in
Natural Language Understanding [60.226644697970116]
Domain classification is the fundamental task in natural language understanding (NLU)
Most existing continual learning approaches suffer from low accuracy and performance fluctuation.
We propose a hyper parameter-free continual learning model for text data that can stably produce high performance under various environments.
arXiv Detail & Related papers (2022-01-05T02:46:16Z) - Bilinear Input Normalization for Neural Networks in Financial
Forecasting [101.89872650510074]
We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series.
The proposed normalization scheme takes into account the bimodal characteristic of financial time-series.
Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
arXiv Detail & Related papers (2021-09-01T07:52:03Z) - Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators.
They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions.
We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
arXiv Detail & Related papers (2021-02-22T07:02:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.