Research on Optimizing Real-Time Data Processing in High-Frequency Trading Algorithms using Machine Learning

• URL: http://arxiv.org/abs/2412.01062v1
• Date: Mon, 02 Dec 2024 02:46:10 GMT
• Title: Research on Optimizing Real-Time Data Processing in High-Frequency Trading Algorithms using Machine Learning
• Authors: Yuxin Fan, Zhuohuan Hu, Lei Fu, Yu Cheng, Liyang Wang, Yuxiang Wang,
• Abstract summary: The objective of this work is to optimise the real-time processing of data in high-frequency trading algorithms. The dynamic feature selection mechanism is responsible for monitoring and analysing market data in real time. The lightweight neural networks are designed in a modular fashion, comprising fast convolutional layers and pruning techniques.
• Score: 19.43972851292453
• License:
• Abstract: High-frequency trading (HFT) represents a pivotal and intensely competitive domain within the financial markets. The velocity and accuracy of data processing exert a direct influence on profitability, underscoring the significance of this field. The objective of this work is to optimise the real-time processing of data in high-frequency trading algorithms. The dynamic feature selection mechanism is responsible for monitoring and analysing market data in real time through clustering and feature weight analysis, with the objective of automatically selecting the most relevant features. This process employs an adaptive feature extraction method, which enables the system to respond and adjust its feature set in a timely manner when the data input changes, thus ensuring the efficient utilisation of data. The lightweight neural networks are designed in a modular fashion, comprising fast convolutional layers and pruning techniques that facilitate the expeditious completion of data processing and output prediction. In contrast to conventional deep learning models, the neural network architecture has been specifically designed to minimise the number of parameters and computational complexity, thereby markedly reducing the inference time. The experimental results demonstrate that the model is capable of maintaining consistent performance in the context of varying market conditions, thereby illustrating its advantages in terms of processing speed and revenue enhancement.

Related papers

• Data-driven tool wear prediction in milling, based on a process-integrated single-sensor approach [1.6574413179773764]
  This study explores data-driven methods, in particular deep learning, for tool wear prediction. The study evaluates several machine learning models, including convolutional neural networks (CNN), long short-term memory networks (LSTM), support vector machines (SVM) and decision trees. The ConvNeXt model has an exceptional performance, achieving a 99.1% accuracy in identifying tool wear using data from only four milling tools operated until they are worn.
  arXiv  Detail & Related papers  (2024-12-27T23:10:32Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• AdapFair: Ensuring Continuous Fairness for Machine Learning Operations [7.909259406397651]
  We present a debiasing framework designed to find an optimal fair transformation of input data. We leverage the normalizing flows to enable efficient, information-preserving data transformation. We introduce an efficient optimization algorithm with closed-formed gradient computations.
  arXiv  Detail & Related papers  (2024-09-23T15:01:47Z)
• Aeroengine performance prediction using a physical-embedded data-driven method [4.450340826544577]
  We propose a strategy that combines domain knowledge from both the aeroengine and neural network realms to enable real-time prediction of engine performance parameters. Leveraging aeroengine domain knowledge, we judiciously design the network structure and regulate the internal information flow. To rigorously evaluate the effectiveness and robustness of our proposed strategy, we conduct comprehensive validation across two distinct datasets.
  arXiv  Detail & Related papers  (2024-06-29T17:56:58Z)
• Automatic Input Feature Relevance via Spectral Neural Networks [0.9236074230806581]
  We propose a novel method to estimate the relative importance of the input components for a Deep Neural Network. This is achieved by leveraging on a spectral re-parametrization of the optimization process. The technique is successfully challenged against both synthetic and real data.
  arXiv  Detail & Related papers  (2024-06-03T10:39:12Z)
• Sparse Attention-driven Quality Prediction for Production Process Optimization in Digital Twins [53.70191138561039]
  We propose to deploy a digital twin of the production line by encoding its operational logic in a data-driven approach. We adopt a quality prediction model for production process based on self-attention-enabled temporal convolutional neural networks. Our operation experiments on a specific tobacco shredding line demonstrate that the proposed digital twin-based production process optimization method fosters seamless integration between virtual and real production lines.
  arXiv  Detail & Related papers  (2024-05-20T09:28:23Z)
• Switchable Decision: Dynamic Neural Generation Networks [98.61113699324429]
  We propose a switchable decision to accelerate inference by dynamically assigning resources for each data instance. Our method benefits from less cost during inference while keeping the same accuracy.
  arXiv  Detail & Related papers  (2024-05-07T17:44:54Z)
• Distributed Neural Representation for Reactive in situ Visualization [23.80657290203846]
  Implicit neural representations (INRs) have emerged as a powerful tool for compressing large-scale volume data. We develop a distributed neural representation and optimize it for in situ visualization. Our technique eliminates data exchanges between processes, achieving state-of-the-art compression speed, quality and ratios.
  arXiv  Detail & Related papers  (2023-03-28T03:55:47Z)
• Bilinear Input Normalization for Neural Networks in Financial Forecasting [101.89872650510074]
  We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series. The proposed normalization scheme takes into account the bimodal characteristic of financial time-series. Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
  arXiv  Detail & Related papers  (2021-09-01T07:52:03Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity [57.275753974812666]
  Federated learning involves learning from data samples distributed across a network of clients while the data remains local. In this paper, we propose a novel straggler-resilient federated learning method that incorporates statistical characteristics of the clients' data to adaptively select the clients in order to speed up the learning procedure.
  arXiv  Detail & Related papers  (2020-12-28T19:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.