Tensor-Efficient High-Dimensional Q-learning

• URL: http://arxiv.org/abs/2511.03595v1
• Date: Wed, 05 Nov 2025 16:16:31 GMT
• Title: Tensor-Efficient High-Dimensional Q-learning
• Abstract summary: High-dimensional reinforcement learning faces challenges with complex calculations and low sample efficiency in large state-action spaces.<n>We propose TEQL, which enhances low-rank tensor decomposition via improved block coordinate descent on discretized state-action spaces.<n>Key innovation is an exploration strategy that combines approximation error with visit count-based upper confidence bound to prioritize actions with high uncertainty.
• Score: 4.503747347543387
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-dimensional reinforcement learning faces challenges with complex calculations and low sample efficiency in large state-action spaces. Q-learning algorithms struggle particularly with the curse of dimensionality, where the number of state-action pairs grows exponentially with problem size. While neural network-based approaches like Deep Q-Networks have shown success, recent tensor-based methods using low-rank decomposition offer more parameter-efficient alternatives. Building upon existing tensor-based methods, we propose Tensor-Efficient Q-Learning (TEQL), which enhances low-rank tensor decomposition via improved block coordinate descent on discretized state-action spaces, incorporating novel exploration and regularization mechanisms. The key innovation is an exploration strategy that combines approximation error with visit count-based upper confidence bound to prioritize actions with high uncertainty, avoiding wasteful random exploration. Additionally, we incorporate a frequency-based penalty term in the objective function to encourage exploration of less-visited state-action pairs and reduce overfitting to frequently visited regions. Empirical results on classic control tasks demonstrate that TEQL outperforms conventional matrix-based methods and deep RL approaches in both sample efficiency and total rewards, making it suitable for resource-constrained applications, such as space and healthcare where sampling costs are high.

Related papers

• Measurement-Driven Adaptive Low-Overhead Implementation of Multi-Controlled Toffoli Gates [1.3267048706241158]
  The Toffoli gate is a fundamental building block for quantum arithmetic and reversible logic.<n>Recent advances in dynamic quantum circuit capabilities, including mid-circuit measurement and classical feedforward, provide new opportunities for reducing the resource overhead of non-Clifford operations.<n>We propose a set of dynamic decomposition strategies for multi-controlled Toffoli gates that exploit adaptive circuit execution and ancilla-assisted constructions.
  arXiv  Detail & Related papers  (2026-05-18T10:03:17Z)
• NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces [13.246738608273711]
  NeuroPareto is a cohesive architecture that integrates rank-centric filtering, uncertainty disentanglement, and history-conditioned acquisition strategies.<n>A lightweight acquisition network, trained online from historical hypervolume improvements, guides expensive evaluations toward regions balancing convergence and diversity.
  arXiv  Detail & Related papers  (2026-02-03T10:02:06Z)
• Breaking Through Barren Plateaus: Reinforcement Learning Initializations for Deep Variational Quantum Circuits [21.491246867521053]
  Variational Quantum Algorithms (VQAs) have gained prominence as a viable framework for exploiting near-term quantum devices.<n>The effectiveness of VQAs is often constrained by the so-called barren plateau problem, wherein gradients diminish exponentially as system size or circuit depth increases.
  arXiv  Detail & Related papers  (2025-08-25T21:37:36Z)
• Resource-Efficient Hadamard Test Circuits for Nonlinear Dynamics on a Trapped-Ion Quantum Computer [1.2063443893298391]
  We propose a low-depth implementation of a class of Hadamard test circuits.<n>We develop a parameterized quantum ansatz specifically tailored for variational algorithms.<n>Our findings demonstrate a significant reduction in single- and two-qubit gate counts.
  arXiv  Detail & Related papers  (2025-07-25T13:16:54Z)
• Q-function Decomposition with Intervention Semantics with Factored Action Spaces [51.01244229483353]
  We consider Q-functions defined over a lower dimensional projected subspace of the original action space, and study the condition for the unbiasedness of decomposed Q-functions.<n>This leads to a general scheme which we call action decomposed reinforcement learning that uses the projected Q-functions to approximate the Q-function in standard model-free reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2025-04-30T05:26:51Z)
• GranQ: Efficient Channel-wise Quantization via Vectorized Pre-Scaling for Zero-Shot QAT [2.510925330348642]
  GranQ is a novel activation quantization framework that introduces an efficient pre-scaling strategy.<n>It consistently outperforms state-of-the-art ZSQ methods across CIFAR and ImageNet.<n>Our method achieves up to 5.45% higher accuracy in the 3-bit setting on CIFAR-100 and even surpasses the full-precision baseline on CIFAR-10.
  arXiv  Detail & Related papers  (2025-03-24T04:44:21Z)
• A Survey of Early Exit Deep Neural Networks in NLP [5.402030962296633]
  Deep Neural Networks (DNNs) have grown increasingly large in size to achieve state of the art performance across a wide range of tasks.<n>High computational requirements make them less suitable for resource-constrained applications.<n>Early exit strategies offer a promising solution by enabling adaptive inference.
  arXiv  Detail & Related papers  (2025-01-13T20:08:52Z)
• Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis [50.18156030818883]
  Anomaly and missing data constitute a thorny problem in industrial applications. Deep learning enabled anomaly detection has emerged as a critical direction. The data collected in edge devices contain user privacy.
  arXiv  Detail & Related papers  (2024-11-06T15:38:31Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• Fast Hierarchical Learning for Few-Shot Object Detection [57.024072600597464]
  Transfer learning approaches have recently achieved promising results on the few-shot detection task. These approaches suffer from catastrophic forgetting'' issue due to finetuning of base detector. We tackle the aforementioned issues in this work.
  arXiv  Detail & Related papers  (2022-10-10T20:31:19Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning [102.78958681141577]
  We present SUNRISE, a simple unified ensemble method, which is compatible with various off-policy deep reinforcement learning algorithms. SUNRISE integrates two key ingredients: (a) ensemble-based weighted Bellman backups, which re-weight target Q-values based on uncertainty estimates from a Q-ensemble, and (b) an inference method that selects actions using the highest upper-confidence bounds for efficient exploration.
  arXiv  Detail & Related papers  (2020-07-09T17:08:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.