Adaptive Data Harvesting for Efficient Neural Network Learning with Universal Constraints

• URL: http://arxiv.org/abs/2605.09707v1
• Date: Sun, 10 May 2026 19:09:49 GMT
• Title: Adaptive Data Harvesting for Efficient Neural Network Learning with Universal Constraints
• Abstract summary: Training neural networks to satisfy universal constraints over continuous domains poses unique challenges.<n>In this paper, we aim to improve upon existing methods by learning, from data and experience, how to dynamically and iteratively adjust the samples in response to the model's evolving learning performance.<n>We validate the approach on both Lyapunov NNs and PINNs, and demonstrate its broader applicability to domains where adaptive input selection is essential for effective training.
• Score: 9.63425149010059
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training neural networks to satisfy universal constraints over continuous domains poses unique challenges. Common examples include Lyapunov Neural Networks (Lyapunov NNs) and Physics-Informed Neural Networks (PINNs), where analytical solutions are generally either unavailable or overly restrictive. Sample-based methods are therefore commonly used to enforce these constraints, and the choice of samples has a substantial impact on convergence speed, stability, and solution quality. Most existing methods rely on fixed heuristics or handcrafted rules, and are suboptimal in practice. In this paper, we aim to improve upon them by learning, from data and experience, how to dynamically and iteratively adjust the samples in response to the model's evolving learning performance. Trained by reinforcement learning, the learned policy improves empirical constraint satisfaction on test problems while significantly improving efficiency. We validate the approach on both Lyapunov NNs and PINNs, and demonstrate its broader applicability to domains where adaptive input selection is essential for effective training.

Related papers

• Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks [56.42974363744522]
  We study the learning dynamics of random feedback local online (RFLO) and truncated backpropagation through time (tBPTT)<n>Our work provides analytical insight into how locality constraints shape learning dynamics, with implications for neuroscientific models of learning and alternative optimization approaches for RNNs.
  arXiv  Detail & Related papers  (2026-05-29T18:19:45Z)
• When Learning Hurts: Fixed-Pole RNN for Real-Time Online Training [58.25341036646294]
  We analytically examine why learning recurrent poles does not provide tangible benefits in data and empirically offer real-time learning scenarios.<n>We show that fixed-pole networks achieve superior performance with lower training complexity, making them more suitable for online real-time tasks.
  arXiv  Detail & Related papers  (2026-02-25T00:15:13Z)
• Self-adaptive weighting and sampling for physics-informed neural networks [0.5302833929459496]
  We introduce a hybrid adaptive sampling and weighting method to enhance the performance of physics-informed neural networks (PINNs)<n>The proposed framework consistently improves prediction accuracy and training efficiency, offering a more robust approach for solving PDEs with PINNs.
  arXiv  Detail & Related papers  (2025-11-07T17:48:11Z)
• Physics-Informed Neural Network Frameworks for the Analysis of Engineering and Biological Dynamical Systems Governed by Ordinary Differential Equations [0.0]
  PINNs offer a powerful approach for handling challenging numerical scenarios.<n> PINNs can achieve superior results by embedding physical laws directly into the learning process.
  arXiv  Detail & Related papers  (2025-10-28T04:00:59Z)
• Optimizers Qualitatively Alter Solutions And We Should Leverage This [62.662640460717476]
  Deep Neural Networks (DNNs) can not guarantee convergence to a unique global minimum of the loss when using only local information, such as SGD.<n>We argue that the community should aim at understanding the biases of already existing methods, as well as aim to build new DNNs with the explicit intent of inducing certain properties of the solution.
  arXiv  Detail & Related papers  (2025-07-16T13:33:31Z)
• Adaptive Physics-informed Neural Networks: A Survey [15.350973327319418]
  Physics-informed neural networks (PINNs) have emerged as a promising approach to solving partial differential equations.<n>This survey reviews existing research that addresses limitations through transfer learning and meta-learning.
  arXiv  Detail & Related papers  (2025-03-23T19:33:05Z)
• Switchable Decision: Dynamic Neural Generation Networks [98.61113699324429]
  We propose a switchable decision to accelerate inference by dynamically assigning resources for each data instance. Our method benefits from less cost during inference while keeping the same accuracy.
  arXiv  Detail & Related papers  (2024-05-07T17:44:54Z)
• Achieving Constraints in Neural Networks: A Stochastic Augmented Lagrangian Approach [49.1574468325115]
  Regularizing Deep Neural Networks (DNNs) is essential for improving generalizability and preventing overfitting. We propose a novel approach to DNN regularization by framing the training process as a constrained optimization problem. We employ the Augmented Lagrangian (SAL) method to achieve a more flexible and efficient regularization mechanism.
  arXiv  Detail & Related papers  (2023-10-25T13:55:35Z)
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation [49.44309457870649]
  Layer-wise Feedback feedback (LFP) is a novel training principle for neural network-like predictors.<n>LFP decomposes a reward to individual neurons based on their respective contributions.<n>Our method then implements a greedy reinforcing approach helpful parts of the network and weakening harmful ones.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• An Expert's Guide to Training Physics-informed Neural Networks [5.198985210238479]
  Physics-informed neural networks (PINNs) have been popularized as a deep learning framework. PINNs can seamlessly synthesize observational data and partial differential equation (PDE) constraints. We present a series of best practices that can significantly improve the training efficiency and overall accuracy of PINNs.
  arXiv  Detail & Related papers  (2023-08-16T16:19:25Z)
• Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings [0.0]
  This paper presents a PINN training framework that employs pre-training steps and a net-to-net knowledge transfer algorithm. A multi-objective optimization algorithm may improve the performance of a physical-informed neural network with competing constraints.
  arXiv  Detail & Related papers  (2021-07-24T00:43:17Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.