Using reinforcement learning to autonomously identify sources of error for agents in group missions

• URL: http://arxiv.org/abs/2107.09232v4
• Date: Sat, 4 Nov 2023 18:11:41 GMT
• Title: Using reinforcement learning to autonomously identify sources of error for agents in group missions
• Abstract summary: We consider the question as to whether artificial intelligence can autonomously generate an action plan to pinpoint the cause as aforedescribed. To overcome this problem, we formulated an action plan using Q-table reinforcement learning. Surprisingly, the optimal action plan generated via reinforcement learning presented a human-like solution to pinpoint the problem.
• Score: 0.22499166814992436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When agents swarm to execute a mission, some of them frequently exhibit sudden failure, as observed from the command base. It is generally difficult to determine whether a failure is caused by actuators (hypothesis, $h_a$) or sensors (hypothesis, $h_s$) by solely relying on the communication between the command base and concerning agent. However, by instigating collusion between the agents, the cause of failure can be identified; in other words, we expect to detect corresponding displacements for $h_a$ but not for $h_s$. In this study, we considered the question as to whether artificial intelligence can autonomously generate an action plan $\boldsymbol{g}$ to pinpoint the cause as aforedescribed. Because the expected response to $\boldsymbol{g}$ generally depends upon the adopted hypothesis [let the difference be denoted by $D(\boldsymbol{g})$], a formulation that uses $D\left(\boldsymbol{g}\right)$ to pinpoint the cause can be made. Although a $\boldsymbol{g}^*$ that maximizes $D(\boldsymbol{g})$ would be a suitable action plan for this task, such an optimization is difficult to achieve using the conventional gradient method, as $D(\boldsymbol{g})$ becomes nonzero in rare events such as collisions with other agents, and most swarm actions $\boldsymbol{g}$ give $D(\boldsymbol{g})=0$. In other words, throughout almost the entire space of $\boldsymbol{g}$, $D(\boldsymbol{g})$ has zero gradient, and the gradient method is not applicable. To overcome this problem, we formulated an action plan using Q-table reinforcement learning. Surprisingly, the optimal action plan generated via reinforcement learning presented a human-like solution to pinpoint the problem by colliding other agents with the failed agent. Using this simple prototype, we demonstrated the potential of applying Q-table reinforcement learning methods to plan autonomous actions to pinpoint the causes of failure.

Related papers

• A Mathematical Theory of Value: a synthesis on goal-directed agency under resource constraints [6.057587531186626]
  We show that value, the quantity goal-directed agents create, destroy, and exchange, is a lawful structural quantity in the same category as information.<n>For populations, value is frame-relative while price is frame-independent; a fleet that pools its resource and fuses its perception inherits the ceiling.
  arXiv  Detail & Related papers  (2026-06-10T16:11:04Z)
• Formal Analysis of AGI Decision-Theoretic Models and the Confrontation Question [0.0]
  Artificial General Intelligence (AGI) may face a confrontation question: under what conditions would a self-interested AGI choose to seize power or eliminate human control (a confrontation) rather than remain cooperative?<n>We formalize this in a Markov decision process with a human-initiated shutdown event.<n>We show that for almost all reward functions a misaligned agent has an incentive to avoid shutdown.
  arXiv  Detail & Related papers  (2026-01-04T08:02:00Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [55.044159987218436]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• FLARE: Robot Learning with Implicit World Modeling [87.81846091038676]
  $textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
  arXiv  Detail & Related papers  (2025-05-21T15:33:27Z)
• Symmetry-Breaking Descent for Invariant Cost Functionals [0.0]
  We study the problem of reducing a task cost functional $W(S)$, defined over Sobolev-class signals $S$, when the cost is invariant under a global symmetry group $G subset mathrmDiff(M)$.<n>We propose a variational method that exploits the symmetry structure to construct explicit, symmetry-breaking deformations of the input signal.
  arXiv  Detail & Related papers  (2025-05-19T15:06:31Z)
• Cracking the Code of Action: a Generative Approach to Affordances for Reinforcement Learning [33.790048240113165]
  In this work, we consider the low-data regime, with limited or no access to expert behavior. We propose $textbfCode as Generative Affordances$ $(textbf$textttCoGA$)$. By greatly reducing the number of actions that an agent must consider, we demonstrate on a wide range of tasks in the MiniWob++ benchmark.
  arXiv  Detail & Related papers  (2025-04-24T06:20:08Z)
• Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecification [50.717692060500696]
  Next-token prediction with the logarithmic loss is a cornerstone of autoregressive sequence modeling. Next-token prediction can be made robust so as to achieve $C=tilde O(H)$, representing moderate error amplification. No computationally efficient algorithm can achieve sub-polynomial approximation factor $C=e(log H)1-Omega(1)$.
  arXiv  Detail & Related papers  (2025-02-18T02:52:00Z)
• Near-Optimal Online Learning for Multi-Agent Submodular Coordination: Tight Approximation and Communication Efficiency [52.60557300927007]
  We present a $textbfMA-OSMA$ algorithm to transfer the discrete submodular problem into a continuous optimization.<n>We also introduce a projection-free $textbfMA-OSEA$ algorithm, which effectively utilizes the KL divergence by mixing a uniform distribution.<n>Our algorithms significantly improve the $(frac11+c)$-approximation provided by the state-of-the-art OSG algorithm.
  arXiv  Detail & Related papers  (2025-02-07T15:57:56Z)
• Conditional Mutual Information Based Diffusion Posterior Sampling for Solving Inverse Problems [3.866047645663101]
  In computer vision, tasks such as inpainting, deblurring, and super-resolution are commonly formulated as inverse problems. Recently, diffusion models (DMs) have emerged as a promising approach for addressing noisy linear inverse problems. We propose an information-theoretic approach to improve the effectiveness of DMs in solving inverse problems.
  arXiv  Detail & Related papers  (2025-01-06T09:45:26Z)
• IT$^3$: Idempotent Test-Time Training [95.78053599609044]
  This paper introduces Idempotent Test-Time Training (IT$3$), a novel approach to addressing the challenge of distribution shift. IT$3$ is based on the universal property of idempotence. We demonstrate the versatility of our approach across various tasks, including corrupted image classification.
  arXiv  Detail & Related papers  (2024-10-05T15:39:51Z)
• Any Target Can be Offense: Adversarial Example Generation via Generalized Latent Infection [83.72430401516674]
  GAKer is able to construct adversarial examples to any target class. Our method achieves an approximately $14.13%$ higher attack success rate for unknown classes.
  arXiv  Detail & Related papers  (2024-07-17T03:24:09Z)
• The Selective G-Bispectrum and its Inversion: Applications to G-Invariant Networks [3.8311785959108637]
  We show that the $G$-Bispectrum can be reduced into a textitselective $G$-Bispectrum. We demonstrate how its integration in neural networks enhances accuracy and robustness compared to traditional approaches.
  arXiv  Detail & Related papers  (2024-07-10T13:35:04Z)
• SynGhost: Imperceptible and Universal Task-agnostic Backdoor Attack in Pre-trained Language Models [15.59613994912779]
  Pre-training has been a necessary phase for deploying pre-trained language models (PLMs) to achieve remarkable performance in downstream tasks. We show that backdoor attacks exploit such a phase as a vulnerable entry point for task-agnostic. We present $mathttSynGhost$, an imperceptible and universal task-agnostic backdoor attack in PLMs.
  arXiv  Detail & Related papers  (2024-02-29T08:20:49Z)
• Federated Learning in the Presence of Adversarial Client Unavailability [16.201377650598516]
  Federated learning is a decentralized machine learning framework that enables collaborative model without revealing raw data. Due to the diverse hardware software limitations, a client may not always be available for the computation requests from the server. In harsh environments like battlefields, adversaries can selectively silence specific clients.
  arXiv  Detail & Related papers  (2023-05-31T15:57:07Z)
• The Sample Complexity of Online Contract Design [120.9833763323407]
  We study the hidden-action principal-agent problem in an online setting. In each round, the principal posts a contract that specifies the payment to the agent based on each outcome. The agent then makes a strategic choice of action that maximizes her own utility, but the action is not directly observable by the principal.
  arXiv  Detail & Related papers  (2022-11-10T17:59:42Z)
• Approximate Function Evaluation via Multi-Armed Bandits [51.146684847667125]
  We study the problem of estimating the value of a known smooth function $f$ at an unknown point $boldsymbolmu in mathbbRn$, where each component $mu_i$ can be sampled via a noisy oracle. We design an instance-adaptive algorithm that learns to sample according to the importance of each coordinate, and with probability at least $1-delta$ returns an $epsilon$ accurate estimate of $f(boldsymbolmu)$.
  arXiv  Detail & Related papers  (2022-03-18T18:50:52Z)
• Fast Gradient Non-sign Methods [67.56549792690706]
  Fast Gradient Non-sign Method (FGNM) is a general routine, which can seamlessly replace the conventional $sign$ operation in gradient-based attacks. Our methods outperform them by textbf27.5% at most and textbf9.5% on average.
  arXiv  Detail & Related papers  (2021-10-25T08:46:00Z)
• What Happens after SGD Reaches Zero Loss? --A Mathematical Framework [35.31946061894308]
  Understanding the implicit bias of Gradient Descent (SGD) is one of the key challenges in deep learning. This paper gives a general framework for such analysis by adapting ideas from Katzenberger (1991). It yields some new results: (1) a global analysis of the implicit bias valid for $eta-2$ steps, in contrast to the local analysis of Blanc et al. ( 2020) that is only valid for $eta-1.6$ steps and (2) allowing arbitrary noise covariance.
  arXiv  Detail & Related papers  (2021-10-13T17:50:46Z)
• Learning to extrapolate using continued fractions: Predicting the critical temperature of superconductor materials [5.905364646955811]
  In the field of Artificial Intelligence (AI) and Machine Learning (ML), the approximation of unknown target functions $y=f(mathbfx)$ is a common objective. We refer to $S$ as the training set and aim to identify a low-complexity mathematical model that can effectively approximate this target function for new instances $mathbfx$.
  arXiv  Detail & Related papers  (2020-11-27T04:57:40Z)
• Agnostic Learning of a Single Neuron with Gradient Descent [92.7662890047311]
  We consider the problem of learning the best-fitting single neuron as measured by the expected square loss. For the ReLU activation, our population risk guarantee is $O(mathsfOPT1/2)+epsilon$. For the ReLU activation, our population risk guarantee is $O(mathsfOPT1/2)+epsilon$.
  arXiv  Detail & Related papers  (2020-05-29T07:20:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.