How to Correctly do Semantic Backpropagation on Language-based Agentic Systems

• URL: http://arxiv.org/abs/2412.03624v1
• Date: Wed, 04 Dec 2024 15:52:03 GMT
• Title: How to Correctly do Semantic Backpropagation on Language-based Agentic Systems
• Authors: Wenyi Wang, Hisham A. Alyahya, Dylan R. Ashley, Oleg Serikov, Dmitrii Khizbullin, Francesco Faccio, Jürgen Schmidhuber,
• Abstract summary: We formalize the concept of semantic backpropagation with semantic gradients.<n>This serves as a method for computing directional information about how changes to each component might improve the system's output.<n>Our results on both BIG-Bench Hard and GSM8K show that our approach outperforms existing state-of-the-art methods for solving GASO problems.
• Score: 23.4193991777817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language-based agentic systems have shown great promise in recent years, transitioning from solving small-scale research problems to being deployed in challenging real-world tasks. However, optimizing these systems often requires substantial manual labor. Recent studies have demonstrated that these systems can be represented as computational graphs, enabling automatic optimization. Despite these advancements, most current efforts in Graph-based Agentic System Optimization (GASO) fail to properly assign feedback to the system's components given feedback on the system's output. To address this challenge, we formalize the concept of semantic backpropagation with semantic gradients -- a generalization that aligns several key optimization techniques, including reverse-mode automatic differentiation and the more recent TextGrad by exploiting the relationship among nodes with a common successor. This serves as a method for computing directional information about how changes to each component of an agentic system might improve the system's output. To use these gradients, we propose a method called semantic gradient descent which enables us to solve GASO effectively. Our results on both BIG-Bench Hard and GSM8K show that our approach outperforms existing state-of-the-art methods for solving GASO problems. A detailed ablation study on the LIAR dataset demonstrates the parsimonious nature of our method. A full copy of our implementation is publicly available at https://github.com/HishamAlyahya/semantic_backprop

Related papers

• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture. Non-smooth regularization is often incorporated into machine learning tasks. We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• GDSG: Graph Diffusion-based Solution Generator for Optimization Problems in MEC Networks [109.17835015018532]
  We present a Graph Diffusion-based Solution Generation (GDSG) method. This approach is designed to work with suboptimal datasets while converging to the optimal solution large probably. We build GDSG as a multi-task diffusion model utilizing a Graph Neural Network (GNN) to acquire the distribution of high-quality solutions.
  arXiv  Detail & Related papers  (2024-12-11T11:13:43Z)
• Graph-Sequential Alignment and Uniformity: Toward Enhanced Recommendation Systems [51.716704243764994]
  Our framework uses Graph Neural Network (GNN)-based and sequential recommenders as separate submodules while sharing a unified embedding space optimized jointly.<n> Experiments on three real-world datasets demonstrate that the proposed method significantly outperforms using either approach alone.
  arXiv  Detail & Related papers  (2024-12-05T15:59:05Z)
• Semantic Tokens in Retrieval Augmented Generation [0.0]
  I propose a novel Comparative RAG system that introduces an evaluator module to bridge the gap between probabilistic RAG systems and deterministically verifiable responses.<n>This framework paves the way for more reliable and scalable question-answering applications in domains requiring high precision and verifiability.
  arXiv  Detail & Related papers  (2024-12-03T16:52:06Z)
• Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models [102.72940700598055]
  In reasoning tasks, even a minor error can cascade into inaccurate results. We develop a method that avoids introducing external resources, relying instead on perturbations to the input. Our training approach randomly masks certain tokens within the chain of thought, a technique we found to be particularly effective for reasoning tasks.
  arXiv  Detail & Related papers  (2024-03-04T16:21:54Z)
• Language Agents as Optimizable Graphs [31.220547147952278]
  We describe Large Language Models (LLMs)-based agents as computational graphs. Our framework can be used to efficiently develop, integrate, and automatically improve various LLM agents.
  arXiv  Detail & Related papers  (2024-02-26T18:48:27Z)
• Self-Supervised Learning with Lie Symmetries for Partial Differential Equations [25.584036829191902]
  We learn general-purpose representations of PDEs by implementing joint embedding methods for self-supervised learning (SSL) Our representation outperforms baseline approaches to invariant tasks, such as regressing the coefficients of a PDE, while also improving the time-stepping performance of neural solvers. We hope that our proposed methodology will prove useful in the eventual development of general-purpose foundation models for PDEs.
  arXiv  Detail & Related papers  (2023-07-11T16:52:22Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Self Correspondence Distillation for End-to-End Weakly-Supervised Semantic Segmentation [13.623713806739271]
  We propose a novel Self Correspondence Distillation (SCD) method to refine pseudo-labels without introducing external supervision. In addition, we design a Variation-aware Refine Module to enhance the local consistency of pseudo-labels. Our method significantly outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-27T13:46:40Z)
• RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation [110.4255414234771]
  Existing solutions require massive training data or lack generalizability to unknown rendering configurations. We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem. Our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.
  arXiv  Detail & Related papers  (2022-05-11T17:59:51Z)
• Beyond backpropagation: implicit gradients for bilevel optimization [0.0]
  Bilevel optimization is a way to frame the learning of systems that are implicitly defined through a quantity that they minimize. Here we focus on gradient-based approaches that solve such problems. We present the mathematical foundations that are behind such methods, introduce the gradient-estimation algorithms in detail and compare the competitive advantages of the different approaches.
  arXiv  Detail & Related papers  (2022-05-06T08:53:46Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• Harnessing Heterogeneity: Learning from Decomposed Feedback in Bayesian Modeling [68.69431580852535]
  We introduce a novel GP regression to incorporate the subgroup feedback. Our modified regression has provably lower variance -- and thus a more accurate posterior -- compared to previous approaches. We execute our algorithm on two disparate social problems.
  arXiv  Detail & Related papers  (2021-07-07T03:57:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.