Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning

• URL: http://arxiv.org/abs/2505.16270v1
• Date: Thu, 22 May 2025 06:00:45 GMT
• Title: Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
• Abstract summary: We introduce the concept of Mistake Log to track the model's learning behavior and recurring errors throughout fine-tuning.<n>Treating the original transformer-based model as the Pilot, we correspondingly design a Copilot model to refine the Pilot's inference performance via logits rectification.<n>We name the overall Pilot-Copilot framework the Transformer Copilot, which introduces (i) a novel Copilot model design, (ii) a joint training paradigm where the Copilot continuously learns from the evolving Mistake Log alongside the Pilot, and (iii) a fused inference paradigm where the Copilot rectifies the Pilot's
• Score: 43.04412816198813
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are typically adapted to downstream tasks through supervised fine-tuning on domain-specific data. While standard fine-tuning focuses on minimizing generation loss to optimize model parameters, we take a deeper step by retaining and leveraging the model's own learning signals, analogous to how human learners reflect on past mistakes to improve future performance. We first introduce the concept of Mistake Log to systematically track the model's learning behavior and recurring errors throughout fine-tuning. Treating the original transformer-based model as the Pilot, we correspondingly design a Copilot model to refine the Pilot's inference performance via logits rectification. We name the overall Pilot-Copilot framework the Transformer Copilot, which introduces (i) a novel Copilot model design, (ii) a joint training paradigm where the Copilot continuously learns from the evolving Mistake Log alongside the Pilot, and (iii) a fused inference paradigm where the Copilot rectifies the Pilot's logits for enhanced generation. We provide both theoretical and empirical analyses on our new learning framework. Experiments on 12 benchmarks spanning commonsense, arithmetic, and recommendation tasks demonstrate that Transformer Copilot consistently improves performance by up to 34.5%, while introducing marginal computational overhead to Pilot models and exhibiting strong scalability and transferability.

Related papers

• Perceptual Flow Matching for Few-Step Generative Modeling [64.26537634881244]
  Perceptual Flow Matching is a framework for few-step generation in flow-matching models.<n>We show that PFM produces high-quality few-step generators when supervised in an appropriate representation space.
  arXiv  Detail & Related papers  (2026-07-03T17:55:01Z)
• Generative Augmentation of Imbalanced Flight Records for Flight Diversion Prediction: A Multi-objective Optimisation Framework [0.0]
  Flight diversions are rare but high-impact events in aviation, making their reliable prediction vital for both safety and operational efficiency.<n>This study investigates how generative models can augment historical flight data with synthetic diversion records to enhance model training and improve predictive accuracy.
  arXiv  Detail & Related papers  (2026-04-22T07:35:11Z)
• Agent2World: Learning to Generate Symbolic World Models via Adaptive Multi-Agent Feedback [51.22403664895878]
  Agent2World is a tool-augmented multi-agent framework that achieves strong inference-time world-model generation.<n>It also serves as a data engine for supervised fine-tuning, by grounding generation in multi-agent feedback.
  arXiv  Detail & Related papers  (2025-12-26T18:54:14Z)
• From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model [72.73512218682187]
  We introduce ReDiff, a refining-enhanced diffusion framework that teaches the model to identify and correct its own errors.<n>Our approach features a two-stage training process: first, we instill a foundational revision capability by training the model to revise synthetic errors; second, we implement a novel online self-correction loop.<n>This mistake-driven learning endows the model with the crucial ability to revisit and refine its already generated output, effectively breaking the error cascade.
  arXiv  Detail & Related papers  (2025-10-22T06:58:55Z)
• ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving [64.42138266293202]
  ResAD is a Normalized Residual Trajectory Modeling framework.<n>It reframes the learning task to predict the residual deviation from an inertial reference.<n>On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy.
  arXiv  Detail & Related papers  (2025-10-09T17:59:36Z)
• The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains [50.66245575710432]
  We show that paired preference data consisting of individually weak data points can enable gains beyond the strength of each individual data point.<n>Our work shows that models can learn surprisingly well from paired data that might typically be considered weak.
  arXiv  Detail & Related papers  (2025-07-08T17:14:44Z)
• Consistency-based Abductive Reasoning over Perceptual Errors of Multiple Pre-trained Models in Novel Environments [5.5855749614100825]
  This paper addresses the hypothesis that leveraging multiple pre-trained models can mitigate this recall reduction.<n>We formulate the challenge of identifying and managing conflicting predictions from various models as a consistency-based abduction problem.<n>Our results validate the use of consistency-based abduction as an effective mechanism to robustly integrate knowledge from multiple imperfect models in challenging, novel scenarios.
  arXiv  Detail & Related papers  (2025-05-25T23:17:47Z)
• Reinforcement Twinning for Hybrid Control of Flapping-Wing Drones [0.0]
  This article presents a novel hybrid model-free/model-based approach to flight control based on the proposed reinforcement twinning algorithm.<n>The algorithm is evaluated for controlling the longitudinal dynamics of a flapping-wing drone.
  arXiv  Detail & Related papers  (2025-05-21T12:27:09Z)
• Flying Quadrotors in Tight Formations using Learning-based Model Predictive Control [30.715469693232492]
  In this work, we propose a framework that combines the benefits of first-principles modeling and data-driven approaches. We show that incorporating the model into a novel learning-based predictive model control framework results in substantial performance improvements. Our framework also achieves exceptional sample efficiency, using only a total of 46 seconds of flight data for training.
  arXiv  Detail & Related papers  (2024-10-13T05:03:16Z)
• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions.<n>During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations.<n>We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion [36.321494200830244]
  Copilot4D is a novel world modeling approach that first tokenizes sensor observations with VQVAE, then predicts the future via discrete diffusion. Our results demonstrate that discrete diffusion on tokenized agent experience can unlock the power of GPT-like unsupervised learning for robotics.
  arXiv  Detail & Related papers  (2023-11-02T06:21:56Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Context-Aware Generative Models for Prediction of Aircraft Ground Tracks [0.004807514276707785]
  Trajectory prediction plays an important role in supporting the decision-making of Air Traffic Controllers. Traditional TP methods are deterministic and physics-based, with parameters calibrated using aircraft surveillance data harvested across the world. This work proposes a generative method for lateral TP, using probabilistic machine learning to model the effect of the unknown effect of pilot behaviour and ATCO intentions.
  arXiv  Detail & Related papers  (2023-09-26T14:20:09Z)
• Learning Generative Models for Climbing Aircraft from Radar Data [0.0]
  This paper proposes a generative model for climbing aircraft in which the standard Base of Aircraft Data (BADA) model is enriched by a functional correction to the thrust that is learned from data.<n>The method offers three features: predictions of the arrival time with 26.7% less error when compared to BADA; generated trajectories that are realistic when compared to test data.
  arXiv  Detail & Related papers  (2023-09-26T13:53:53Z)
• SO(2)-Equivariant Downwash Models for Close Proximity Flight [6.297269227845377]
  We present a novel learning-based approach for modelling the downwash forces that exploits the latent geometries (i.e. symmetries) present in the problem. We demonstrate that when trained with only 5 minutes of real-world flight data, our geometry-aware model outperforms state-of-the-art baseline models trained with more than 15 minutes of data.
  arXiv  Detail & Related papers  (2023-05-30T12:27:47Z)
• Emergent Agentic Transformer from Chain of Hindsight Experience [96.56164427726203]
  We show that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches. This is the first time that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
  arXiv  Detail & Related papers  (2023-05-26T00:43:02Z)
• Inferring Traffic Models in Terminal Airspace from Flight Tracks and Procedures [52.25258289718559]
  We propose a probabilistic model that can learn the variability from procedural data and flight tracks collected from radar surveillance data. We show how a pairwise model can be used to generate traffic involving an arbitrary number of aircraft.
  arXiv  Detail & Related papers  (2023-03-17T13:58:06Z)
• Data-Efficient Modeling for Precise Power Consumption Estimation of Quadrotor Operations Using Ensemble Learning [3.722516004544342]
  Electric Take-Off and Landing (eVTOL) aircraft is considered as the major aircraft type in the emerging urban air mobility. In this study, a framework for power consumption modeling of eVTOL aircraft was established. We employed an ensemble learning method, namely stacking, to develop a data-driven model using flight records of three different types of quadrotors.
  arXiv  Detail & Related papers  (2022-05-23T02:16:43Z)
• Churn Reduction via Distillation [54.5952282395487]
  We show an equivalence between training with distillation using the base model as the teacher and training with an explicit constraint on the predictive churn. We then show that distillation performs strongly for low churn training against a number of recent baselines.
  arXiv  Detail & Related papers  (2021-06-04T18:03:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.