A Systematic Study of Behavioral Cloning for Scientific Data Annotation

• URL: http://arxiv.org/abs/2606.07568v1
• Date: Tue, 26 May 2026 02:19:47 GMT
• Title: A Systematic Study of Behavioral Cloning for Scientific Data Annotation
• Abstract summary: We introduce a framework for studying behavioral cloning on scientific annotation.<n>We show that models learn GUI mechanics before task-critical decisions, and commit fewer mistakes than the training data.<n> scaling models on multi-task behavioral cloning shows that larger models are more data efficient within our scale range.
• Score: 5.5575224613422725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific data annotation, such as tracking animals in video or proofreading neural reconstructions, remains bottlenecked by the "last mile" problem: even with strong automation, verification and correction consume substantial human effort. Standard approaches train models to directly predict annotations, discarding the rich supervision in how experts navigate, click, verify, and correct. We introduce a framework for studying behavioral cloning on scientific annotation: 9 synthetic tasks paired with synthetic annotations that simulate realistic human strategies including exploration, mistake correction, and strategic decision-making. Our experiments reveal several findings. First, skills emerge hierarchically: models learn GUI mechanics before task-critical decisions, and commit fewer mistakes than the training data while retaining the ability to correct errors when they occur. Second, scaling models on multi-task behavioral cloning shows that larger models are more data efficient within our scale range. Third, multi-task pretraining enables efficient fine-tuning to new tasks, while training from scratch fails entirely. Fourth, linear probes reveal that models internally represent latent variables of the annotation process such as task phase and data position; interestingly, we find a shared mistake representation that generalizes across different annotation tasks. Overall, our framework establishes systematic benchmarks and identifies key bottlenecks, providing a foundation for scaling behavioral cloning to real-world scientific data annotation.

Related papers

• Learning Feature Encoder with Synthetic Anomalies for Weakly Supervised Graph Anomaly Detection [66.0756595768692]
  We introduce a weakly supervised graph anomaly detection method that leverages a feature learning strategy tailored for graph anomalies.<n>Our approach is built upon a multi-task learning scheme that extracts robust feature representations through synthesized anomalies.<n>Experiments on public datasets demonstrate the superior performance of our method over its competitors.
  arXiv  Detail & Related papers  (2026-05-12T08:27:06Z)
• What Do Learning Dynamics Reveal About Generalization in LLM Reasoning? [83.83230167222852]
  We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy. By connecting a model's learning behavior to its generalization, pre-memorization train accuracy can guide targeted improvements to training strategies.
  arXiv  Detail & Related papers  (2024-11-12T09:52:40Z)
• Inverse Scaling: When Bigger Isn't Better [80.42834197416444]
  Large language models (LMs) show predictable improvements to overall loss with increased scale. We present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale.
  arXiv  Detail & Related papers  (2023-06-15T20:11:23Z)
• CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation [4.38301148531795]
  We address the problem of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently.
  arXiv  Detail & Related papers  (2022-12-28T23:00:50Z)
• Learning Curves for Sequential Training of Neural Networks: Self-Knowledge Transfer and Forgetting [9.734033555407406]
  We consider neural networks in the neural tangent kernel regime that continually learn target functions from task to task. We investigate a variant of continual learning where the model learns the same target function in multiple tasks. Even for the same target, the trained model shows some transfer and forgetting depending on the sample size of each task.
  arXiv  Detail & Related papers  (2021-12-03T00:25:01Z)
• X-model: Improving Data Efficiency in Deep Learning with A Minimax Model [78.55482897452417]
  We aim at improving data efficiency for both classification and regression setups in deep learning. To take the power of both worlds, we propose a novel X-model. X-model plays a minimax game between the feature extractor and task-specific heads.
  arXiv  Detail & Related papers  (2021-10-09T13:56:48Z)
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach [80.8446673089281]
  We propose a new learning paradigm with graph representation and learning. Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
  arXiv  Detail & Related papers  (2021-10-09T09:02:45Z)
• Annotating and Modeling Fine-grained Factuality in Summarization [36.88018450067003]
  A major barrier to their use in practice is their propensity to output summaries that are not faithful to the input and that contain factual errors. We explore both synthetic and human-labeled data sources for training models to identify factual errors in summarization. We show that our best factuality detection model enables training of more factual XSum summarization models by allowing us to identify non-factual tokens in the training data.
  arXiv  Detail & Related papers  (2021-04-09T11:20:44Z)
• Self-supervised driven consistency training for annotation efficient histopathology image analysis [13.005873872821066]
  Training a neural network with a large labeled dataset is still a dominant paradigm in computational histopathology. We propose a self-supervised pretext task that harnesses the underlying multi-resolution contextual cues in histology whole-slide images to learn a powerful supervisory signal for unsupervised representation learning. We also propose a new teacher-student semi-supervised consistency paradigm that learns to effectively transfer the pretrained representations to downstream tasks based on prediction consistency with the task-specific un-labeled data.
  arXiv  Detail & Related papers  (2021-02-07T19:46:21Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.