A Domain-Specific Language for LLM-Driven Trigger Generation in Multimodal Data Collection

• URL: http://arxiv.org/abs/2604.13046v1
• Date: Fri, 13 Mar 2026 11:53:42 GMT
• Title: A Domain-Specific Language for LLM-Driven Trigger Generation in Multimodal Data Collection
• Abstract summary: This paper proposes a declarative framework for intent-driven, on-device data collection.<n>It enables selective collection of multimodal sensor data based on high-level user requests.<n>It achieves higher generation consistency and lower execution latency than unconstrained code generation.
• Score: 0.8466401378239364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-driven systems depend on task-relevant data, yet data collection pipelines remain passive and indiscriminate. Continuous logging of multimodal sensor streams incurs high storage costs and captures irrelevant data. This paper proposes a declarative framework for intent-driven, on-device data collection that enables selective collection of multimodal sensor data based on high-level user requests. The framework combines natural language interaction with a formally specified domain-specific language (DSL). Large language models translate user-defined requirements into verifiable and composable DSL programs that define conditional triggers across heterogeneous sensors, including cameras, LiDAR, and system telemetry. Empirical evaluation on vehicular and robotic perception tasks shows that the DSL-based approach achieves higher generation consistency and lower execution latency than unconstrained code generation while maintaining comparable detection performance. The structured abstraction supports modular trigger composition and concurrent deployment on resource-constrained edge platforms. This approach replaces passive logging with a verifiable, intent-driven mechanism for multimodal data collection in real-time systems.

Related papers

• Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges [58.887152635127165]
  Diffusion language models (DLMs) offer a non-autoregressive alternative for mobile edge agentic artificial intelligence (AI)<n>DLMs can update multiple uncertain tokens in parallel and exploit bidirectional context throughout the generation process.<n>This survey reviews DLM foundations and analyzes their suitability for edge settings under latency, memory, energy, bandwidth, privacy, and reliability constraints.
  arXiv  Detail & Related papers  (2026-09-04T06:16:49Z)
• Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data [52.80567461420316]
  We propose a Memory-Augmented LLM-based Multi-Agent System (textbfMALMAS) for automated feature generation.<n>MALMAS decomposes the generation process into agents with distinct responsibilities, and a Router Agent activates an appropriate subset of agents per iteration.<n>We further integrate a memory module comprising procedural memory, feedback memory, and conceptual memory, enabling iterative refinement that adaptively guides subsequent feature generation.
  arXiv  Detail & Related papers  (2026-04-22T07:09:30Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Explainable Semantic Textual Similarity via Dissimilar Span Detection [53.32175252285023]
  We introduce the task of Dissimilar Span Detection (DSD), which aims to identify semantically differing spans between pairs of texts.<n>This can help users understand which particular words or tokens negatively affect the similarity score, or be used to improve performance in STS-dependent downstream tasks.
  arXiv  Detail & Related papers  (2026-03-22T11:32:31Z)
• OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMs [31.589945976149973]
  We introduce OmniTrace, a lightweight and model-agnostic framework that formalizes attribution as a generation-time tracing problem.<n>We show that generation-aware span-level attribution produces more stable and interpretable explanations than naive self-attribution.<n>Our results suggest that treating attribution as a structured generation-time tracing problem provides a scalable foundation for transparency in omni-modal language models.
  arXiv  Detail & Related papers  (2026-03-20T17:25:00Z)
• Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining [59.2578488860426]
  Heterogeneous multi-modal remote sensing object detection aims to accurately detect objects from diverse sensors.<n>Existing approaches largely adopt a late alignment paradigm, in which modality alignment and task-specific optimization are entangled during downstream fine-tuning.<n>We propose BabelRS, a unified language-pivoted pretraining framework that explicitly decouples modality alignment from downstream task learning.
  arXiv  Detail & Related papers  (2026-03-02T11:38:12Z)
• AgentSkiller: Scaling Generalist Agent Intelligence through Semantically Integrated Cross-Domain Data Synthesis [30.512393568258105]
  Large Language Model agents demonstrate potential in solving real-world problems via tools, yet generalist intelligence is bottlenecked by scarce high-quality, long-horizon data.<n>We propose AgentSkiller, a fully automated framework synthesizing multi-turn interaction data across realistic, semantically linked domains.
  arXiv  Detail & Related papers  (2026-02-10T03:21:42Z)
• Refer-Agent: A Collaborative Multi-Agent System with Reasoning and Reflection for Referring Video Object Segmentation [50.22481337087162]
  Referring Video Object (RVOS) aims to segment objects in videos based on textual queries.<n>Refer-Agent is a collaborative multi-agent system with alternating reasoning-reflection mechanisms.
  arXiv  Detail & Related papers  (2026-02-03T14:48:12Z)
• CaveAgent: Transforming LLMs into Stateful Runtime Operators [31.548422546991915]
  We present CaveAgent, a framework that transforms the paradigm from "LLM-as-Text-Generator" to "LLM-as-As-Runtime-Runtime"<n>CaveAgent achieves a 10.5% success rate improvement on retail tasks and reduces total token consumption by 28.4% in multi-turn scenarios.
  arXiv  Detail & Related papers  (2026-01-04T15:32:47Z)
• FABRIC: Framework for Agent-Based Realistic Intelligence Creation [3.940391073007047]
  Large language models (LLMs) are increasingly deployed as agents, expected to decompose goals, invoke tools, and verify results in dynamic environments.<n>We present a unified framework for synthesizing agentic data using only LLMs, without any human-in-the-loop supervision.
  arXiv  Detail & Related papers  (2025-10-20T18:20:22Z)
• SAM2Auto: Auto Annotation Using FLASH [13.638155035372835]
  Vision-Language Models (VLMs) lag behind Large Language Models due to the scarcity of annotated datasets.<n>We introduce SAM2Auto, the first fully automated annotation pipeline for video datasets requiring no human intervention or dataset-specific training.<n>Our system employs statistical approaches to minimize detection errors while ensuring consistent object tracking throughout entire video sequences.
  arXiv  Detail & Related papers  (2025-06-09T15:15:15Z)
• Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
  Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification. In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction. Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
  arXiv  Detail & Related papers  (2024-10-24T09:09:20Z)
• ToolACE: Winning the Points of LLM Function Calling [139.07157814653638]
  ToolACE is an automatic agentic pipeline designed to generate accurate, complex, and diverse tool-learning data.<n>We demonstrate that models trained on our synthesized data, even with only 8B parameters, achieve state-of-the-art performance on the Berkeley Function-Calling Leaderboard.
  arXiv  Detail & Related papers  (2024-09-02T03:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.