IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

• URL: http://arxiv.org/abs/2604.08033v1
• Date: Thu, 09 Apr 2026 09:38:15 GMT
• Title: IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling
• Abstract summary: IoT-Brain is a framework to interact with the physical world with unprecedented reliability and efficiency.<n>We show that IoT-Brain boosts task success rate by 37.6% over the strongest search-intensive methods.<n>In real-world deployment, it approaches the reliability upper bound while reducing 4.1 times network bandwidth.
• Score: 9.212512544511576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Intelligent systems powered by large-scale sensor networks are shifting from predefined monitoring to intent-driven operation, revealing a critical Semantic-to-Physical Mapping Gap. While large language models (LLMs) excel at semantic understanding, existing perception-centric pipelines operate retrospectively, overlooking the fundamental decision of what to sense and when. We formalize this proactive decision as Semantic-Spatial Sensor Scheduling (S3) and demonstrate that direct LLM planning is unreliable due to inherent gaps in representation, reasoning, and optimization. To bridge these gaps, we introduce the Spatial Trajectory Graph (STG), a neuro-symbolic paradigm governed by a verify-before-commit discipline that transforms open-ended planning into a verifiable graph optimization problem. Based on STG, we implement IoT-Brain, a concrete system embodiment, and construct TopoSense-Bench, a campus-scale benchmark with 5,250 natural-language queries across 2,510 cameras. Evaluations show that IoT-Brain boosts task success rate by 37.6% over the strongest search-intensive methods while running nearly 2 times faster and using 6.6 times fewer prompt tokens. In real-world deployment, it approaches the reliability upper bound while reducing 4.1 times network bandwidth, providing a foundational framework for LLMs to interact with the physical world with unprecedented reliability and efficiency.

Related papers

• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling [63.679448814185456]
  Test-time scaling (TTS) has become an effective approach for improving large language model performance.<n>Existing TTS strategies are largely hand-crafted: researchers design patterns and tune them by intuition, leaving much of the computation-allocation space unexplored.<n>We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTSs to environments where TTS strategies can be discovered automatically.
  arXiv  Detail & Related papers  (2026-05-08T17:59:40Z)
• Beyond Test-Time Training: Learning to Reason via Hardware-Efficient Optimal Control [86.63490309209378]
  We formulate reasoning as optimal control and introduce the Test-Time Control layer, which performs finite-horizon LQR planning over latent states at inference time.<n>We demonstrate that embedding optimal control as an architectural layer provides an effective and scalable mechanism for reasoning beyond test-time training.
  arXiv  Detail & Related papers  (2026-03-10T05:42:13Z)
• STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learning [16.11676643415448]
  patio-temporal reasoning in time series involves the explicit synthesis of temporal dynamics, spatial dependencies, and textual context.<n>This capability is vital for high-stakes decision-making in systems such as traffic networks, power grids, and disease propagation.<n>To address the gap, we introduce ST-Bench, a benchmark consisting of four core tasks, including etiological reasoning, entity identification, correlation reasoning, and in-context forecasting.<n>We then propose STReasoner, which empowers LLM to integrate time series, graph structure, and text for explicit reasoning.
  arXiv  Detail & Related papers  (2026-01-06T18:46:12Z)
• From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs [65.04549036809557]
  We introduce a benchmark built from pedestrian-perspective videos captured with synchronized stereo cameras, LiDAR, and IMU/GPS sensors.<n>This dataset provides metrically precise 3D information, enabling the automatic generation of spatial reasoning questions.<n> Evaluations reveal that the performance gains observed in structured indoor benchmarks vanish in open-world settings.
  arXiv  Detail & Related papers  (2025-12-22T18:58:12Z)
• Think Fast: Real-Time IoT Intrusion Reasoning Using IDS and LLMs at the Edge Gateway [5.541753997410371]
  This paper presents an edge-centric Intrusion Detection System (IDS) framework that integrates lightweight machine learning (ML) based IDS models with pre-trained large language models (LLMs)<n>The system evaluates six ML-based IDS models: Decision Tree (DT), K-Nearest Neighbors (KNN), Random Forest (RF), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and a hybrid CNN-LSTM model on low-power edge gateways.<n>For anomaly detection, the system transmits a compact and secure telemetry snapshot via low-bandwidth API calls to LLMs.
  arXiv  Detail & Related papers  (2025-11-23T00:33:51Z)
• LIFT: Automating Symbolic Execution Optimization with Large Language Models for AI Networks [31.69036961288122]
  LIFT (Large-language-model Integrated Functional-equivalent-IR Transformation) is a novel framework that automates the optimization of Intermediate Representations (IRs) in symbolic execution.<n> Experiments on real-world binaries demonstrated significant performance improvements.
  arXiv  Detail & Related papers  (2025-07-07T12:26:56Z)
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks [64.86209459039313]
  ThinkGeo is an agentic benchmark designed to evaluate tool-augmented agents on remote sensing tasks via structured tool use and multi-step planning.<n>We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs on 486 structured agentic tasks with 1,773 expert-verified reasoning steps.<n>Our analysis reveals notable disparities in tool accuracy and planning consistency across models.
  arXiv  Detail & Related papers  (2025-05-29T17:59:38Z)
• Benchmarking LLMs' Swarm intelligence [51.648605206159125]
  Large Language Models (LLMs) show potential for complex reasoning, yet their capacity for emergent coordination in Multi-Agent Systems (MAS) remains largely unexplored.<n>We introduce SwarmBench, a novel benchmark designed to systematically evaluate tasks of LLMs acting as decentralized agents.<n>We propose metrics for coordination effectiveness and analyze emergent group dynamics.
  arXiv  Detail & Related papers  (2025-05-07T12:32:01Z)
• General-Purpose Aerial Intelligent Agents Empowered by Large Language Models [9.603293922137965]
  This paper presents the first aerial intelligent agent capable of open-world task execution.<n>Our hardware-software co-designed system addresses two fundamental limitations.<n>The system demonstrates reliable task planning and scene understanding in communication-constrained environments.
  arXiv  Detail & Related papers  (2025-03-11T11:13:58Z)
• Cognitive TransFuser: Semantics-guided Transformer-based Sensor Fusion for Improved Waypoint Prediction [38.971222477695214]
  RGB-LIDAR-based multi-task feature fusion network, coined Cognitive TransFuser, augments and exceeds the baseline network by a significant margin for safer and more complete road navigation. We validate the proposed network on the Town05 Short and Town05 Long Benchmark through extensive experiments, achieving up to 44.2 FPS real-time inference time.
  arXiv  Detail & Related papers  (2023-08-04T03:59:10Z)
• Multi-Exit Semantic Segmentation Networks [78.44441236864057]
  We propose a framework for converting state-of-the-art segmentation models to MESS networks. specially trained CNNs that employ parametrised early exits along their depth to save during inference on easier samples. We co-optimise the number, placement and architecture of the attached segmentation heads, along with the exit policy, to adapt to the device capabilities and application-specific requirements.
  arXiv  Detail & Related papers  (2021-06-07T11:37:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.