Structure-Aware Multimodal LLM Framework for Trustworthy Near-Field Beam Prediction

• URL: http://arxiv.org/abs/2603.16143v1
• Date: Tue, 17 Mar 2026 05:52:13 GMT
• Title: Structure-Aware Multimodal LLM Framework for Trustworthy Near-Field Beam Prediction
• Abstract summary: In near-field systems, spherical wavefront propagation expands the traditional beam codebook into the joint angular-distance domain.<n>We propose a large language model (LLM)-driven multimodal framework that fuses historical GPS data, RGB image, LiDAR data, and strategically designed task-specific textual prompts.
• Score: 58.26409205915576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In near-field extremely large-scale multiple-input multiple-output (XL-MIMO) systems, spherical wavefront propagation expands the traditional beam codebook into the joint angular-distance domain, rendering conventional beam training prohibitively inefficient, especially in complex 3-dimensional (3D) low-altitude environments. Furthermore, since near-field beam variations are deeply coupled not only with user positions but also with the physical surroundings, precise beam alignment demands profound environmental understanding capabilities. To address this, we propose a large language model (LLM)-driven multimodal framework that fuses historical GPS data, RGB image, LiDAR data, and strategically designed task-specific textual prompts. By utilizing the powerful emergent reasoning and generalization capabilities of the LLM, our approach learns complex spatial dynamics to achieve superior environmental comprehension...

Related papers

• GeoLanG: Geometry-Aware Language-Guided Grasping with Unified RGB-D Multimodal Learning [17.389094942979455]
  GeoLanG is an end-to-end multi-task framework for language-guided grasping.<n>It unifies visual and linguistic inputs into a shared representation space for robust semantic alignment and improved generalization.<n>We show that GeoLanG enables precise and robust language-guided grasping in complex, cluttered environments.
  arXiv  Detail & Related papers  (2026-02-04T05:42:55Z)
• SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Models [62.14165748145729]
  We introduce SPUR, a lightweight, plug-in approach that equips large audio-speaker models with spatial perception.<n>SPUR consists of: (i) a First-Order Ambisonics (FOA) encoder that maps channels to rotation-aware, listener-centric spatial features, integrated into target LALMs via a multimodal adapter; and (ii) SPUR-Set, a spatial QA dataset combining open-source FOA recordings with controlled simulations, emphasizing relative direction, elevation, distance, and overlap for supervised spatial reasoning.
  arXiv  Detail & Related papers  (2025-11-10T01:29:26Z)
• OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment [79.98946571424607]
  We present OmniBridge, a unified framework that supports vision-language understanding, generation, and retrieval within a unified architecture.<n>To address the challenge of task interference, we propose a two-stage decoupled training strategy.<n>Experiments demonstrate that OmniBridge achieves competitive or state-of-the-art performance in all three tasks.
  arXiv  Detail & Related papers  (2025-09-23T13:57:55Z)
• SmolRGPT: Efficient Spatial Reasoning for Warehouse Environments with 600M Parameters [0.0]
  We present SmolRGPT, a compact vision-language architecture that explicitly incorporates region-level spatial reasoning.<n>SmolRGPT employs a three-stage curriculum that progressively align visual and language features, enables spatial relationship understanding, and adapts to task-specific datasets.<n>We demonstrate that with only 600M parameters, SmolRGPT achieves competitive results on challenging warehouse spatial reasoning benchmarks, matching or exceeding the performance of much larger alternatives.
  arXiv  Detail & Related papers  (2025-09-18T23:55:51Z)
• Spatial Knowledge Graph-Guided Multimodal Synthesis [78.11669780958657]
  We introduce a novel multimodal synthesis approach guided by spatial knowledge graphs, grounded in the concept of knowledge-to-data generation.<n>In experiments, data synthesized from diverse types of spatial knowledge, including direction and distance, enhance the spatial perception and reasoning abilities of MLLMs markedly.<n>We hope that the idea of knowledge-based data synthesis can advance the development of spatial intelligence.
  arXiv  Detail & Related papers  (2025-05-28T17:50:21Z)
• Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models [70.41727912081463]
  Multi-modal large language models (MLLMs) have rapidly advanced in visual tasks, yet their spatial understanding remains limited to single images.<n>We propose a framework to equip MLLMs with robust multi-frame spatial understanding by integrating depth perception, visual correspondence, and dynamic perception.<n>Our model, Multi-SpatialMLLM, achieves significant gains over baselines and proprietary systems, demonstrating scalable, generalizable multi-frame reasoning.
  arXiv  Detail & Related papers  (2025-05-22T17:59:39Z)
• EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting [46.44805092655782]
  EarthGPT-X is proposed, the first flexible spatial MLLM that unifies multi-source RS imagery comprehension.<n>It accomplishes both coarse-grained and fine-grained visual tasks under diverse visual prompts in a single framework.
  arXiv  Detail & Related papers  (2025-04-17T09:56:35Z)
• Advancing Fine-Grained Visual Understanding with Multi-Scale Alignment in Multi-Modal Models [11.151736352865921]
  We introduce a novel fine-grained visual knowledge alignment method. This method integrates multi-scale knowledge of objects, including texts, coordinates, and images. We also present TinyGroundingGPT, a series of compact models optimized for high-level alignments.
  arXiv  Detail & Related papers  (2024-11-14T18:57:07Z)
• Near-field Beam training for Extremely Large-scale MIMO Based on Deep Learning [20.67122533341949]
  We propose a near-field beam training method based on deep learning. We use a convolutional neural network (CNN) to efficiently learn channel characteristics from historical data. The proposed scheme achieves a more stable beamforming gain and significantly improves performance compared to the traditional beam training method.
  arXiv  Detail & Related papers  (2024-06-05T13:26:25Z)
• MuRF: Multi-Baseline Radiance Fields [117.55811938988256]
  We present Multi-Baseline Radiance Fields (MuRF), a feed-forward approach to solving sparse view synthesis. MuRF achieves state-of-the-art performance across multiple different baseline settings. We also show promising zero-shot generalization abilities on the Mip-NeRF 360 dataset.
  arXiv  Detail & Related papers  (2023-12-07T18:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.