AI-Powered Augmented Reality for Satellite Assembly, Integration and Test

• URL: http://arxiv.org/abs/2409.18101v1
• Date: Thu, 26 Sep 2024 17:44:52 GMT
• Title: AI-Powered Augmented Reality for Satellite Assembly, Integration and Test
• Abstract summary: This paper presents a technical description of the European Space Agency's (ESA) project "AI for AR in Satellite AIT" The project combines real-time computer vision and AR systems to assist technicians during satellite assembly. All AI models demonstrated over 70% accuracy, with the detection model exceeding 95% accuracy, indicating a high level of performance and reliability.
• Score: 2.5069344340760713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of Artificial Intelligence (AI) and Augmented Reality (AR) is set to transform satellite Assembly, Integration, and Testing (AIT) processes by enhancing precision, minimizing human error, and improving operational efficiency in cleanroom environments. This paper presents a technical description of the European Space Agency's (ESA) project "AI for AR in Satellite AIT," which combines real-time computer vision and AR systems to assist technicians during satellite assembly. Leveraging Microsoft HoloLens 2 as the AR interface, the system delivers context-aware instructions and real-time feedback, tackling the complexities of object recognition and 6D pose estimation in AIT workflows. All AI models demonstrated over 70% accuracy, with the detection model exceeding 95% accuracy, indicating a high level of performance and reliability. A key contribution of this work lies in the effective use of synthetic data for training AI models in AR applications, addressing the significant challenges of obtaining real-world datasets in highly dynamic satellite environments, as well as the creation of the Segmented Anything Model for Automatic Labelling (SAMAL), which facilitates the automatic annotation of real data, achieving speeds up to 20 times faster than manual human annotation. The findings demonstrate the efficacy of AI-driven AR systems in automating critical satellite assembly tasks, setting a foundation for future innovations in the space industry.

Related papers

• RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills [90.48218237655708]
  RoboSynChallenge aims to foster the development of broadly capable, data-efficient, and adaptable manipulation systems.<n>Challenge integrates large-scale synthetic data generation with standardized real-world robotic evaluation.
  arXiv  Detail & Related papers  (2026-08-12T04:29:00Z)
• A Self-Calibrating Agentic AI Framework for Autonomous Edge Resource Allocation [2.131323683158522]
  Large Language Models (LLMs) are increasingly deployed as autonomous agents, transitioning from static conversational interfaces to dynamic systems capable of complex reasoning, tool execution, and decision-making.<n>We propose and experimentally evaluate an agentic AI framework, designed to enforce autonomous integrity within LLM-driven systems.<n>We design a self-calibration mechanism that mitigates drift and dynamically approximates ground truth by incorporating an ARIMA forecaster, without requiring continuous human oversight.
  arXiv  Detail & Related papers  (2026-07-24T15:21:54Z)
• EmboCoach-Bench: Benchmarking AI Agents on Developing Embodied Robots [68.29056647487519]
  Embodied AI is fueled by high-fidelity simulation and large-scale data collection.<n>However, this scaling capability remains bottlenecked by a reliance on labor-intensive manual oversight.<n>We introduce textscEmboCoach-Bench, a benchmark evaluating the capacity of LLM agents to autonomously engineer embodied policies.
  arXiv  Detail & Related papers  (2026-01-29T11:33:49Z)
• Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methods [5.278929538141005]
  We benchmark a range of domain randomization (DR) and domain adaptation (DA) techniques, including feature-based methods, generative AI (GenAI) and classical rendering approaches.<n>Our evaluation focuses on the effectiveness and efficiency of low-level and high-level feature alignment, as well as a controlled diffusion-based DA method guided by prompts generated from real-world contexts.<n>Results show that if render-based data with enough variability is available as seed, simpler feature-based methods, such as brightness-based and perceptual hashing filtering, outperform more complex GenAI-based approaches in both accuracy and resource efficiency
  arXiv  Detail & Related papers  (2025-11-28T14:51:08Z)
• AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarios [64.51320327698231]
  We introduce AerialMind, the first large-scale RMOT benchmark in UAV scenarios.<n>We develop an innovative semi-automated collaborative agent-based labeling assistant framework.<n>We also propose HawkEyeTrack, a novel method that collaboratively enhances vision-language representation learning.
  arXiv  Detail & Related papers  (2025-11-26T04:44:27Z)
• Accurate and Efficient World Modeling with Masked Latent Transformers [58.0159270859475]
  We introduce EMERALD, a world model using a spatial latent state with MaskGIT predictions to generate accurate trajectories in latent space.<n>On the Crafter benchmark, EMERALD achieves new state-of-the-art performance, becoming the first method to surpass human experts performance within 10M environment steps.
  arXiv  Detail & Related papers  (2025-07-05T15:49:21Z)
• FieldWorkArena: Agentic AI Benchmark for Real Field Work Tasks [52.47895046206854]
  FieldWorkArena is a benchmark for agentic AI targeting real-world field work.<n>This paper defines a new action space that agentic AI should possess for real world work environment benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T08:21:46Z)
• Satellite Edge Artificial Intelligence with Large Models: Architectures and Technologies [43.24943201324192]
  Extreme weather nowcasting, disaster monitoring, and battlefield surveillance require realtime data processing. Satellite edge large AI model (LAM) involves both the training (i.e., finetuning) and inference phases. LAM deployment in resource-constrained space networks with time-varying topologies.
  arXiv  Detail & Related papers  (2025-04-02T12:25:57Z)
• You Only Crash Once v2: Perceptually Consistent Strong Features for One-Stage Domain Adaptive Detection of Space Terrain [4.339510167603377]
  In-situ detection of planetary, lunar, and small-body surface terrain is crucial for autonomous spacecraft applications. Unsupervised Domain Adaptation (UDA) offers a promising solution by facilitating model training with disparate data sources. We propose novel additions to the VSA scheme that enhance terrain detection capabilities under UDA.
  arXiv  Detail & Related papers  (2025-01-23T14:58:49Z)
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [52.46737975742287]
  We build a self-contained environment with data that mimics a small software company environment. We find that with the most competitive agent, 24% of the tasks can be completed autonomously. This paints a nuanced picture on task automation with LM agents.
  arXiv  Detail & Related papers  (2024-12-18T18:55:40Z)
• Enhancing Object Detection Accuracy in Autonomous Vehicles Using Synthetic Data [0.8267034114134277]
  Performance of machine learning models depends on the nature and size of the training data sets. High-quality, diverse, relevant and representative training data is essential to build accurate and reliable machine learning models. It is hypothesised that well-designed synthetic data can improve the performance of a machine learning algorithm.
  arXiv  Detail & Related papers  (2024-11-23T16:38:02Z)
• On-Air Deep Learning Integrated Semantic Inference Models for Enhanced Earth Observation Satellite Networks [28.69148416385582]
  Domain-adapted Large Language Models (LLMs) provide a solution by enabling the integration of raw and processed EO data. This study provides a thorough examination of using semantic inference and deep learning for sophisticated EO systems. It presents an innovative architecture for semantic communication in EO satellite networks, designed to improve data transmission efficiency.
  arXiv  Detail & Related papers  (2024-09-23T17:42:05Z)
• AIvril: AI-Driven RTL Generation With Verification In-The-Loop [0.7831852829409273]
  Large Language Models (LLMs) are computational models capable of performing complex natural language processing tasks. This paper introduces AIvril, a framework designed to enhance the accuracy and reliability of RTL-aware LLMs.
  arXiv  Detail & Related papers  (2024-09-03T15:07:11Z)
• ArK: Augmented Reality with Knowledge Interactive Emergent Ability [115.72679420999535]
  We develop an infinite agent that learns to transfer knowledge memory from general foundation models to novel domains. The heart of our approach is an emerging mechanism, dubbed Augmented Reality with Knowledge Inference Interaction (ArK) We show that our ArK approach, combined with large foundation models, significantly improves the quality of generated 2D/3D scenes.
  arXiv  Detail & Related papers  (2023-05-01T17:57:01Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• LaMAR: Benchmarking Localization and Mapping for Augmented Reality [80.23361950062302]
  We introduce LaMAR, a new benchmark with a comprehensive capture and GT pipeline that co-registers realistic trajectories and sensor streams captured by heterogeneous AR devices. We publish a benchmark dataset of diverse and large-scale scenes recorded with head-mounted and hand-held AR devices.
  arXiv  Detail & Related papers  (2022-10-19T17:58:17Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Autonomous Aerial Robot for High-Speed Search and Intercept Applications [86.72321289033562]
  A fully-autonomous aerial robot for high-speed object grasping has been proposed. As an additional sub-task, our system is able to autonomously pierce balloons located in poles close to the surface. Our approach has been validated in a challenging international competition and has shown outstanding results.
  arXiv  Detail & Related papers  (2021-12-10T11:49:51Z)
• Rethinking Drone-Based Search and Rescue with Aerial Person Detection [79.76669658740902]
  The visual inspection of aerial drone footage is an integral part of land search and rescue (SAR) operations today. We propose a novel deep learning algorithm to automate this aerial person detection (APD) task. We present the novel Aerial Inspection RetinaNet (AIR) algorithm as the combination of these contributions.
  arXiv  Detail & Related papers  (2021-11-17T21:48:31Z)
• A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks [4.04540578484476]
  Intelligence Task Ontology and Knowledge Graph (ITO) is a comprehensive resource on artificial intelligence tasks, benchmark results and performance metrics. ITO is a richly structured and manually curated resource on artificial intelligence tasks, benchmark results and performance metrics. The goal of ITO is to enable precise and network-based analyses of the global landscape of AI tasks and capabilities.
  arXiv  Detail & Related papers  (2021-10-04T13:25:53Z)
• An Efficient UAV-based Artificial Intelligence Framework for Real-Time Visual Tasks [33.489573797811474]
  We introduce a multi-layer AI (MLAI) framework to allow easy integration of ad-hoc visual-based AI applications. To show its features and its advantages, we implemented and evaluated different modern visual-based deep learning models for object detection, target tracking and target handover.
  arXiv  Detail & Related papers  (2020-04-13T18:53:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.