A Self-Supervised Robotic System for Autonomous Contact-Based Spatial Mapping of Semiconductor Properties

• URL: http://arxiv.org/abs/2411.09892v2
• Date: Sun, 29 Dec 2024 20:13:04 GMT
• Title: A Self-Supervised Robotic System for Autonomous Contact-Based Spatial Mapping of Semiconductor Properties
• Abstract summary: We build self-supervised autonomy into contact-based robotic systems that teach the robot to follow domain expert measurement principles.<n>We demonstrate the performance of this approach by autonomously driving a 4-degree-of-freedom robotic probe for 24 hours to characterize semiconductor photoconductivity.
• Score: 40.361306070887366
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Integrating robotically driven contact-based material characterization techniques into self-driving laboratories can enhance measurement quality, reliability, and throughput. While deep learning models support robust autonomy, current methods lack reliable pixel-precision positioning and require extensive labeled data. To overcome these challenges, we propose an approach for building self-supervised autonomy into contact-based robotic systems that teach the robot to follow domain expert measurement principles at high-throughputs. Firstly, we design a vision-based, self-supervised convolutional neural network (CNN) architecture that uses differentiable image priors to optimize domain-specific objectives, refining the pixel precision of predicted robot contact poses by 20.0% relative to existing approaches. Secondly, we design a reliable graph-based planner for generating distance-minimizing paths to accelerate the robot measurement throughput and decrease planning variance by 6x. We demonstrate the performance of this approach by autonomously driving a 4-degree-of-freedom robotic probe for 24 hours to characterize semiconductor photoconductivity at 3,025 uniquely predicted poses across a gradient of drop-casted perovskite film compositions, achieving throughputs over 125 measurements per hour. Spatially mapping photoconductivity onto each drop-casted film reveals compositional trends and regions of inhomogeneity, valuable for identifying manufacturing process defects. With this self-supervised CNN-driven robotic system, we enable high-precision and reliable automation of contact-based characterization techniques at high throughputs, thereby allowing the measurement of previously inaccessible yet important semiconductor properties for self-driving laboratories.

Related papers

• RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction [51.76737991200891]
  Human-to-robot (H2R) object handover is a fundamental capability for human-robot collaboration, yet progress is hindered by the scarcity of large-scale, human-centric datasets.<n>We introduce Hand2Bot, an RGB-D video dataset that provides rich contextual information such as body posture and facial expressions.<n>We propose PassGen, a generative pipeline that synthesizes realistic handover sequences while ensuring hand-object consistency.
  arXiv  Detail & Related papers  (2026-08-13T09:55:59Z)
• Fast Human Attention Prediction for Fixation-guided Active Perception in Autonomous Navigation [10.364779390403337]
  We introduce GazeLNN, a lightweight,scanpath prediction model.<n> operating auto-regressively, the architecture predicts sequential fixation heatmaps.<n>It achieves state-of-the-art performance on the MIT Low Resolution dataset.
  arXiv  Detail & Related papers  (2026-06-18T17:08:06Z)
• Shape Control of a Planar Hyper-Redundant Robot via Hybrid Kinematics-Informed and Learning-based Approach [42.55947704321053]
  Hyper-redundant robots offer high dexterity, making them good at operating in confined and unstructured environments.<n>To extend the reachable workspace, we built a multi-segment flexible rack actuated planar robot.<n>We propose a hybrid kinematics-informed and learning-based shape control method, named SpatioCoupledNet.
  arXiv  Detail & Related papers  (2026-03-11T04:35:06Z)
• Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery [64.82869118243723]
  We present a framework that integrates EEG-based visual and motor imagery (VI/MI) with robotic control to enable real-time, intention-driven grasping and placement.<n>Motivated by the promise of BCI-driven robotics, this system bridges neural signals with physical control by deploying offline-pretrained decoders in a zero-shot manner.
  arXiv  Detail & Related papers  (2026-03-03T17:41:42Z)
• Physics-Informed Machine Learning for Efficient Sim-to-Real Data Augmentation in Micro-Object Pose Estimation [6.975054201075641]
  This work proposes a novel physics-informed deep generative learning framework for pose estimation of microrobots.<n>It integrates wave optics-based physical rendering and depth alignment into a generative adversarial network (GAN)<n>Our method improves the structural similarity index (SSIM) by 35.6% compared to purely AI-driven methods.<n>It achieves 93.9%/91.9% (pitch/roll) accuracy, just 5.0%/5.4% (pitch/roll) below that of an estimator trained exclusively on real data.
  arXiv  Detail & Related papers  (2025-11-20T16:10:53Z)
• Efficient Surgical Robotic Instrument Pose Reconstruction in Real World Conditions Using Unified Feature Detection [21.460727996614704]
  MIS robots have long kinematic chains and partial visibility of their degrees of freedom in the camera.<n>We propose a novel framework that unifies the detection of geometric primitives through a shared encoding.<n>This architecture detects both keypoints and edges in a single inference and is trained on large-scale synthetic data with projective labeling.
  arXiv  Detail & Related papers  (2025-10-03T22:03:28Z)
• Generating and Customizing Robotic Arm Trajectories using Neural Networks [0.0]
  We introduce a neural network approach for generating and customizing the trajectory of a robotic arm.<n>Our approach successfully generates precise trajectories that could be customized in their shape and adapted to different settings.
  arXiv  Detail & Related papers  (2025-06-25T09:05:58Z)
• Advancing from Automated to Autonomous Beamline by Leveraging Computer Vision [16.747469612768917]
  Current state-of-the-art synchrotron beamlines still heavily rely on human safety oversight.<n>A computer vision-based system is proposed, integrating deep learning and multiview cameras for real-time collision detection.<n> Experiments on a real beamline dataset demonstrate high accuracy, real-time performance, and strong potential for autonomous synchrotron beamline operations.
  arXiv  Detail & Related papers  (2025-06-01T04:53:55Z)
• Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation [50.34179054785646]
  We present Taccel, a high-performance simulation platform that integrates IPC and ABD to model robots, tactile sensors, and objects with both accuracy and unprecedented speed. Taccel provides precise physics simulation and realistic tactile signals while supporting flexible robot-sensor configurations through user-friendly APIs. These capabilities position Taccel as a powerful tool for scaling up tactile robotics research and development.
  arXiv  Detail & Related papers  (2025-04-17T12:57:11Z)
• A Real-time Anomaly Detection Method for Robots based on a Flexible and Sparse Latent Space [2.0186752447895993]
  Deep learning-based models in robotics face challenges due to limited training data and highly noisy signal features. We present Sparse Masked Autoregressive Flow-based Adversarial AutoEncoders model to address these problems. Our model performs inferences within 1 millisecond, ensuring real-time anomaly detection.
  arXiv  Detail & Related papers  (2025-04-15T13:17:14Z)
• ARC-Calib: Autonomous Markerless Camera-to-Robot Calibration via Exploratory Robot Motions [15.004750210002152]
  ARC-Calib is a model-based markerless camera-to-robot calibration framework. It is fully autonomous and generalizable across diverse robots.
  arXiv  Detail & Related papers  (2025-03-18T20:03:32Z)
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics [68.36528819227641]
  This paper systematically evaluates the robustness of Vision-Language-Action (VLA) models.<n>We introduce two untargeted attack objectives that leverage spatial foundations to destabilize robotic actions, and a targeted attack objective that manipulates the robotic trajectory.<n>We design an adversarial patch generation approach that places a small, colorful patch within the camera's view, effectively executing the attack in both digital and physical environments.
  arXiv  Detail & Related papers  (2024-11-18T01:52:20Z)
• Benchmarking Adaptive Intelligence and Computer Vision on Human-Robot Collaboration [0.0]
  Human-Robot Collaboration (HRC) is vital in Industry 4.0, using sensors, digital twins, collaborative robots (cobots) and intention-recognition models to have efficient manufacturing processes. We address concept drift by integrating Adaptive Intelligence and self-labeling to improve the resilience of intention-recognition in an HRC system.
  arXiv  Detail & Related papers  (2024-09-30T01:25:48Z)
• GISR: Geometric Initialization and Silhouette-based Refinement for Single-View Robot Pose and Configuration Estimation [0.0]
  GISR is a robot-to-camera pose estimation method that prioritizes execution in real-time. We evaluate GISR on publicly available data and show that it outperforms existing methods of the same class in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2024-05-08T08:39:25Z)
• Enhancing Apparent Personality Trait Analysis with Cross-Modal Embeddings [0.5461938536945723]
  We present a multimodal deep neural network with a Siamese extension for apparent personality trait prediction trained on short video recordings. Due to the highly centralized target distribution of the analyzed dataset, the changes in the third digit are relevant.
  arXiv  Detail & Related papers  (2024-05-06T20:51:28Z)
• Learning-based adaption of robotic friction models [48.453527255659296]
  We introduce a novel approach to adapt an existing friction model to new dynamics using as little data as possible. Our proposed estimator outperforms the conventional model-based approach and the base neural network significantly. Our method does not rely on data with external load during training, eliminating the need for external torque sensors.
  arXiv  Detail & Related papers  (2023-10-25T14:50:15Z)
• Intelligent machines work in unstructured environments by differential neuromorphic computing [20.05765768946624]
  We present a memristor-based differential neuromorphic computing, perceptual signal processing and learning method for intelligent machines. The developed method takes advantage of the intrinsic multi-state property of memristors and exhibits good scalability and generalization. By mimicking the intrinsic nature of human low-level perception mechanisms, the electronic memristive neuromorphic circuit-based method, presented here shows the potential for adapting to diverse sensing technologies.
  arXiv  Detail & Related papers  (2023-09-16T01:45:13Z)
• Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks [1.9809266426888898]
  Mixed-signal neuromorphic systems represent a promising solution for solving extreme-edge computing tasks. Their spiking neural network circuits are optimized for processing sensory data on-line in continuous-time. We design on-chip learning circuits with short-term analog dynamics and long-term tristate discretization mechanisms.
  arXiv  Detail & Related papers  (2023-07-12T11:14:25Z)
• Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics [11.946807588018595]
  This paper presents a unified model-based reinforcement learning framework that bridges active exploration and uncertainty-aware deployment. The two opposing tasks of exploration and deployment are optimized through state-of-the-art sampling-based MPC. We conduct experiments on both autonomous vehicles and wheeled robots, showing promising results for both exploration and deployment.
  arXiv  Detail & Related papers  (2023-05-20T17:20:12Z)
• Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing [88.99939660183881]
  We propose a framework for autonomous robotic navigation for subretinal injection. Our method consists of an instrument pose estimation method, an online registration between the robotic and the i OCT system, and trajectory planning tailored for navigation to an injection target. Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method.
  arXiv  Detail & Related papers  (2023-01-17T21:41:21Z)
• A Spatio-Temporal Multilayer Perceptron for Gesture Recognition [70.34489104710366]
  We propose a multilayer state-weighted perceptron for gesture recognition in the context of autonomous vehicles. An evaluation of TCG and Drive&Act datasets is provided to showcase the promising performance of our approach. We deploy our model to our autonomous vehicle to show its real-time capability and stable execution.
  arXiv  Detail & Related papers  (2022-04-25T08:42:47Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)
• Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition [54.749127627191655]
  The ability to recognize human partners is an important social skill to build personalized and long-term human-robot interactions. Deep learning networks have achieved state-of-the-art results and demonstrated to be suitable tools to address such a task. One solution is to make robots learn from their first-hand sensory data with self-supervision.
  arXiv  Detail & Related papers  (2021-03-16T13:50:24Z)
• Where is my hand? Deep hand segmentation for visual self-recognition in humanoid robots [129.46920552019247]
  We propose the use of a Convolution Neural Network (CNN) to segment the robot hand from an image in an egocentric view. We fine-tuned the Mask-RCNN network for the specific task of segmenting the hand of the humanoid robot Vizzy.
  arXiv  Detail & Related papers  (2021-02-09T10:34:32Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)
• Intrinsic Motivation and Episodic Memories for Robot Exploration of High-Dimensional Sensory Spaces [0.0]
  This work presents an architecture that generates curiosity-driven goal-directed exploration behaviours for an image sensor of a microfarming robot. A combination of deep neural networks for offline unsupervised learning of low-dimensional features from images, and of online learning of shallow neural networks representing the inverse and forward kinematics of the system have been used. The artificial curiosity system assigns interest values to a set of pre-defined goals, and drives the exploration towards those that are expected to maximise the learning progress.
  arXiv  Detail & Related papers  (2020-01-07T11:39:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.