Green Robotic Mixed Reality with Gaussian Splatting

• URL: http://arxiv.org/abs/2504.13697v1
• Date: Fri, 18 Apr 2025 13:57:28 GMT
• Title: Green Robotic Mixed Reality with Gaussian Splatting
• Authors: Chenxuan Liu, He Li, Zongze Li, Shuai Wang, Wei Xu, Kejiang Ye, Derrick Wing Kwan Ng, Chengzhong Xu,
• Abstract summary: Gaussian splatting (GS) RoboMR achieves a lower energy consumption and makes a concrete step towards green RoboMR.<n>The crux to GSRMR is to build a GS model which enables the simulator to opportunistically render a photo-realistic view from the robot's pose.<n> Experiments demonstrate that the proposed GSRMR reduces the communication energy by over 10x compared with RoboMR.
• Score: 61.04193266633848
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Realizing green communication in robotic mixed reality (RoboMR) systems presents a challenge, due to the necessity of uploading high-resolution images at high frequencies through wireless channels. This paper proposes Gaussian splatting (GS) RoboMR (GSRMR), which achieves a lower energy consumption and makes a concrete step towards green RoboMR. The crux to GSRMR is to build a GS model which enables the simulator to opportunistically render a photo-realistic view from the robot's pose, thereby reducing the need for excessive image uploads. Since the GS model may involve discrepancies compared to the actual environments, a GS cross-layer optimization (GSCLO) framework is further proposed, which jointly optimizes content switching (i.e., deciding whether to upload image or not) and power allocation across different frames. The GSCLO problem is solved by an accelerated penalty optimization (APO) algorithm. Experiments demonstrate that the proposed GSRMR reduces the communication energy by over 10x compared with RoboMR. Furthermore, the proposed GSRMR with APO outperforms extensive baseline schemes, in terms of peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

Related papers

• Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution [10.074968164380314]
  Implicit Neural Representation (INR) has been successfully employed for Arbitrary-scale Super-Resolution (ASR)<n>We develop two novel techniques to generalize GS for ASR.<n>We implement an efficient differentiable 2D GPU/CUDA-based scale-awareization to render super-aware images.
  arXiv  Detail & Related papers  (2025-01-12T15:14:58Z)
• Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution [49.902047563260496]
  We develop the first attempt to integrate the Vision State Space Model (Mamba) for remote sensing image (RSI) super-resolution. To achieve better SR reconstruction, building upon Mamba, we devise a Frequency-assisted Mamba framework, dubbed FMSR. Our FMSR features a multi-level fusion architecture equipped with the Frequency Selection Module (FSM), Vision State Space Module (VSSM), and Hybrid Gate Module (HGM)
  arXiv  Detail & Related papers  (2024-05-08T11:09:24Z)
• Fortifying Fully Convolutional Generative Adversarial Networks for Image Super-Resolution Using Divergence Measures [17.517010701323823]
  Super-Resolution (SR) is a time-hallowed image processing problem. We introduce a fully-conversaal Generative Adrial Network (GAN)-based architecture for SR. We show that distinct convolutional features obtained at increasing depths of a GAN generator can be optimally combined by a set of learnable convex weights.
  arXiv  Detail & Related papers  (2024-04-09T13:19:43Z)
• Uni-COAL: A Unified Framework for Cross-Modality Synthesis and Super-Resolution of MR Images [34.79786909094346]
  Cross-modality synthesis (CMS), super-resolution (SR), and their combination (CMSR) have been extensively studied for magnetic resonance imaging (MRI) None of the previous works is capable of performing CMS, SR, and CMSR using a unified network. We propose a Unified Co-Modulated Alias-free framework (Uni-COAL) to accomplish the aforementioned tasks with a single network.
  arXiv  Detail & Related papers  (2023-11-14T15:05:59Z)
• RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution [57.98314517861539]
  Burst super-resolution (BurstSR) aims at reconstructing a high-resolution (HR) image from a sequence of low-resolution (LR) and noisy images. In this paper, we suggest fusing cues frame-by-frame with an efficient and flexible recurrent network.
  arXiv  Detail & Related papers  (2023-06-30T12:14:13Z)
• DCS-RISR: Dynamic Channel Splitting for Efficient Real-world Image Super-Resolution [15.694407977871341]
  Real-world image super-resolution (RISR) has received increased focus for improving the quality of SR images under unknown complex degradation. Existing methods rely on the heavy SR models to enhance low-resolution (LR) images of different degradation levels. We propose a novel Dynamic Channel Splitting scheme for efficient Real-world Image Super-Resolution, termed DCS-RISR.
  arXiv  Detail & Related papers  (2022-12-15T04:34:57Z)
• Context-Aware Video Reconstruction for Rolling Shutter Cameras [52.28710992548282]
  In this paper, we propose a context-aware GS video reconstruction architecture. We first estimate the bilateral motion field so that the pixels of the two RS frames are warped to a common GS frame. Then, a refinement scheme is proposed to guide the GS frame synthesis along with bilateral occlusion masks to produce high-fidelity GS video frames.
  arXiv  Detail & Related papers  (2022-05-25T17:05:47Z)
• Learning Multiple Probabilistic Degradation Generators for Unsupervised Real World Image Super Resolution [5.987801889633082]
  Unsupervised real world super resolution aims at restoring high-resolution (HR) images given low-resolution (LR) inputs when paired data is unavailable. One of the most common approaches is synthesizing noisy LR images using GANs and utilizing a synthetic dataset to train the model in a supervised manner. We propose a probabilistic degradation generator to approximate the distribution of LR images given a HR image.
  arXiv  Detail & Related papers  (2022-01-26T04:49:11Z)
• Unsupervised Single Image Super-resolution Under Complex Noise [60.566471567837574]
  This paper proposes a model-based unsupervised SISR method to deal with the general SISR task with unknown degradations. The proposed method can evidently surpass the current state of the art (SotA) method (about 1dB PSNR) not only with a slighter model (0.34M vs. 2.40M) but also faster speed.
  arXiv  Detail & Related papers  (2021-07-02T11:55:40Z)
• Blur, Noise, and Compression Robust Generative Adversarial Networks [85.68632778835253]
  We propose blur, noise, and compression robust GAN (BNCR-GAN) to learn a clean image generator directly from degraded images. Inspired by NR-GAN, BNCR-GAN uses a multiple-generator model composed of image, blur- Kernel, noise, and quality-factor generators. We demonstrate the effectiveness of BNCR-GAN through large-scale comparative studies on CIFAR-10 and a generality analysis on FFHQ.
  arXiv  Detail & Related papers  (2020-03-17T17:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.