Towards a Modern and Lightweight Rendering Engine for Dynamic Robotic Simulations

• URL: http://arxiv.org/abs/2410.05095v1
• Date: Mon, 7 Oct 2024 14:50:19 GMT
• Title: Towards a Modern and Lightweight Rendering Engine for Dynamic Robotic Simulations
• Authors: Christopher John Allison, Haoying Zhou, Adnan Munawar, Peter Kazanzides, Juan Antonio Barragan,
• Abstract summary: This paper presents a performance-focused and lightweight rendering engine supporting the Vulkan graphics API. The engine is designed to modernize the legacy rendering pipeline of Asynchronous Multi-Body Framework (AMBF) Experiments show that the engine can render a simulated scene with over seven million triangles while maintaining GPU computation times within two milliseconds.
• Score: 4.226502078427161
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interactive dynamic simulators are an accelerator for developing novel robotic control algorithms and complex systems involving humans and robots. In user training and synthetic data generation applications, a high-fidelity visualization of the simulation is essential. Visual fidelity is dependent on the quality of the computer graphics algorithms used to render the simulated scene. Furthermore, the rendering algorithms must be implemented on the graphics processing unit (GPU) to achieve real-time performance, requiring the use of a graphics application programming interface (API). This paper presents a performance-focused and lightweight rendering engine supporting the Vulkan graphics API. The engine is designed to modernize the legacy rendering pipeline of Asynchronous Multi-Body Framework (AMBF), a dynamic simulation framework used extensively for interactive robotics simulation development. This new rendering engine implements graphical features such as physically based rendering (PBR), anti-aliasing, and ray-traced shadows, significantly improving the image quality of AMBF. Computational experiments show that the engine can render a simulated scene with over seven million triangles while maintaining GPU computation times within two milliseconds.

Related papers

• Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigation [62.5805866419814]
  Vid2Sim is a novel framework that bridges the sim2real gap through a scalable and cost-efficient real2sim pipeline for neural 3D scene reconstruction and simulation. Experiments demonstrate that Vid2Sim significantly improves the performance of urban navigation in the digital twins and real world by 31.2% and 68.3% in success rate.
  arXiv  Detail & Related papers  (2025-01-12T03:01:15Z)
• ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI [27.00155119759743]
  ManiSkill3 is the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation. ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more.
  arXiv  Detail & Related papers  (2024-10-01T06:10:39Z)
• FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces [21.946327323788275]
  3D rendering of dynamic face is a challenging problem. We present a novel representation that enables high-quality rendering of an actor's dynamic facial performances.
  arXiv  Detail & Related papers  (2024-04-22T00:44:13Z)
• EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction [53.28220984270622]
  3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time. Our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets. Our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40 FPS (Frames Per Second)
  arXiv  Detail & Related papers  (2023-11-16T11:30:56Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Learning from synthetic data generated with GRADE [0.6982738885923204]
  We present a framework for generating realistic animated dynamic environments (GRADE) for robotics research. GRADE supports full simulation control, ROS integration, realistic physics, while being in an engine that produces high visual fidelity images and ground truth data. We show that, even training using only synthetic data, can generalize well to real-world images in the same application domain.
  arXiv  Detail & Related papers  (2023-05-07T14:13:04Z)
• Large Batch Simulation for Deep Reinforcement Learning [101.01408262583378]
  We accelerate deep reinforcement learning-based training in visually complex 3D environments by two orders of magnitude over prior work. We realize end-to-end training speeds of over 19,000 frames of experience per second on a single and up to 72,000 frames per second on a single eight- GPU machine. By combining batch simulation and performance optimizations, we demonstrate that Point navigation agents can be trained in complex 3D environments on a single GPU in 1.5 days to 97% of the accuracy of agents trained on a prior state-of-the-art system.
  arXiv  Detail & Related papers  (2021-03-12T00:22:50Z)
• GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition [81.24107630746508]
  We present GeoSim, a geometry-aware image composition process that synthesizes novel urban driving scenes. We first build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. The resulting synthetic images are photorealistic, traffic-aware, and geometrically consistent, allowing image simulation to scale to complex use cases.
  arXiv  Detail & Related papers  (2021-01-16T23:00:33Z)
• Photorealism in Driving Simulations: Blending Generative Adversarial Image Synthesis with Rendering [0.0]
  We introduce a hybrid generative neural graphics pipeline for improving the visual fidelity of driving simulations. We form 2D semantic images from 3D scenery consisting of simple object models without textures. These semantic images are then converted into photorealistic RGB images with a state-of-the-art Generative Adrial Network (GAN) trained on real-world driving scenes.
  arXiv  Detail & Related papers  (2020-07-31T03:25:17Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.