Towards Affordance-Aware Articulation Synthesis for Rigged Objects

• URL: http://arxiv.org/abs/2501.12393v1
• Date: Tue, 21 Jan 2025 18:59:59 GMT
• Title: Towards Affordance-Aware Articulation Synthesis for Rigged Objects
• Authors: Yu-Chu Yu, Chieh Hubert Lin, Hsin-Ying Lee, Chaoyang Wang, Yu-Chiang Frank Wang, Ming-Hsuan Yang,
• Abstract summary: A3Syn synthesizes articulation parameters for arbitrary and open-domain rigged objects obtained from the Internet. A3Syn has stable convergence, completes in minutes, and synthesizes plausible affordance on different combinations of in-the-wild object rigs and scenes.
• Score: 82.08199697616917
• License:
• Abstract: Rigged objects are commonly used in artist pipelines, as they can flexibly adapt to different scenes and postures. However, articulating the rigs into realistic affordance-aware postures (e.g., following the context, respecting the physics and the personalities of the object) remains time-consuming and heavily relies on human labor from experienced artists. In this paper, we tackle the novel problem and design A3Syn. With a given context, such as the environment mesh and a text prompt of the desired posture, A3Syn synthesizes articulation parameters for arbitrary and open-domain rigged objects obtained from the Internet. The task is incredibly challenging due to the lack of training data, and we do not make any topological assumptions about the open-domain rigs. We propose using 2D inpainting diffusion model and several control techniques to synthesize in-context affordance information. Then, we develop an efficient bone correspondence alignment using a combination of differentiable rendering and semantic correspondence. A3Syn has stable convergence, completes in minutes, and synthesizes plausible affordance on different combinations of in-the-wild object rigs and scenes.

Related papers

• Drive-1-to-3: Enriching Diffusion Priors for Novel View Synthesis of Real Vehicles [81.29018359825872]
  This paper consolidates a set of good practices to finetune large pretrained models for a real-world task. Specifically, we develop several strategies to account for discrepancies between the synthetic data and real driving data. Our insights lead to effective finetuning that results in a $68.8%$ reduction in FID for novel view synthesis over prior arts.
  arXiv  Detail & Related papers  (2024-12-19T03:39:13Z)
• Mixed Diffusion for 3D Indoor Scene Synthesis [55.94569112629208]
  We present MiDiffusion, a novel mixed discrete-continuous diffusion model designed to synthesize plausible 3D indoor scenes. We show it outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis.
  arXiv  Detail & Related papers  (2024-05-31T17:54:52Z)
• PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI [38.03745740636854]
  PhyScene is a method dedicated to generating interactive 3D scenes characterized by realistic layouts, articulated objects, and rich physical interactivity tailored for embodied agents. We demonstrate that PhyScene effectively leverages these guidance functions for physically interactable scene synthesis, outperforming existing state-of-the-art scene synthesis methods by a large margin.
  arXiv  Detail & Related papers  (2024-04-15T05:29:23Z)
• Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Models [8.933560282929726]
  We introduce a novel affordance representation, named Comprehensive Affordance (ComA) Given a 3D object mesh, ComA models the distribution of relative orientation and proximity of vertices in interacting human meshes. We demonstrate that ComA outperforms competitors that rely on human annotations in modeling contact-based affordance.
  arXiv  Detail & Related papers  (2024-01-23T18:59:59Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)
• Towards 3D Scene Understanding by Referring Synthetic Models [65.74211112607315]
  Methods typically alleviate on-extensive annotations on real scene scans. We explore how synthetic models rely on real scene categories of synthetic features to a unified feature space. Experiments show that our method achieves the average mAP of 46.08% on the ScanNet S3DIS dataset and 55.49% by learning datasets.
  arXiv  Detail & Related papers  (2022-03-20T13:06:15Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.