Implicit Modeling of Non-rigid Objects with Cross-Category Signals

• URL: http://arxiv.org/abs/2312.10246v1
• Date: Fri, 15 Dec 2023 22:34:17 GMT
• Title: Implicit Modeling of Non-rigid Objects with Cross-Category Signals
• Authors: Yuchun Liu, Benjamin Planche, Meng Zheng, Zhongpai Gao, Pierre Sibut-Bourde, Fan Yang, Terrence Chen, Ziyan Wu
• Abstract summary: MODIF is a multi-object deep implicit function that jointly learns the deformation fields and instance-specific latent codes for multiple objects at once. We show that MODIF can proficiently learn the shape representation of each organ and their relations to others, to the point that shapes missing from unseen instances can be consistently recovered.
• Score: 28.956412015920936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep implicit functions (DIFs) have emerged as a potent and articulate means of representing 3D shapes. However, methods modeling object categories or non-rigid entities have mainly focused on single-object scenarios. In this work, we propose MODIF, a multi-object deep implicit function that jointly learns the deformation fields and instance-specific latent codes for multiple objects at once. Our emphasis is on non-rigid, non-interpenetrating entities such as organs. To effectively capture the interrelation between these entities and ensure precise, collision-free representations, our approach facilitates signaling between category-specific fields to adequately rectify shapes. We also introduce novel inter-object supervision: an attraction-repulsion loss is formulated to refine contact regions between objects. Our approach is demonstrated on various medical benchmarks, involving modeling different groups of intricate anatomical entities. Experimental results illustrate that our model can proficiently learn the shape representation of each organ and their relations to others, to the point that shapes missing from unseen instances can be consistently recovered by our method. Finally, MODIF can also propagate semantic information throughout the population via accurate point correspondences

Related papers

• Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models [65.82564074712836]
  We introduce DIFfusionHOI, a new HOI detector shedding light on text-to-image diffusion models. We first devise an inversion-based strategy to learn the expression of relation patterns between humans and objects in embedding space. These learned relation embeddings then serve as textual prompts, to steer diffusion models generate images that depict specific interactions.
  arXiv  Detail & Related papers  (2024-10-26T12:00:33Z)
• pix2gestalt: Amodal Segmentation by Synthesizing Wholes [34.45464291259217]
  pix2gestalt is a framework for zero-shot amodal segmentation. We learn a conditional diffusion model for reconstructing whole objects in challenging zero-shot cases.
  arXiv  Detail & Related papers  (2024-01-25T18:57:36Z)
• DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field [29.42222066097076]
  Estimating 6D poses and reconstructing 3D shapes of objects in open-world scenes from RGB-depth image pairs is challenging. We propose the DTF-Net, a novel framework for pose estimation and shape reconstruction based on implicit neural fields of object categories.
  arXiv  Detail & Related papers  (2023-08-04T10:35:40Z)
• Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding [93.82542533426766]
  We propose a Suspected Object Transformation mechanism (SOT) to encourage the target object selection among the suspected ones. SOT can be seamlessly integrated into existing CNN and Transformer-based one-stage visual grounders. Extensive experiments demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2022-03-10T06:41:07Z)
• SPAMs: Structured Implicit Parametric Models [30.19414242608965]
  We learn Structured-implicit PArametric Models (SPAMs) as a deformable object representation that structurally decomposes non-rigid object motion into part-based disentangled representations of shape and pose. Experiments demonstrate that our part-aware shape and pose understanding lead to state-of-the-art performance in reconstruction and tracking of depth sequences of complex deforming object motion.
  arXiv  Detail & Related papers  (2022-01-20T12:33:46Z)
• NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go [109.88509362837475]
  We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes. NeuroMorph produces smooth and point-to-point correspondences between them. It works well for a large variety of input shapes, including non-isometric pairs from different object categories.
  arXiv  Detail & Related papers  (2021-06-17T12:25:44Z)
• A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation [62.517760545209065]
  We introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space. We demonstrate our model generalize well to out-of-distribution and unseen data, e.g., partial point clouds and real-world depth images.
  arXiv  Detail & Related papers  (2021-04-15T17:53:54Z)
• Continuous Surface Embeddings [76.86259029442624]
  We focus on the task of learning and representing dense correspondences in deformable object categories. We propose a new, learnable image-based representation of dense correspondences. We demonstrate that the proposed approach performs on par or better than the state-of-the-art methods for dense pose estimation for humans.
  arXiv  Detail & Related papers  (2020-11-24T22:52:15Z)
• Intrinsic Relationship Reasoning for Small Object Detection [44.68289739449486]
  Small objects in images and videos are usually not independent individuals. Instead, they more or less present some semantic and spatial layout relationships with each other. We propose a novel context reasoning approach for small object detection which models and infers the intrinsic semantic and spatial layout relationships between objects.
  arXiv  Detail & Related papers  (2020-09-02T06:03:05Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.