UGG: Unified Generative Grasping
- URL: http://arxiv.org/abs/2311.16917v1
- Date: Tue, 28 Nov 2023 16:20:33 GMT
- Title: UGG: Unified Generative Grasping
- Authors: Jiaxin Lu, Hao Kang, Haoxiang Li, Bo Liu, Yiding Yang, Qixing Huang,
Gang Hua
- Abstract summary: Generation-based methods that generate grasping postures conditioned on the object can often produce diverse grasping.
We introduce a unified diffusion-based dexterous grasp generation model, dubbed the name UGG.
Our model achieves state-of-the-art dexterous grasping on the large-scale DexGraspNet dataset.
- Score: 43.26740792207497
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Dexterous grasping aims to produce diverse grasping postures with a high
grasping success rate. Regression-based methods that directly predict grasping
parameters given the object may achieve a high success rate but often lack
diversity. Generation-based methods that generate grasping postures conditioned
on the object can often produce diverse grasping, but they are insufficient for
high grasping success due to lack of discriminative information. To mitigate,
we introduce a unified diffusion-based dexterous grasp generation model, dubbed
the name UGG, which operates within the object point cloud and hand parameter
spaces. Our all-transformer architecture unifies the information from the
object, the hand, and the contacts, introducing a novel representation of
contact points for improved contact modeling. The flexibility and quality of
our model enable the integration of a lightweight discriminator, benefiting
from simulated discriminative data, which pushes for a high success rate while
preserving high diversity. Beyond grasp generation, our model can also generate
objects based on hand information, offering valuable insights into object
design and studying how the generative model perceives objects. Our model
achieves state-of-the-art dexterous grasping on the large-scale DexGraspNet
dataset while facilitating human-centric object design, marking a significant
advancement in dexterous grasping research. Our project page is
https://jiaxin-lu.github.io/ugg/ .
Related papers
- Regularized Training with Generated Datasets for Name-Only Transfer of Vision-Language Models [36.59260354292177]
Recent advancements in text-to-image generation have inspired researchers to generate datasets tailored for perception models using generative models.
We aim to fine-tune vision-language models to a specific classification model without access to any real images.
Despite the high fidelity of generated images, we observed a significant performance degradation when fine-tuning the model using the generated datasets.
arXiv Detail & Related papers (2024-06-08T10:43:49Z) - Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
We develop an efficient, autoregression-based vision model on a limited dataset.
We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding.
Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
arXiv Detail & Related papers (2024-02-07T13:41:53Z) - Segue: Side-information Guided Generative Unlearnable Examples for
Facial Privacy Protection in Real World [64.4289385463226]
We propose Segue: Side-information guided generative unlearnable examples.
To improve transferability, we introduce side information such as true labels and pseudo labels.
It can resist JPEG compression, adversarial training, and some standard data augmentations.
arXiv Detail & Related papers (2023-10-24T06:22:37Z) - Progressive reduced order modeling: empowering data-driven modeling with
selective knowledge transfer [0.0]
We propose a progressive reduced order modeling framework that minimizes data cravings and enhances data-driven modeling's practicality.
Our approach selectively transfers knowledge from previously trained models through gates, similar to how humans selectively use valuable knowledge while ignoring unuseful information.
We have tested our framework in several cases, including transport in porous media, gravity-driven flow, and finite deformation in hyperelastic materials.
arXiv Detail & Related papers (2023-10-04T23:50:14Z) - Diversity vs. Recognizability: Human-like generalization in one-shot
generative models [5.964436882344729]
We propose a new framework to evaluate one-shot generative models along two axes: sample recognizability vs. diversity.
We first show that GAN-like and VAE-like models fall on opposite ends of the diversity-recognizability space.
In contrast, disentanglement transports the model along a parabolic curve that could be used to maximize recognizability.
arXiv Detail & Related papers (2022-05-20T13:17:08Z) - StyleGAN-Human: A Data-Centric Odyssey of Human Generation [96.7080874757475]
This work takes a data-centric perspective and investigates multiple critical aspects in "data engineering"
We collect and annotate a large-scale human image dataset with over 230K samples capturing diverse poses and textures.
We rigorously investigate three essential factors in data engineering for StyleGAN-based human generation, namely data size, data distribution, and data alignment.
arXiv Detail & Related papers (2022-04-25T17:55:08Z) - InvGAN: Invertible GANs [88.58338626299837]
InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.
This allows us to perform image inpainting, merging, and online data augmentation.
arXiv Detail & Related papers (2021-12-08T21:39:00Z) - REGRAD: A Large-Scale Relational Grasp Dataset for Safe and
Object-Specific Robotic Grasping in Clutter [52.117388513480435]
We present a new dataset named regrad to sustain the modeling of relationships among objects and grasps.
Our dataset is collected in both forms of 2D images and 3D point clouds.
Users are free to import their own object models for the generation of as many data as they want.
arXiv Detail & Related papers (2021-04-29T05:31:21Z) - Graph-based Normalizing Flow for Human Motion Generation and
Reconstruction [20.454140530081183]
We propose a probabilistic generative model to synthesize and reconstruct long horizon motion sequences conditioned on past information and control signals.
We evaluate the models on a mixture of motion capture datasets of human locomotion with foot-step and bone-length analysis.
arXiv Detail & Related papers (2021-04-07T09:51:15Z) - Attribute-Based Robotic Grasping with One-Grasp Adaptation [9.255994599301712]
We introduce an end-to-end learning method of attribute-based robotic grasping with one-grasp adaptation capability.
Our approach fuses the embeddings of a workspace image and a query text using a gated-attention mechanism and learns to predict instance grasping affordances.
Experimental results in both simulation and the real world demonstrate that our approach achieves over 80% instance grasping success rate on unknown objects.
arXiv Detail & Related papers (2021-04-06T03:40:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.