DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization

• URL: http://arxiv.org/abs/2502.04370v1
• Date: Wed, 05 Feb 2025 11:03:08 GMT
• Title: DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
• Authors: Zhenglin Zhou, Xiaobo Xia, Fan Ma, Hehe Fan, Yi Yang, Tat-Seng Chua,
• Abstract summary: We propose DreamDPO, an optimization-based framework that integrates human preferences into the 3D generation process. DreamDPO reduces reliance on precise pointwise quality evaluations while enabling fine-grained controllability. Experiments demonstrate that DreamDPO achieves competitive results, and provides higher-quality and more controllable 3D content.
• Score: 75.55167570591063
• License:
• Abstract: Text-to-3D generation automates 3D content creation from textual descriptions, which offers transformative potential across various fields. However, existing methods often struggle to align generated content with human preferences, limiting their applicability and flexibility. To address these limitations, in this paper, we propose DreamDPO, an optimization-based framework that integrates human preferences into the 3D generation process, through direct preference optimization. Practically, DreamDPO first constructs pairwise examples, then compare their alignment with human preferences using reward or large multimodal models, and lastly optimizes the 3D representation with a preference-driven loss function. By leveraging pairwise comparison to reflect preferences, DreamDPO reduces reliance on precise pointwise quality evaluations while enabling fine-grained controllability through preference-guided optimization. Experiments demonstrate that DreamDPO achieves competitive results, and provides higher-quality and more controllable 3D content compared to existing methods. The code and models will be open-sourced.

Related papers

• IPO: Iterative Preference Optimization for Text-to-Video Generation [15.763879468841818]
  We introduce an Iterative Preference Optimization strategy to enhance generated video quality by incorporating human feedback. IPO exploits a critic model to justify video generations for pairwise ranking as in Direct Preference Optimization or point-wise scoring. In addition, IPO incorporates the critic model with the multi-modality large language model, which enables it to automatically assign preference labels without need of retraining or relabeling.
  arXiv  Detail & Related papers  (2025-02-04T08:14:34Z)
• Personalized Preference Fine-tuning of Diffusion Models [75.22218338096316]
  We introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences. With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way. Our approach achieves an average win rate of 76% over Stable Cascade, generating images that more accurately reflect specific user preferences.
  arXiv  Detail & Related papers  (2025-01-11T22:38:41Z)
• VideoDPO: Omni-Preference Alignment for Video Diffusion Generation [48.36302380755874]
  Direct Preference Optimization (DPO) has demonstrated significant improvements in language and image generation. We propose a VideoDPO pipeline by making several key adjustments. Our experiments demonstrate substantial improvements in both visual quality and semantic alignment.
  arXiv  Detail & Related papers  (2024-12-18T18:59:49Z)
• Scalable Ranked Preference Optimization for Text-to-Image Generation [76.16285931871948]
  We investigate a scalable approach for collecting large-scale and fully synthetic datasets for DPO training. The preferences for paired images are generated using a pre-trained reward function, eliminating the need for involving humans in the annotation process. We introduce RankDPO to enhance DPO-based methods using the ranking feedback.
  arXiv  Detail & Related papers  (2024-10-23T16:42:56Z)
• mDPO: Conditional Preference Optimization for Multimodal Large Language Models [52.607764280030196]
  Direct preference optimization (DPO) has shown to be an effective method for large language model (LLM) alignment. Recent works have attempted to apply DPO to multimodal scenarios but have found it challenging to achieve consistent improvement. We propose mDPO, a multimodal DPO objective that prevents the over-prioritization of language-only preferences by also optimizing image preference.
  arXiv  Detail & Related papers  (2024-06-17T17:59:58Z)
• Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization [105.3612692153615]
  We propose a new axis based on eliciting preferences jointly over instruction-response pairs. Joint preferences over instruction and response pairs can significantly enhance the alignment of large language models.
  arXiv  Detail & Related papers  (2024-03-31T02:05:40Z)
• Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution [28.526714129927093]
  We propose DreamPortrait, which aims to generate text-guided 3D-aware portraits in a single-forward pass for efficiency. We further design a 3D-aware gated cross-attention mechanism to explicitly let the model perceive the correspondence between the text and the 3D-aware space.
  arXiv  Detail & Related papers  (2023-06-03T11:08:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.