Preserving Product Fidelity in Large Scale Image Recontextualization with Diffusion Models

• URL: http://arxiv.org/abs/2503.08729v1
• Date: Tue, 11 Mar 2025 01:24:39 GMT
• Title: Preserving Product Fidelity in Large Scale Image Recontextualization with Diffusion Models
• Authors: Ishaan Malhi, Praneet Dutta, Ellie Talius, Sally Ma, Brendan Driscoll, Krista Holden, Garima Pruthi, Arunachalam Narayanaswamy,
• Abstract summary: We present a framework for high-fidelity product image recontextualization using text-to-image diffusion models and a novel data augmentation pipeline.<n>Our method improves the quality and diversity of generated images by disentangling product representations and enhancing the model's understanding of product characteristics.
• Score: 1.8606057023042066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a framework for high-fidelity product image recontextualization using text-to-image diffusion models and a novel data augmentation pipeline. This pipeline leverages image-to-video diffusion, in/outpainting & negatives to create synthetic training data, addressing limitations of real-world data collection for this task. Our method improves the quality and diversity of generated images by disentangling product representations and enhancing the model's understanding of product characteristics. Evaluation on the ABO dataset and a private product dataset, using automated metrics and human assessment, demonstrates the effectiveness of our framework in generating realistic and compelling product visualizations, with implications for applications such as e-commerce and virtual product showcasing.

Related papers

• Your Image Generator Is Your New Private Dataset [4.09225917049674]
  Generative diffusion models have emerged as powerful tools to synthetically produce training data. This paper proposes the Text-Conditioned Knowledge Recycling pipeline to tackle these challenges. The pipeline is rigorously evaluated on ten diverse image classification benchmarks.
  arXiv  Detail & Related papers  (2025-04-06T18:46:08Z)
• Augmented Conditioning Is Enough For Effective Training Image Generation [11.60839452103417]
  We find that conditioning the generation process on an augmented real image and text prompt produces generations that serve as effective synthetic datasets for downstream training.<n>We validate augmentation-conditioning on a total of five established long-tail and few-shot image classification benchmarks.
  arXiv  Detail & Related papers  (2025-02-06T19:57:33Z)
• Visual Autoregressive Modeling for Image Super-Resolution [14.935662351654601]
  We propose a novel visual autoregressive modeling for ISR framework with the form of next-scale prediction.<n>We collect large-scale data and design a training process to obtain robust generative priors.
  arXiv  Detail & Related papers  (2025-01-31T09:53:47Z)
• Dataset Augmentation by Mixing Visual Concepts [3.5420134832331334]
  This paper proposes a dataset augmentation method by fine-tuning pre-trained diffusion models.<n>We adapt the diffusion model by conditioning it with real images and novel text embeddings.<n>Our approach outperforms state-of-the-art augmentation techniques on benchmark classification tasks.
  arXiv  Detail & Related papers  (2024-12-19T19:42:22Z)
• AssemAI: Interpretable Image-Based Anomaly Detection for Manufacturing Pipelines [0.0]
  Anomaly detection in manufacturing pipelines remains a critical challenge, intensified by the complexity and variability of industrial environments. This paper introduces AssemAI, an interpretable image-based anomaly detection system tailored for smart manufacturing pipelines.
  arXiv  Detail & Related papers  (2024-08-05T01:50:09Z)
• A Simple Background Augmentation Method for Object Detection with Diffusion Model [53.32935683257045]
  In computer vision, it is well-known that a lack of data diversity will impair model performance. We propose a simple yet effective data augmentation approach by leveraging advancements in generative models. Background augmentation, in particular, significantly improves the models' robustness and generalization capabilities.
  arXiv  Detail & Related papers  (2024-08-01T07:40:00Z)
• YaART: Yet Another ART Rendering Technology [119.09155882164573]
  This study introduces YaART, a novel production-grade text-to-image cascaded diffusion model aligned to human preferences. We analyze how these choices affect both the efficiency of the training process and the quality of the generated images. We demonstrate that models trained on smaller datasets of higher-quality images can successfully compete with those trained on larger datasets.
  arXiv  Detail & Related papers  (2024-04-08T16:51:19Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• On quantifying and improving realism of images generated with diffusion [50.37578424163951]
  We propose a metric, called Image Realism Score (IRS), computed from five statistical measures of a given image. IRS is easily usable as a measure to classify a given image as real or fake. We experimentally establish the model- and data-agnostic nature of the proposed IRS by successfully detecting fake images generated by Stable Diffusion Model (SDM), Dalle2, Midjourney and BigGAN. Our efforts have also led to Gen-100 dataset, which provides 1,000 samples for 100 classes generated by four high-quality models.
  arXiv  Detail & Related papers  (2023-09-26T08:32:55Z)
• StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data [129.92449761766025]
  We propose a novel data collection methodology that synchronously synthesizes images and dialogues for visual instruction tuning. This approach harnesses the power of generative models, marrying the abilities of ChatGPT and text-to-image generative models. Our research includes comprehensive experiments conducted on various datasets.
  arXiv  Detail & Related papers  (2023-08-20T12:43:52Z)
• UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning [86.91893533388628]
  This paper presents UniDiff, a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) UniDiff demonstrates versatility in both multi-modal understanding and generative tasks.
  arXiv  Detail & Related papers  (2023-06-01T15:39:38Z)
• Generative Adversarial Transformers [13.633811200719627]
  We introduce the GANsformer, a novel and efficient type of transformer, and explore it for the task of visual generative modeling. The network employs a bipartite structure that enables long-range interactions across the image, while maintaining computation of linearly efficiency. We show it achieves state-of-the-art results in terms of image quality and diversity, while enjoying fast learning and better data-efficiency.
  arXiv  Detail & Related papers  (2021-03-01T18:54:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.