Generative AI for Vision: A Comprehensive Study of Frameworks and Applications

• URL: http://arxiv.org/abs/2501.18033v1
• Date: Wed, 29 Jan 2025 22:42:05 GMT
• Title: Generative AI for Vision: A Comprehensive Study of Frameworks and Applications
• Authors: Fouad Bousetouane,
• Abstract summary: Generative AI is transforming image synthesis, enabling the creation of high-quality, diverse, and photorealistic visuals. This work presents a structured classification of image generation techniques based on the nature of the input. We highlight key frameworks including DALL-E, ControlNet, and DeepSeek Janus-Pro, and address challenges such as computational costs, data biases, and output alignment with user intent.
• Score: 0.0
• License:
• Abstract: Generative AI is transforming image synthesis, enabling the creation of high-quality, diverse, and photorealistic visuals across industries like design, media, healthcare, and autonomous systems. Advances in techniques such as image-to-image translation, text-to-image generation, domain transfer, and multimodal alignment have broadened the scope of automated visual content creation, supporting a wide spectrum of applications. These advancements are driven by models like Generative Adversarial Networks (GANs), conditional frameworks, and diffusion-based approaches such as Stable Diffusion. This work presents a structured classification of image generation techniques based on the nature of the input, organizing methods by input modalities like noisy vectors, latent representations, and conditional inputs. We explore the principles behind these models, highlight key frameworks including DALL-E, ControlNet, and DeepSeek Janus-Pro, and address challenges such as computational costs, data biases, and output alignment with user intent. By offering this input-centric perspective, this study bridges technical depth with practical insights, providing researchers and practitioners with a comprehensive resource to harness generative AI for real-world applications.

Related papers

• A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationships [0.5639904484784127]
  Transformer-based models have transformed the landscape of natural language processing (NLP) These models are renowned for their ability to capture long-range dependencies and contextual information. We discuss potential research directions and applications of transformer-based models in computer vision.
  arXiv  Detail & Related papers  (2024-08-27T16:22:18Z)
• Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation [87.50120181861362]
  VisionPrefer is a high-quality and fine-grained preference dataset that captures multiple preference aspects. We train a reward model VP-Score over VisionPrefer to guide the training of text-to-image generative models and the preference prediction accuracy of VP-Score is comparable to human annotators.
  arXiv  Detail & Related papers  (2024-04-23T14:53:15Z)
• Generative AI in Vision: A Survey on Models, Metrics and Applications [0.0]
  Generative AI models have revolutionized various fields by enabling the creation of realistic and diverse data samples. Among these models, diffusion models have emerged as a powerful approach for generating high-quality images, text, and audio. This survey paper provides a comprehensive overview of generative AI diffusion and legacy models, focusing on their underlying techniques, applications across different domains, and their challenges.
  arXiv  Detail & Related papers  (2024-02-26T07:47:12Z)
• An Interactive Agent Foundation Model [49.77861810045509]
  We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents. Our training paradigm unifies diverse pre-training strategies, including visual masked auto-encoders, language modeling, and next-action prediction. We demonstrate the performance of our framework across three separate domains -- Robotics, Gaming AI, and Healthcare.
  arXiv  Detail & Related papers  (2024-02-08T18:58:02Z)
• State of the Art on Diffusion Models for Visual Computing [191.6168813012954]
  This report introduces the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model. We also give a comprehensive overview of the rapidly growing literature on diffusion-based generation and editing. We discuss available datasets, metrics, open challenges, and social implications.
  arXiv  Detail & Related papers  (2023-10-11T05:32:29Z)
• Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models [55.04969603431266]
  This paper proposes a method for generating images of customized objects specified by users. The method is based on a general framework that bypasses the lengthy optimization required by previous approaches. We demonstrate through experiments that our proposed method is able to synthesize images with compelling output quality, appearance diversity, and object fidelity.
  arXiv  Detail & Related papers  (2023-04-05T17:59:32Z)
• Deep Image Synthesis from Intuitive User Input: A Review and Perspectives [23.01321275304037]
  It is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph or layout, and have a computer system automatically generate photo-realistic images that adhere to the input content. Recent advances in deep generative models such as generative adversarial networks (GANs), variational autoencoders (VAEs), and flow-based methods have enabled more powerful and versatile image generation tasks. This paper reviews recent works for image synthesis given intuitive user input, covering advances in input versatility, image generation methodology, benchmark datasets, and evaluation metrics.
  arXiv  Detail & Related papers  (2021-07-09T06:31:47Z)
• Generating Annotated High-Fidelity Images Containing Multiple Coherent Objects [10.783993190686132]
  We propose a multi-object generation framework that can synthesize images with multiple objects without explicitly requiring contextual information. We demonstrate how coherency and fidelity are preserved with our method through experiments on the Multi-MNIST and CLEVR datasets.
  arXiv  Detail & Related papers  (2020-06-22T11:33:55Z)
• Image Segmentation Using Deep Learning: A Survey [58.37211170954998]
  Image segmentation is a key topic in image processing and computer vision. There has been a substantial amount of works aimed at developing image segmentation approaches using deep learning models.
  arXiv  Detail & Related papers  (2020-01-15T21:37:47Z)
• Towards Coding for Human and Machine Vision: A Scalable Image Coding Approach [104.02201472370801]
  We come up with a novel image coding framework by leveraging both the compressive and the generative models. By introducing advanced generative models, we train a flexible network to reconstruct images from compact feature representations and the reference pixels. Experimental results demonstrate the superiority of our framework in both human visual quality and facial landmark detection.
  arXiv  Detail & Related papers  (2020-01-09T10:37:17Z)
• Contextual Encoder-Decoder Network for Visual Saliency Prediction [42.047816176307066]
  We propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. We combine the resulting representations with global scene information for accurately predicting visual saliency. Compared to state of the art approaches, the network is based on a lightweight image classification backbone.
  arXiv  Detail & Related papers  (2019-02-18T16:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.