CGI: Identifying Conditional Generative Models with Example Images

• URL: http://arxiv.org/abs/2501.13991v1
• Date: Thu, 23 Jan 2025 09:31:06 GMT
• Title: CGI: Identifying Conditional Generative Models with Example Images
• Authors: Zhi Zhou, Hao-Zhe Tan, Peng-Xiao Song, Lan-Zhe Guo,
• Abstract summary: Generative models have achieved remarkable performance recently, and thus model hubs have emerged. It is not easy for users to review model descriptions and example images, choosing which model best meets their needs. We propose Generative Model Identification (CGI), which aims to identify the most suitable model using user-provided example images.
• Score: 14.453885742032481
• License:
• Abstract: Generative models have achieved remarkable performance recently, and thus model hubs have emerged. Existing model hubs typically assume basic text matching is sufficient to search for models. However, in reality, due to different abstractions and the large number of models in model hubs, it is not easy for users to review model descriptions and example images, choosing which model best meets their needs. Therefore, it is necessary to describe model functionality wisely so that future users can efficiently search for the most suitable model for their needs. Efforts to address this issue remain limited. In this paper, we propose Conditional Generative Model Identification (CGI), which aims to provide an effective way to identify the most suitable model using user-provided example images rather than requiring users to manually review a large number of models with example images. To address this problem, we propose the PromptBased Model Identification (PMI) , which can adequately describe model functionality and precisely match requirements with specifications. To evaluate PMI approach and promote related research, we provide a benchmark comprising 65 models and 9100 identification tasks. Extensive experimental and human evaluation results demonstrate that PMI is effective. For instance, 92% of models are correctly identified with significantly better FID scores when four example images are provided.

Related papers

• Model Provenance Testing for Large Language Models [14.949325775620439]
  We develop a framework for testing model provenance: Whether one model is derived from another. Our approach is based on the key observation that real-world model derivations preserve significant similarities in model outputs. Using only black-box access to models, we employ multiple hypothesis testing to compare model similarities against a baseline established by unrelated models.
  arXiv  Detail & Related papers  (2025-02-02T07:39:37Z)
• You Only Submit One Image to Find the Most Suitable Generative Model [48.67303250592189]
  We propose a novel setting called Generative Model Identification (GMI) GMI aims to enable the user to identify the most appropriate generative model(s) for the user's requirements efficiently.
  arXiv  Detail & Related papers  (2024-12-16T14:46:57Z)
• Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models [51.067146460271466]
  Evaluation of visual generative models can be time-consuming and computationally expensive. We propose the Evaluation Agent framework, which employs human-like strategies for efficient, dynamic, multi-round evaluations. It offers four key advantages: 1) efficiency, 2) promptable evaluation tailored to diverse user needs, 3) explainability beyond single numerical scores, and 4) scalability across various models and tools.
  arXiv  Detail & Related papers  (2024-12-10T18:52:39Z)
• Enabling Small Models for Zero-Shot Selection and Reuse through Model Label Learning [50.68074833512999]
  We introduce a novel paradigm, Model Label Learning (MLL), which bridges the gap between models and their functionalities. Experiments on seven real-world datasets validate the effectiveness and efficiency of MLL.
  arXiv  Detail & Related papers  (2024-08-21T09:08:26Z)
• Revealing the Underlying Patterns: Investigating Dataset Similarity, Performance, and Generalization [0.0]
  Supervised deep learning models require significant amount of labeled data to achieve an acceptable performance on a specific task. We establish image-image, dataset-dataset, and image-dataset distances to gain insights into the model's behavior.
  arXiv  Detail & Related papers  (2023-08-07T13:35:53Z)
• Earning Extra Performance from Restrictive Feedbacks [41.05874087063763]
  We set up a challenge named emphEarning eXtra PerformancE from restriCTive feEDdbacks (EXPECTED) to describe this form of model tuning problems. The goal of the model provider is to eventually deliver a satisfactory model to the local user(s) by utilizing the feedbacks. We propose to characterize the geometry of the model performance with regard to model parameters through exploring the parameters' distribution.
  arXiv  Detail & Related papers  (2023-04-28T13:16:54Z)
• PAMI: partition input and aggregate outputs for model interpretation [69.42924964776766]
  In this study, a simple yet effective visualization framework called PAMI is proposed based on the observation that deep learning models often aggregate features from local regions for model predictions. The basic idea is to mask majority of the input and use the corresponding model output as the relative contribution of the preserved input part to the original model prediction. Extensive experiments on multiple tasks confirm the proposed method performs better than existing visualization approaches in more precisely finding class-specific input regions.
  arXiv  Detail & Related papers  (2023-02-07T08:48:34Z)
• Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
  Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. This creates a barrier to fusing knowledge across individual models to yield a better single model. We propose a dataless knowledge fusion method that merges models in their parameter space.
  arXiv  Detail & Related papers  (2022-12-19T20:46:43Z)
• Content-Based Search for Deep Generative Models [45.322081206025544]
  We introduce the task of content-based model search: given a query and a large set of generative models, finding the models that best match the query. As each generative model produces a distribution of images, we formulate the search task as an optimization problem to select the model with the highest probability of generating similar content as the query. We demonstrate that our method outperforms several baselines on Generative Model Zoo, a new benchmark we create for the model retrieval task.
  arXiv  Detail & Related papers  (2022-10-06T17:59:51Z)
• Incorporating Vision Bias into Click Models for Image-oriented Search Engine [51.192784793764176]
  In this paper, we assume that vision bias exists in an image-oriented search engine as another crucial factor affecting the examination probability aside from position. We use regression-based EM algorithm to predict the vision bias given the visual features extracted from candidate documents.
  arXiv  Detail & Related papers  (2021-01-07T10:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.