Independent Density Estimation

• URL: http://arxiv.org/abs/2512.10067v1
• Date: Wed, 10 Dec 2025 20:43:03 GMT
• Title: Independent Density Estimation
• Authors: Jiahao Liu,
• Abstract summary: We propose a new method called Independent Density Estimation (IDE) to tackle this challenge.<n>IDE aims to learn the connec- tion between individual words in a sentence and the corresponding features in an image, enabling compositional generalization.<n>Our models exhibit superior generalization to unseen compositions compared to current models when evaluated on various datasets.
• Score: 27.51041148291178
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale Vision-Language models have achieved remarkable results in various domains, such as image captioning and conditioned image generation. Neverthe- less, these models still encounter difficulties in achieving human-like composi- tional generalization. In this study, we propose a new method called Independent Density Estimation (IDE) to tackle this challenge. IDE aims to learn the connec- tion between individual words in a sentence and the corresponding features in an image, enabling compositional generalization. We build two models based on the philosophy of IDE. The first one utilizes fully disentangled visual representations as input, and the second leverages a Variational Auto-Encoder to obtain partially disentangled features from raw images. Additionally, we propose an entropy- based compositional inference method to combine predictions of each word in the sentence. Our models exhibit superior generalization to unseen compositions compared to current models when evaluated on various datasets.

Related papers

• Evaluating the encoding competence of visual language models using uncommon actions [5.816389980109022]
  UAIT is a new evaluation benchmark designed to test the semantic understanding ability of visual language models (VLMs) in uncommon-sense action scenes.<n>We synthesize high-quality uncommon-sense image-text samples using large language models, few-shot prompt engineering, and text-to-image generation.<n>We evaluate multiple state-of-the-art visual language models and compare them with models based on contrastive learning.
  arXiv  Detail & Related papers  (2026-01-12T17:15:45Z)
• Towards Generalized Multi-Image Editing for Unified Multimodal Models [56.620038824933566]
  Unified Multimodal Models (UMMs) integrate multimodal understanding and generation.<n>UMMs are limited to maintaining visual consistency and disambiguating visual cues when referencing details across multiple input images.<n>We propose a scalable multi-image editing framework for UMMs that explicitly distinguishes image identities and generalizes to variable input counts.
  arXiv  Detail & Related papers  (2026-01-09T06:42:49Z)
• GRADE: Quantifying Sample Diversity in Text-to-Image Models [66.12068246962762]
  GRADE is an automatic method for quantifying sample diversity in text-to-image models.<n>We use GRADE to measure the diversity of 12 models over a total of 720K images.
  arXiv  Detail & Related papers  (2024-10-29T23:10:28Z)
• Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models [65.82564074712836]
  We introduce DIFfusionHOI, a new HOI detector shedding light on text-to-image diffusion models. We first devise an inversion-based strategy to learn the expression of relation patterns between humans and objects in embedding space. These learned relation embeddings then serve as textual prompts, to steer diffusion models generate images that depict specific interactions.
  arXiv  Detail & Related papers  (2024-10-26T12:00:33Z)
• Translatotron-V(ison): An End-to-End Model for In-Image Machine Translation [81.45400849638347]
  In-image machine translation (IIMT) aims to translate an image containing texts in source language into an image containing translations in target language. In this paper, we propose an end-to-end IIMT model consisting of four modules. Our model achieves competitive performance compared to cascaded models with only 70.9% of parameters, and significantly outperforms the pixel-level end-to-end IIMT model.
  arXiv  Detail & Related papers  (2024-07-03T08:15:39Z)
• Semantic Approach to Quantifying the Consistency of Diffusion Model Image Generation [0.40792653193642503]
  We identify the need for an interpretable, quantitative score of the repeatability, or consistency, of image generation in diffusion models. We propose a semantic approach, using a pairwise mean CLIP score as our semantic consistency score.
  arXiv  Detail & Related papers  (2024-04-12T20:16:03Z)
• A Unified Understanding of Deep NLP Models for Text Classification [88.35418976241057]
  We have developed a visual analysis tool, DeepNLPVis, to enable a unified understanding of NLP models for text classification. The key idea is a mutual information-based measure, which provides quantitative explanations on how each layer of a model maintains the information of input words in a sample. A multi-level visualization, which consists of a corpus-level, a sample-level, and a word-level visualization, supports the analysis from the overall training set to individual samples.
  arXiv  Detail & Related papers  (2022-06-19T08:55:07Z)
• Compositional Visual Generation with Composable Diffusion Models [80.75258849913574]
  We propose an alternative structured approach for compositional generation using diffusion models. An image is generated by composing a set of diffusion models, with each of them modeling a certain component of the image. The proposed method can generate scenes at test time that are substantially more complex than those seen in training.
  arXiv  Detail & Related papers  (2022-06-03T17:47:04Z)
• Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets [56.018551958004814]
  This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources. Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision. We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
  arXiv  Detail & Related papers  (2021-11-24T19:00:05Z)
• Image Captioning with Compositional Neural Module Networks [18.27510863075184]
  We introduce a hierarchical framework for image captioning that explores both compositionality and sequentiality of natural language. Our algorithm learns to compose a detail-rich sentence by selectively attending to different modules corresponding to unique aspects of each object detected in an input image.
  arXiv  Detail & Related papers  (2020-07-10T20:58:04Z)
• Using Human Psychophysics to Evaluate Generalization in Scene Text Recognition Models [7.294729862905325]
  We characterize two important scene text recognition models by measuring their domains. The domains specifies the ability of readers to generalize to different word lengths, fonts, and amounts of occlusion.
  arXiv  Detail & Related papers  (2020-06-30T19:51:26Z)
• A Study of Compositional Generalization in Neural Models [22.66002315559978]
  We introduce ConceptWorld, which enables the generation of images from compositional and relational concepts. We perform experiments to test the ability of standard neural networks to generalize on relations with compositional arguments. For simple problems, all models generalize well to close concepts but struggle with longer compositional chains.
  arXiv  Detail & Related papers  (2020-06-16T18:29:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.