Visual Concept-Metaconcept Learning

• URL: http://arxiv.org/abs/2002.01464v1
• Date: Tue, 4 Feb 2020 18:42:30 GMT
• Title: Visual Concept-Metaconcept Learning
• Authors: Chi Han, Jiayuan Mao, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu
• Abstract summary: We propose the visual concept-metaconcept learner (VCML) for joint learning of concepts and metaconcepts from images and associated question-answer pairs. Knowing that red and green describe the same property of objects, we generalize to the fact that cube and sphere also describe the same property of objects.
• Score: 101.62725114966211
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans reason with concepts and metaconcepts: we recognize red and green from visual input; we also understand that they describe the same property of objects (i.e., the color). In this paper, we propose the visual concept-metaconcept learner (VCML) for joint learning of concepts and metaconcepts from images and associated question-answer pairs. The key is to exploit the bidirectional connection between visual concepts and metaconcepts. Visual representations provide grounding cues for predicting relations between unseen pairs of concepts. Knowing that red and green describe the same property of objects, we generalize to the fact that cube and sphere also describe the same property of objects, since they both categorize the shape of objects. Meanwhile, knowledge about metaconcepts empowers visual concept learning from limited, noisy, and even biased data. From just a few examples of purple cubes we can understand a new color purple, which resembles the hue of the cubes instead of the shape of them. Evaluation on both synthetic and real-world datasets validates our claims.

Related papers

• Pre-trained Vision-Language Models Learn Discoverable Visual Concepts [33.302556000017844]
  We aim to answer this question as visual concepts learned "for free" would enable wide applications. We assume that the visual concepts, if captured by pre-trained VLMs, can be extracted by their vision-language interface with text-based concept prompts. Our proposed concept discovery and learning framework is thus designed to identify a diverse list of generic visual concepts.
  arXiv  Detail & Related papers  (2024-04-19T06:41:32Z)
• Knowledge graphs for empirical concept retrieval [1.06378109904813]
  Concept-based explainable AI is promising as a tool to improve the understanding of complex models at the premises of a given user. Here, we present a workflow for user-driven data collection in both text and image domains. We test the retrieved concept datasets on two concept-based explainability methods, namely concept activation vectors (CAVs) and concept activation regions (CARs)
  arXiv  Detail & Related papers  (2024-04-10T13:47:22Z)
• Concept2Box: Joint Geometric Embeddings for Learning Two-View Knowledge Graphs [77.10299848546717]
  Concept2Box is a novel approach that jointly embeds the two views of a KG. Box embeddings learn the hierarchy structure and complex relations such as overlap and disjoint among them. We propose a novel vector-to-box distance metric and learn both embeddings jointly.
  arXiv  Detail & Related papers  (2023-07-04T21:37:39Z)
• ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models [79.10890337599166]
  We introduce ConceptBed, a large-scale dataset that consists of 284 unique visual concepts and 33K composite text prompts. We evaluate visual concepts that are either objects, attributes, or styles, and also evaluate four dimensions of compositionality: counting, attributes, relations, and actions. Our results point to a trade-off between learning the concepts and preserving the compositionality which existing approaches struggle to overcome.
  arXiv  Detail & Related papers  (2023-06-07T18:00:38Z)
• Intrinsic Physical Concepts Discovery with Object-Centric Predictive Models [86.25460882547581]
  We introduce the PHYsical Concepts Inference NEtwork (PHYCINE), a system that infers physical concepts in different abstract levels without supervision. We show that object representations containing the discovered physical concepts variables could help achieve better performance in causal reasoning tasks.
  arXiv  Detail & Related papers  (2023-03-03T11:52:21Z)
• FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relations [99.54048050189971]
  We present a framework for learning new visual concepts quickly, guided by multiple naturally occurring data streams. The learned concepts support downstream applications, such as answering questions by reasoning about unseen images. We demonstrate the effectiveness of our model on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-03-30T19:45:00Z)
• PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning [135.2892665079159]
  We introduce a new large-scale diagnostic visual reasoning dataset named PTR. PTR contains around 70k RGBD synthetic images with ground truth object and part level annotations. We examine several state-of-the-art visual reasoning models on this dataset and observe that they still make many surprising mistakes.
  arXiv  Detail & Related papers  (2021-12-09T18:59:34Z)
• Towards Visual Semantics [17.1623244298824]
  We study how humans build mental representations, i.e., concepts, of what they visually perceive. In this paper we provide a theory and an algorithm which learns substance concepts which correspond to the concepts, that we call classification concepts. The experiments, though preliminary, show that the algorithm manages to acquire the notions of Genus and Differentia with reasonable accuracy.
  arXiv  Detail & Related papers  (2021-04-26T07:28:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.