The joint role of geometry and illumination on material recognition
- URL: http://arxiv.org/abs/2101.02496v2
- Date: Thu, 4 Feb 2021 12:35:25 GMT
- Title: The joint role of geometry and illumination on material recognition
- Authors: Manuel Lagunas, Ana Serrano, Diego Gutierrez, Belen Masia
- Abstract summary: We study how the interplay of geometry, illumination, and their spatial frequencies affects human performance on material recognition tasks.
We train a deep neural network on material recognition tasks to accurately classify materials.
- Score: 16.01513204879645
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Observing and recognizing materials is a fundamental part of our daily life.
Under typical viewing conditions, we are capable of effortlessly identifying
the objects that surround us and recognizing the materials they are made of.
Nevertheless, understanding the underlying perceptual processes that take place
to accurately discern the visual properties of an object is a long-standing
problem. In this work, we perform a comprehensive and systematic analysis of
how the interplay of geometry, illumination, and their spatial frequencies
affects human performance on material recognition tasks. We carry out
large-scale behavioral experiments where participants are asked to recognize
different reference materials among a pool of candidate samples. In the
different experiments, we carefully sample the information in the frequency
domain of the stimuli. From our analysis, we find significant first-order
interactions between the geometry and the illumination, of both the reference
and the candidates. In addition, we observe that simple image statistics and
higher-order image histograms do not correlate with human performance.
Therefore, we perform a high-level comparison of highly non-linear statistics
by training a deep neural network on material recognition tasks. Our results
show that such models can accurately classify materials, which suggests that
they are capable of defining a meaningful representation of material appearance
from labeled proximal image data. Last, we find preliminary evidence that these
highly non-linear models and humans may use similar high-level factors for
material recognition tasks.
Related papers
- Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance [2.2284248642626103]
This paper introduces a novel approach to material identification by encoding perceptual features obtained from dynamic visual stimuli.
We conducted a psychophysical experiment to select and validate 16 particularly significant perceptual attributes obtained from videos of 347 materials.
We then gathered attribute ratings from over twenty participants for each material, creating a'material fingerprint' that encodes the unique perceptual properties of each material.
arXiv Detail & Related papers (2024-10-17T14:47:53Z) - AI-based Density Recognition [7.106165417217771]
This paper introduces an AI-based concept for assigning physical properties to objects through the use of associated images.
We derive specific patterns from 2D images using a neural network to extract further information such as volume, material, or density.
arXiv Detail & Related papers (2024-07-24T07:45:37Z) - Semantic-Based Active Perception for Humanoid Visual Tasks with Foveal Sensors [49.99728312519117]
The aim of this work is to establish how accurately a recent semantic-based active perception model is able to complete visual tasks that are regularly performed by humans.
This model exploits the ability of current object detectors to localize and classify a large number of object classes and to update a semantic description of a scene across multiple fixations.
In the task of scene exploration, the semantic-based method demonstrates superior performance compared to the traditional saliency-based model.
arXiv Detail & Related papers (2024-04-16T18:15:57Z) - Intrinsic Image Diffusion for Indoor Single-view Material Estimation [55.276815106443976]
We present Intrinsic Image Diffusion, a generative model for appearance decomposition of indoor scenes.
Given a single input view, we sample multiple possible material explanations represented as albedo, roughness, and metallic maps.
Our method produces significantly sharper, more consistent, and more detailed materials, outperforming state-of-the-art methods by $1.5dB$ on PSNR and by $45%$ better FID score on albedo prediction.
arXiv Detail & Related papers (2023-12-19T15:56:19Z) - The Influences of Color and Shape Features in Visual Contrastive
Learning [0.0]
This paper investigates the influences of individual image features (e.g., color and shape) to model performance remain ambiguous.
Experimental results show that compared with supervised representations, contrastive representations tend to cluster with objects of similar color.
arXiv Detail & Related papers (2023-01-29T15:10:14Z) - One-shot recognition of any material anywhere using contrastive learning
with physics-based rendering [0.0]
We present MatSim: a synthetic dataset, a benchmark, and a method for computer vision based recognition of similarities and transitions between materials and textures.
The visual recognition of materials is essential to everything from examining food while cooking to inspecting agriculture, chemistry, and industrial products.
arXiv Detail & Related papers (2022-12-01T16:49:53Z) - A Symbolic Representation of Human Posture for Interpretable Learning
and Reasoning [2.678461526933908]
We introduce a qualitative spatial reasoning approach that describes the human posture in terms that are more familiar to people.
This paper explores the derivation of our symbolic representation at two levels of detail and its preliminary use as features for interpretable activity recognition.
arXiv Detail & Related papers (2022-10-17T12:22:13Z) - ComPhy: Compositional Physical Reasoning of Objects and Events from
Videos [113.2646904729092]
The compositionality between the visible and hidden properties poses unique challenges for AI models to reason from the physical world.
Existing studies on video reasoning mainly focus on visually observable elements such as object appearance, movement, and contact interaction.
We propose an oracle neural-symbolic framework named Compositional Physics Learner (CPL), combining visual perception, physical property learning, dynamic prediction, and symbolic execution.
arXiv Detail & Related papers (2022-05-02T17:59:13Z) - Exploring Visual Engagement Signals for Representation Learning [56.962033268934015]
We present VisE, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals.
We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection.
arXiv Detail & Related papers (2021-04-15T20:50:40Z) - Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units
and a Unified Framework [83.21732533130846]
The paper focuses on large in-the-wild databases, i.e., Aff-Wild and Aff-Wild2.
It presents the design of two classes of deep neural networks trained with these databases.
A novel multi-task and holistic framework is presented which is able to jointly learn and effectively generalize and perform affect recognition.
arXiv Detail & Related papers (2021-03-29T17:36:20Z) - What Can You Learn from Your Muscles? Learning Visual Representation
from Human Interactions [50.435861435121915]
We use human interaction and attention cues to investigate whether we can learn better representations compared to visual-only representations.
Our experiments show that our "muscly-supervised" representation outperforms a visual-only state-of-the-art method MoCo.
arXiv Detail & Related papers (2020-10-16T17:46:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.