Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance

• URL: http://arxiv.org/abs/2410.13615v1
• Date: Thu, 17 Oct 2024 14:47:53 GMT
• Title: Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance
• Authors: Jiri Filip, Filip Dechterenko, Filipp Schmidt, Jiri Lukavsky, Veronika Vilimovska, Jan Kotera, Roland W. Fleming,
• Abstract summary: This paper introduces a novel approach to material identification by encoding perceptual features obtained from dynamic visual stimuli. We conducted a psychophysical experiment to select and validate 16 particularly significant perceptual attributes obtained from videos of 347 materials. We then gathered attribute ratings from over twenty participants for each material, creating a'material fingerprint' that encodes the unique perceptual properties of each material.
• Score: 2.2284248642626103
• License:
• Abstract: The world is abundant with diverse materials, each possessing unique surface appearances that play a crucial role in our daily perception and understanding of their properties. Despite advancements in technology enabling the capture and realistic reproduction of material appearances for visualization and quality control, the interoperability of material property information across various measurement representations and software platforms remains a complex challenge. A key to overcoming this challenge lies in the automatic identification of materials' perceptual features, enabling intuitive differentiation of properties stored in disparate material data representations. We reasoned that for many practical purposes, a compact representation of the perceptual appearance is more useful than an exhaustive physical description.This paper introduces a novel approach to material identification by encoding perceptual features obtained from dynamic visual stimuli. We conducted a psychophysical experiment to select and validate 16 particularly significant perceptual attributes obtained from videos of 347 materials. We then gathered attribute ratings from over twenty participants for each material, creating a 'material fingerprint' that encodes the unique perceptual properties of each material. Finally, we trained a multi-layer perceptron model to predict the relationship between statistical and deep learning image features and their corresponding perceptual properties. We demonstrate the model's performance in material retrieval and filtering according to individual attributes. This model represents a significant step towards simplifying the sharing and understanding of material properties in diverse digital environments regardless of their digital representation, enhancing both the accuracy and efficiency of material identification.

Related papers

• Data-efficient and Interpretable Inverse Materials Design using a Disentangled Variational Autoencoder [2.563209727695243]
  Inverse materials design has proven successful in accelerating novel material discovery. Many inverse materials design methods use unsupervised learning where a latent space is learned to offer a compact description of materials representations. Here, we present a semi-supervised learning approach based on a disentangled variational autoencoder to learn a probabilistic relationship between features, latent variables and target properties.
  arXiv  Detail & Related papers  (2024-09-10T02:21:13Z)
• Intrinsic Image Diffusion for Indoor Single-view Material Estimation [55.276815106443976]
  We present Intrinsic Image Diffusion, a generative model for appearance decomposition of indoor scenes. Given a single input view, we sample multiple possible material explanations represented as albedo, roughness, and metallic maps. Our method produces significantly sharper, more consistent, and more detailed materials, outperforming state-of-the-art methods by $1.5dB$ on PSNR and by $45%$ better FID score on albedo prediction.
  arXiv  Detail & Related papers  (2023-12-19T15:56:19Z)
• Alchemist: Parametric Control of Material Properties with Diffusion Models [51.63031820280475]
  Our method capitalizes on the generative prior of text-to-image models known for photorealism. We show the potential application of our model to material edited NeRFs.
  arXiv  Detail & Related papers  (2023-12-05T18:58:26Z)
• Multimodal Learning for Materials [7.167520424757711]
  We introduce Multimodal Learning for Materials (MultiMat), which enables self-supervised multi-modality training of foundation models for materials. We demonstrate our framework's potential using data from the Materials Project database on multiple axes.
  arXiv  Detail & Related papers  (2023-11-30T18:35:29Z)
• A General Protocol to Probe Large Vision Models for 3D Physical Understanding [84.54972153436466]
  We introduce a general protocol to evaluate whether features of an off-the-shelf large vision model encode a number of physical 'properties' of the 3D scene. We apply this protocol to properties covering scene geometry, scene material, support relations, lighting, and view-dependent measures. We find that features from Stable Diffusion and DINOv2 are good for discriminative learning of a number of properties.
  arXiv  Detail & Related papers  (2023-10-10T17:59:28Z)
• Interpretable Multimodal Emotion Recognition using Facial Features and Physiological Signals [16.549488750320336]
  It introduces a multimodal framework for emotion understanding by fusing information from visual facial features and r signals extracted from the input videos. An interpretability technique based on permutation importance analysis has also been implemented.
  arXiv  Detail & Related papers  (2023-06-05T12:57:07Z)
• ComPhy: Compositional Physical Reasoning of Objects and Events from Videos [113.2646904729092]
  The compositionality between the visible and hidden properties poses unique challenges for AI models to reason from the physical world. Existing studies on video reasoning mainly focus on visually observable elements such as object appearance, movement, and contact interaction. We propose an oracle neural-symbolic framework named Compositional Physics Learner (CPL), combining visual perception, physical property learning, dynamic prediction, and symbolic execution.
  arXiv  Detail & Related papers  (2022-05-02T17:59:13Z)
• Deep Collaborative Multi-Modal Learning for Unsupervised Kinship Estimation [53.62256887837659]
  Kinship verification is a long-standing research challenge in computer vision. We propose a novel deep collaborative multi-modal learning (DCML) to integrate the underlying information presented in facial properties. Our DCML method is always superior to some state-of-the-art kinship verification methods.
  arXiv  Detail & Related papers  (2021-09-07T01:34:51Z)
• What Image Features Boost Housing Market Predictions? [81.32205133298254]
  We propose a set of techniques for the extraction of visual features for efficient numerical inclusion in predictive algorithms. We discuss techniques such as Shannon's entropy, calculating the center of gravity, employing image segmentation, and using Convolutional Neural Networks. The set of 40 image features selected here carries a significant amount of predictive power and outperforms some of the strongest metadata predictors.
  arXiv  Detail & Related papers  (2021-07-15T06:32:10Z)
• The joint role of geometry and illumination on material recognition [16.01513204879645]
  We study how the interplay of geometry, illumination, and their spatial frequencies affects human performance on material recognition tasks. We train a deep neural network on material recognition tasks to accurately classify materials.
  arXiv  Detail & Related papers  (2021-01-07T11:29:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.