Physics-Informed Computer Vision: A Review and Perspectives
- URL: http://arxiv.org/abs/2305.18035v3
- Date: Mon, 13 May 2024 01:06:58 GMT
- Title: Physics-Informed Computer Vision: A Review and Perspectives
- Authors: Chayan Banerjee, Kien Nguyen, Clinton Fookes, George Karniadakis,
- Abstract summary: incorporation of physical information in machine learning frameworks is opening and transforming many application domains.
We present a systematic literature review of more than 250 papers on formulation and approaches to computer vision tasks guided by physical laws.
- Score: 22.71741766133866
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The incorporation of physical information in machine learning frameworks is opening and transforming many application domains. Here the learning process is augmented through the induction of fundamental knowledge and governing physical laws. In this work, we explore their utility for computer vision tasks in interpreting and understanding visual data. We present a systematic literature review of more than 250 papers on formulation and approaches to computer vision tasks guided by physical laws. We begin by decomposing the popular computer vision pipeline into a taxonomy of stages and investigate approaches to incorporate governing physical equations in each stage. Existing approaches in computer vision tasks are analyzed with regard to what governing physical processes are modeled and formulated, and how they are incorporated, i.e. modification of input data (observation bias), modification of network architectures (inductive bias), and modification of training losses (learning bias). The taxonomy offers a unified view of the application of the physics-informed capability, highlighting where physics-informed learning has been conducted and where the gaps and opportunities are. Finally, we highlight open problems and challenges to inform future research. While still in its early days, the study of physics-informed computer vision has the promise to develop better computer vision models that can improve physical plausibility, accuracy, data efficiency, and generalization in increasingly realistic applications.
Related papers
- Fairness and Bias Mitigation in Computer Vision: A Survey [61.01658257223365]
Computer vision systems are increasingly being deployed in high-stakes real-world applications.
There is a dire need to ensure that they do not propagate or amplify any discriminatory tendencies in historical or human-curated data.
This paper presents a comprehensive survey on fairness that summarizes and sheds light on ongoing trends and successes in the context of computer vision.
arXiv Detail & Related papers (2024-08-05T13:44:22Z) - A Survey on Physics Informed Reinforcement Learning: Review and Open
Problems [25.3906503332344]
We present a review of the literature on incorporating physics information, as known as physics priors, in reinforcement learning approaches.
We introduce a novel taxonomy with the reinforcement learning pipeline as the backbone to classify existing works.
This nascent field holds great potential for enhancing reinforcement learning algorithms by increasing their physical plausibility, precision, data efficiency, and applicability in real-world scenarios.
arXiv Detail & Related papers (2023-09-05T02:45:18Z) - Physics-Informed Machine Learning: A Survey on Problems, Methods and
Applications [31.157298426186653]
Recent work shows that it provides potential benefits for machine learning models by incorporating the physical prior and collected data.
We present this learning paradigm called Physics-Informed Machine Learning (PIML) which is to build a model that leverages empirical data and available physical prior knowledge.
arXiv Detail & Related papers (2022-11-15T11:34:30Z) - A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective [71.03621840455754]
Graph Neural Networks (GNNs) have gained momentum in graph representation learning.
graph Transformers embed a graph structure into the Transformer architecture to overcome the limitations of local neighborhood aggregation.
This paper presents a comprehensive review of GNNs and graph Transformers in computer vision from a task-oriented perspective.
arXiv Detail & Related papers (2022-09-27T08:10:14Z) - Deep Learning to See: Towards New Foundations of Computer Vision [88.69805848302266]
This book criticizes the supposed scientific progress in the field of computer vision.
It proposes the investigation of vision within the framework of information-based laws of nature.
arXiv Detail & Related papers (2022-06-30T15:20:36Z) - K-LITE: Learning Transferable Visual Models with External Knowledge [242.3887854728843]
K-LITE (Knowledge-augmented Language-Image Training and Evaluation) is a strategy to leverage external knowledge to build transferable visual systems.
In training, it enriches entities in natural language with WordNet and Wiktionary knowledge.
In evaluation, the natural language is also augmented with external knowledge and then used to reference learned visual concepts.
arXiv Detail & Related papers (2022-04-20T04:47:01Z) - Physics-informed Reinforcement Learning for Perception and Reasoning
about Fluids [0.0]
We propose a physics-informed reinforcement learning strategy for fluid perception and reasoning from observations.
We develop a method for the tracking (perception) and analysis (reasoning) of any previously unseen liquid whose free surface is observed with a commodity camera.
arXiv Detail & Related papers (2022-03-11T07:01:23Z) - Knowledge as Invariance -- History and Perspectives of
Knowledge-augmented Machine Learning [69.99522650448213]
Research in machine learning is at a turning point.
Research interests are shifting away from increasing the performance of highly parameterized models to exceedingly specific tasks.
This white paper provides an introduction and discussion of this emerging field in machine learning research.
arXiv Detail & Related papers (2020-12-21T15:07:19Z) - Physical reservoir computing -- An introductory perspective [0.0]
Physical reservoir computing allows one to exploit the complex dynamics of physical systems as information-processing devices.
This paper aims to illustrate the potentials of the framework using examples from soft robotics.
arXiv Detail & Related papers (2020-05-03T05:39:06Z) - Visual Grounding of Learned Physical Models [66.04898704928517]
Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions.
We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors.
Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
arXiv Detail & Related papers (2020-04-28T17:06:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.