Learning Compositional Neural Information Fusion for Human Parsing
- URL: http://arxiv.org/abs/2001.06804v1
- Date: Sun, 19 Jan 2020 10:35:14 GMT
- Title: Learning Compositional Neural Information Fusion for Human Parsing
- Authors: Wenguan Wang, Zhijie Zhang, Siyuan Qi, Jianbing Shen, Yanwei Pang, and
Ling Shao
- Abstract summary: We formulate the approach as a neural information fusion framework.
Our model assembles the information from three inference processes over the hierarchy.
The whole model is end-to-end differentiable, explicitly modeling information flows and structures.
- Score: 181.48380078517525
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This work proposes to combine neural networks with the compositional
hierarchy of human bodies for efficient and complete human parsing. We
formulate the approach as a neural information fusion framework. Our model
assembles the information from three inference processes over the hierarchy:
direct inference (directly predicting each part of a human body using image
information), bottom-up inference (assembling knowledge from constituent
parts), and top-down inference (leveraging context from parent nodes). The
bottom-up and top-down inferences explicitly model the compositional and
decompositional relations in human bodies, respectively. In addition, the
fusion of multi-source information is conditioned on the inputs, i.e., by
estimating and considering the confidence of the sources. The whole model is
end-to-end differentiable, explicitly modeling information flows and
structures. Our approach is extensively evaluated on four popular datasets,
outperforming the state-of-the-arts in all cases, with a fast processing speed
of 23fps. Our code and results have been released to help ease future research
in this direction.
Related papers
- Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models [65.82564074712836]
We introduce DIFfusionHOI, a new HOI detector shedding light on text-to-image diffusion models.
We first devise an inversion-based strategy to learn the expression of relation patterns between humans and objects in embedding space.
These learned relation embeddings then serve as textual prompts, to steer diffusion models generate images that depict specific interactions.
arXiv Detail & Related papers (2024-10-26T12:00:33Z) - Evaluating alignment between humans and neural network representations in image-based learning tasks [5.657101730705275]
We tested how well the representations of $86$ pretrained neural network models mapped to human learning trajectories.
We found that while training dataset size was a core determinant of alignment with human choices, contrastive training with multi-modal data (text and imagery) was a common feature of currently publicly available models that predicted human generalisation.
In conclusion, pretrained neural networks can serve to extract representations for cognitive models, as they appear to capture some fundamental aspects of cognition that are transferable across tasks.
arXiv Detail & Related papers (2023-06-15T08:18:29Z) - Deep Learning for Human Parsing: A Survey [54.812353922568995]
We provide an analysis of state-of-the-art human parsing methods, covering a broad spectrum of pioneering works for semantic human parsing.
We introduce five insightful categories: (1) structure-driven architectures exploit the relationship of different human parts and the inherent hierarchical structure of a human body, (2) graph-based networks capture the global information to achieve an efficient and complete human body analysis, (3) context-aware networks explore useful contexts across all pixel to characterize a pixel of the corresponding class, and (4) LSTM-based methods can combine short-distance and long-distance spatial dependencies to better exploit abundant local and global contexts.
arXiv Detail & Related papers (2023-01-29T10:54:56Z) - COAP: Compositional Articulated Occupancy of People [28.234772596912162]
We present a novel neural implicit representation for articulated human bodies.
We employ a part-aware encoder-decoder architecture to learn neural articulated occupancy.
Our method largely outperforms existing solutions in terms of both efficiency and accuracy.
arXiv Detail & Related papers (2022-04-13T06:02:20Z) - Learning to Segment Human Body Parts with Synthetically Trained Deep
Convolutional Networks [58.0240970093372]
This paper presents a new framework for human body part segmentation based on Deep Convolutional Neural Networks trained using only synthetic data.
The proposed approach achieves cutting-edge results without the need of training the models with real annotated data of human body parts.
arXiv Detail & Related papers (2021-02-02T12:26:50Z) - HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular
Multi-Person 3D Pose Estimation [54.23770284299979]
This paper introduces a novel form of supervision - Hierarchical Multi-person Ordinal Relations (HMOR)
HMOR encodes interaction information as the ordinal relations of depths and angles hierarchically.
An integrated top-down model is designed to leverage these ordinal relations in the learning process.
The proposed method significantly outperforms state-of-the-art methods on publicly available multi-person 3D pose datasets.
arXiv Detail & Related papers (2020-08-01T07:53:27Z) - Generating Hierarchical Explanations on Text Classification via Feature
Interaction Detection [21.02924712220406]
We build hierarchical explanations by detecting feature interactions.
Such explanations visualize how words and phrases are combined at different levels of the hierarchy.
Experiments show the effectiveness of the proposed method in providing explanations both faithful to models and interpretable to humans.
arXiv Detail & Related papers (2020-04-04T20:56:37Z) - Hierarchical Human Parsing with Typed Part-Relation Reasoning [179.64978033077222]
How to model human structures is the central theme in this task.
We seek to simultaneously exploit the representational capacity of deep graph networks and the hierarchical human structures.
arXiv Detail & Related papers (2020-03-10T16:45:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.