Diagnosing Errors in Video Relation Detectors
- URL: http://arxiv.org/abs/2110.13110v1
- Date: Mon, 25 Oct 2021 17:04:08 GMT
- Title: Diagnosing Errors in Video Relation Detectors
- Authors: Shuo Chen, Pascal Mettes, Cees G.M. Snoek
- Abstract summary: Video relation detection forms a new and challenging problem in computer vision.
Overall performance is still marginal and it remains unclear what the key factors are towards solving the problem.
We introduce a diagnostic tool for analyzing the sources of detection errors.
- Score: 46.792264699927436
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Video relation detection forms a new and challenging problem in computer
vision, where subjects and objects need to be localized spatio-temporally and a
predicate label needs to be assigned if and only if there is an interaction
between the two. Despite recent progress in video relation detection, overall
performance is still marginal and it remains unclear what the key factors are
towards solving the problem. Following examples set in the object detection and
action localization literature, we perform a deep dive into the error diagnosis
of current video relation detection approaches. We introduce a diagnostic tool
for analyzing the sources of detection errors. Our tool evaluates and compares
current approaches beyond the single scalar metric of mean Average Precision by
defining different error types specific to video relation detection, used for
false positive analyses. Moreover, we examine different factors of influence on
the performance in a false negative analysis, including relation length, number
of subject/object/predicate instances, and subject/object size. Finally, we
present the effect on video relation performance when considering an oracle fix
for each error type. On two video relation benchmarks, we show where current
approaches excel and fall short, allowing us to pinpoint the most important
future directions in the field. The tool is available at
\url{https://github.com/shanshuo/DiagnoseVRD}.
Related papers
- Unsupervised Model Diagnosis [49.36194740479798]
This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance.
Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
arXiv Detail & Related papers (2024-10-08T17:59:03Z) - Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detection [14.22646492640906]
We propose a simple and highly efficient decoder-free architecture for open-vocabulary visual relationship detection.
Our model consists of a Transformer-based image encoder that represents objects as tokens and models their relationships implicitly.
Our approach achieves state-of-the-art relationship detection performance on Visual Genome and on the large-vocabulary GQA benchmark at real-time inference speeds.
arXiv Detail & Related papers (2024-03-21T10:15:57Z) - Causal Feature Selection via Transfer Entropy [59.999594949050596]
Causal discovery aims to identify causal relationships between features with observational data.
We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures.
We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
arXiv Detail & Related papers (2023-10-17T08:04:45Z) - Diagnosing Human-object Interaction Detectors [42.283857276076596]
We introduce a diagnosis toolbox to provide detailed quantitative break-down analysis of HOI detection models.
We analyze eight state-of-the-art HOI detection models and provide valuable diagnosis insights to foster future research.
arXiv Detail & Related papers (2023-08-16T17:39:15Z) - Understanding Factual Errors in Summarization: Errors, Summarizers,
Datasets, Error Detectors [105.12462629663757]
In this work, we aggregate factuality error annotations from nine existing datasets and stratify them according to the underlying summarization model.
We compare performance of state-of-the-art factuality metrics, including recent ChatGPT-based metrics, on this stratified benchmark and show that their performance varies significantly across different types of summarization models.
arXiv Detail & Related papers (2022-05-25T15:26:48Z) - Exploiting Multi-Object Relationships for Detecting Adversarial Attacks
in Complex Scenes [51.65308857232767]
Vision systems that deploy Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples.
Recent research has shown that checking the intrinsic consistencies in the input data is a promising way to detect adversarial attacks.
We develop a novel approach to perform context consistency checks using language models.
arXiv Detail & Related papers (2021-08-19T00:52:10Z) - Graph Neural Network-Based Anomaly Detection in Multivariate Time Series [17.414474298706416]
We develop a new way to detect anomalies in high-dimensional time series data.
Our approach combines a structure learning approach with graph neural networks.
We show that our method detects anomalies more accurately than baseline approaches.
arXiv Detail & Related papers (2021-06-13T09:07:30Z) - Unsupervised Domain Adaption of Object Detectors: A Survey [87.08473838767235]
Recent advances in deep learning have led to the development of accurate and efficient models for various computer vision applications.
Learning highly accurate models relies on the availability of datasets with a large number of annotated images.
Due to this, model performance drops drastically when evaluated on label-scarce datasets having visually distinct images.
arXiv Detail & Related papers (2021-05-27T23:34:06Z) - Video Relation Detection with Trajectory-aware Multi-modal Features [13.358584829993193]
We present video relation detection with trajectory-aware multi-modal features to solve this task.
Our method won the first place on the video relation detection task of Video Relation Understanding Grand Challenge in ACM Multimedia 2020 with 11.74% mAP.
arXiv Detail & Related papers (2021-01-20T14:49:02Z) - Anomaly Detection for Aggregated Data Using Multi-Graph Autoencoder [21.81622481466591]
We focus on creating an Anomaly detection models for system logs.
We present a thorough analysis of the aggregated data and the relationships between aggregated events.
We propose Multiple-graphs autoencoder MGAE, a novel convolutional graphs-autoencoder model.
arXiv Detail & Related papers (2021-01-11T17:38:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.