Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning
for Video Question Answering
- URL: http://arxiv.org/abs/2401.01510v1
- Date: Wed, 3 Jan 2024 02:29:34 GMT
- Title: Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning
for Video Question Answering
- Authors: Haopeng Li, Qiuhong Ke, Mingming Gong, and Tom Drummond
- Abstract summary: We introduce the concept of uncertainty-aware curriculum learning (CL)
Here, uncertainty serves as the guiding principle for dynamically adjusting the difficulty.
In practice, we seamlessly integrate the VideoQA model into our framework and conduct comprehensive experiments.
- Score: 63.12469700986452
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While significant advancements have been made in video question answering
(VideoQA), the potential benefits of enhancing model generalization through
tailored difficulty scheduling have been largely overlooked in existing
research. This paper seeks to bridge that gap by incorporating VideoQA into a
curriculum learning (CL) framework that progressively trains models from
simpler to more complex data. Recognizing that conventional self-paced CL
methods rely on training loss for difficulty measurement, which might not
accurately reflect the intricacies of video-question pairs, we introduce the
concept of uncertainty-aware CL. Here, uncertainty serves as the guiding
principle for dynamically adjusting the difficulty. Furthermore, we address the
challenge posed by uncertainty by presenting a probabilistic modeling approach
for VideoQA. Specifically, we conceptualize VideoQA as a stochastic computation
graph, where the hidden representations are treated as stochastic variables.
This yields two distinct types of uncertainty: one related to the inherent
uncertainty in the data and another pertaining to the model's confidence. In
practice, we seamlessly integrate the VideoQA model into our framework and
conduct comprehensive experiments. The findings affirm that our approach not
only achieves enhanced performance but also effectively quantifies uncertainty
in the context of VideoQA.
Related papers
- Uncertainty-aware self-training with expectation maximization basis transformation [9.7527450662978]
We propose a new self-training framework to combine uncertainty information of both model and dataset.
Specifically, we propose to use Expectation-Maximization (EM) to smooth the labels and comprehensively estimate the uncertainty information.
arXiv Detail & Related papers (2024-05-02T11:01:31Z) - Grounded Question-Answering in Long Egocentric Videos [39.281013854331285]
open-ended question-answering (QA) in long, egocentric videos allows individuals or robots to inquire about their own past visual experiences.
This task presents unique challenges, including the complexity of temporally grounding queries within extensive video content.
Our proposed approach tackles these challenges by (i) integrating query grounding and answering within a unified model to reduce error propagation.
arXiv Detail & Related papers (2023-12-11T16:31:55Z) - Uncertainty-aware Language Modeling for Selective Question Answering [107.47864420630923]
We present an automatic large language model (LLM) conversion approach that produces uncertainty-aware LLMs.
Our approach is model- and data-agnostic, is computationally-efficient, and does not rely on external models or systems.
arXiv Detail & Related papers (2023-11-26T22:47:54Z) - Improving the Reliability of Large Language Models by Leveraging
Uncertainty-Aware In-Context Learning [76.98542249776257]
Large-scale language models often face the challenge of "hallucination"
We introduce an uncertainty-aware in-context learning framework to empower the model to enhance or reject its output in response to uncertainty.
arXiv Detail & Related papers (2023-10-07T12:06:53Z) - CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms.
Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner.
Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
arXiv Detail & Related papers (2022-06-29T15:22:01Z) - Invariant Grounding for Video Question Answering [72.87173324555846]
Video Question Answering (VideoQA) is the task of answering questions about a video.
In leading VideoQA models, the typical learning objective, empirical risk minimization (ERM), latches on superficial correlations between video-question pairs and answers.
We propose a new learning framework, Invariant Grounding for VideoQA (IGV), to ground the question-critical scene.
arXiv Detail & Related papers (2022-06-06T04:37:52Z) - A Hierarchical Variational Neural Uncertainty Model for Stochastic Video
Prediction [45.6432265855424]
We introduce Neural Uncertainty Quantifier (NUQ) - a principled quantification of the model's predictive uncertainty.
Our proposed framework trains more effectively compared to the state-of-theart models.
arXiv Detail & Related papers (2021-10-06T00:25:22Z) - HySTER: A Hybrid Spatio-Temporal Event Reasoner [75.41988728376081]
We present the HySTER: a Hybrid Spatio-Temporal Event Reasoner to reason over physical events in videos.
We define a method based on general temporal, causal and physics rules which can be transferred across tasks.
This work sets the foundations for the incorporation of inductive logic programming in the field of VideoQA.
arXiv Detail & Related papers (2021-01-17T11:07:17Z) - Self-supervised pre-training and contrastive representation learning for
multiple-choice video QA [39.78914328623504]
Video Question Answering (Video QA) requires fine-grained understanding of both video and language modalities to answer the given questions.
We propose novel training schemes for multiple-choice video question answering with a self-supervised pre-training stage and a supervised contrastive learning in the main stage as an auxiliary learning.
We evaluate our proposed model on highly competitive benchmark datasets related to multiple-choice video QA: TVQA, TVQA+, and DramaQA.
arXiv Detail & Related papers (2020-09-17T03:37:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.