ReINTEL Challenge 2020: A Multimodal Ensemble Model for Detecting Unreliable Information on Vietnamese SNS

• URL: http://arxiv.org/abs/2012.10267v1
• Date: Fri, 18 Dec 2020 14:33:08 GMT
• Title: ReINTEL Challenge 2020: A Multimodal Ensemble Model for Detecting Unreliable Information on Vietnamese SNS
• Authors: Nguyen Manh Duc Tuan, Pham Quang Nhat Minh
• Abstract summary: We propose a novel multimodal ensemble model which combines two multimodal models to solve the task. Experimental results showed that our proposed multimodal ensemble model improved against single models in term of ROC AUC score.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we present our methods for unrealiable information identification task at VLSP 2020 ReINTEL Challenge. The task is to classify a piece of information into reliable or unreliable category. We propose a novel multimodal ensemble model which combines two multimodal models to solve the task. In each multimodal model, we combined feature representations acquired from three different data types: texts, images, and metadata. Multimodal features are derived from three neural networks and fused for classification. Experimental results showed that our proposed multimodal ensemble model improved against single models in term of ROC AUC score. We obtained 0.9445 AUC score on the private test of the challenge.

Related papers

• MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching [54.740256498985026]
  Keypoint detection and description methods often struggle with multimodal data. We propose a modality-invariant feature learning network (MIFNet) to compute modality-invariant features for keypoint descriptions in multimodal image matching.
  arXiv  Detail & Related papers  (2025-01-20T06:56:30Z)
• SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection [73.49799596304418]
  This paper introduces a new task called Multi-Modal datasets and Multi-Task Object Detection (M2Det) for remote sensing. It is designed to accurately detect horizontal or oriented objects from any sensor modality. This task poses challenges due to 1) the trade-offs involved in managing multi-modal modelling and 2) the complexities of multi-task optimization.
  arXiv  Detail & Related papers  (2024-12-30T02:47:51Z)
• VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks [60.5257456681402]
  We study the potential for building universal embeddings capable of handling a wide range of downstream tasks. We build a series of VLM2Vec models on SoTA VLMs like Phi-3.5-V, LLaVA-1.6 and evaluate them on MMEB's evaluation split. Our results show that VLM2Vec achieves an absolute average improvement of 10% to 20% over existing multimodal embedding models.
  arXiv  Detail & Related papers  (2024-10-07T16:14:05Z)
• 4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities [17.374241865041856]
  We show the possibility of training one model to solve at least 3x more tasks/modalities than existing ones and doing so without a loss in performance. We successfully scale the training to a three billion parameter model using tens of modalities and different datasets. The resulting models and training code are open sourced at 4m.epfl.ch.
  arXiv  Detail & Related papers  (2024-06-13T17:59:42Z)
• Integrating Text and Image Pre-training for Multi-modal Algorithmic Reasoning [7.84845040922464]
  We present our solution for SMART-101 Challenge of CVPR Multi-modal Algorithmic Reasoning Task 2024. Unlike traditional visual questions and answer tasks, this challenge evaluates abstraction, deduction and generalization ability of neural network. Our model is based on two pre-trained models, dedicated to extract features from text and image respectively.
  arXiv  Detail & Related papers  (2024-06-08T01:45:06Z)
• On Uni-Modal Feature Learning in Supervised Multi-Modal Learning [21.822251958013737]
  We abstract the features (i.e. learned representations) of multi-modal data into 1) uni-modal features, which can be learned from uni-modal training, and 2) paired features, which can only be learned from cross-modal interactions. We demonstrate that, under a simple guiding strategy, we can achieve comparable results to other complex late-fusion or intermediate-fusion methods on various multi-modal datasets.
  arXiv  Detail & Related papers  (2023-05-02T07:15:10Z)
• An Empirical Study of Multimodal Model Merging [148.48412442848795]
  Model merging is a technique that fuses multiple models trained on different tasks to generate a multi-task solution. We conduct our study for a novel goal where we can merge vision, language, and cross-modal transformers of a modality-specific architecture. We propose two metrics that assess the distance between weights to be merged and can serve as an indicator of the merging outcomes.
  arXiv  Detail & Related papers  (2023-04-28T15:43:21Z)
• Informative Data Selection with Uncertainty for Multi-modal Object Detection [25.602915381482468]
  We propose a universal uncertainty-aware multi-modal fusion model. Our model reduces the randomness in fusion and generates reliable output. Our fusion model is proven to resist severe noise interference like Gaussian, motion blur, and frost, with only slight degradation.
  arXiv  Detail & Related papers  (2023-04-23T16:36:13Z)
• MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models [103.9987158554515]
  MultiViz is a method for analyzing the behavior of multimodal models by scaffolding the problem of interpretability into 4 stages. We show that the complementary stages in MultiViz together enable users to simulate model predictions, assign interpretable concepts to features, perform error analysis on model misclassifications, and use insights from error analysis to debug models.
  arXiv  Detail & Related papers  (2022-06-30T18:42:06Z)
• Logically at the Factify 2022: Multimodal Fact Verification [2.8914815569249823]
  This paper describes our participant system for the multi-modal fact verification (Factify) challenge at AAAI 2022. Two baseline approaches are proposed and explored including an ensemble model and a multi-modal attention network. Our best model is ranked first in leaderboard which obtains a weighted average F-measure of 0.77 on both validation and test set.
  arXiv  Detail & Related papers  (2021-12-16T23:34:07Z)
• InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining [76.32065400614162]
  We propose a novel model, namely InterBERT (BERT for Interaction), which is the first model of our series of multimodal pretraining methods M6. The model owns strong capability of modeling interaction between the information flows of different modalities. We propose a large-scale dataset for multi-modal pretraining in Chinese, and we develop the Chinese InterBERT which is the first Chinese multi-modal pretrained model.
  arXiv  Detail & Related papers  (2020-03-30T03:13:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.