OpenViewer: Openness-Aware Multi-View Learning

• URL: http://arxiv.org/abs/2412.12596v1
• Date: Tue, 17 Dec 2024 06:54:54 GMT
• Title: OpenViewer: Openness-Aware Multi-View Learning
• Authors: Shide Du, Zihan Fang, Yanchao Tan, Changwei Wang, Shiping Wang, Wenzhong Guo,
• Abstract summary: Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views.<n> deploying these models in real-world scenarios presents two primary openness challenges.<n>We propose OpenViewer, an openness-aware multi-view learning framework with theoretical support.
• Score: 22.31635020800168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views, typically relying on predefined categories. However, deploying these models in real-world scenarios presents two primary openness challenges. 1) Lack of Interpretability: The integration mechanisms of multi-view data in existing black-box models remain poorly explained; 2) Insufficient Generalization: Most models are not adapted to multi-view scenarios involving unknown categories. To address these challenges, we propose OpenViewer, an openness-aware multi-view learning framework with theoretical support. This framework begins with a Pseudo-Unknown Sample Generation Mechanism to efficiently simulate open multi-view environments and previously adapt to potential unknown samples. Subsequently, we introduce an Expression-Enhanced Deep Unfolding Network to intuitively promote interpretability by systematically constructing functional prior-mapping modules and effectively providing a more transparent integration mechanism for multi-view data. Additionally, we establish a Perception-Augmented Open-Set Training Regime to significantly enhance generalization by precisely boosting confidences for known categories and carefully suppressing inappropriate confidences for unknown ones. Experimental results demonstrate that OpenViewer effectively addresses openness challenges while ensuring recognition performance for both known and unknown samples. The code is released at https://github.com/dushide/OpenViewer.

Related papers

• Generalized Deep Multi-view Clustering via Causal Learning with Partially Aligned Cross-view Correspondence [72.41989962665285]
  Multi-view clustering (MVC) aims to explore the common clustering structure across multiple views.<n>However, real-world scenarios often present a challenge as only partial data is consistently aligned across different views.<n>We design a causal multi-view clustering network, termed CauMVC, to tackle this problem.
  arXiv  Detail & Related papers  (2025-09-19T14:31:40Z)
• Enhancing Multi-view Open-set Learning via Ambiguity Uncertainty Calibration and View-wise Debiasing [10.975705043375212]
  We propose a multi-view open-set learning framework via ambiguity uncertainty calibration and view-wise debiasing.<n>Experiments on diverse multi-view benchmarks demonstrate that the proposed framework consistently enhances unknown-class recognition.
  arXiv  Detail & Related papers  (2025-08-02T06:46:16Z)
• MemoryOut: Learning Principal Features via Multimodal Sparse Filtering Network for Semi-supervised Video Anomaly Detection [30.470777079947958]
  Video Anomaly Detection (VAD) methods based on reconstruction or prediction face two critical challenges.<n>Strong generalization capability often results in accurate reconstruction or prediction of abnormal events.<n>reliance only on low-level appearance and motion cues limits their ability to identify high-level semantic in abnormal events from complex scenes.
  arXiv  Detail & Related papers  (2025-06-03T07:14:57Z)
• VL-SAM-V2: Open-World Object Detection with General and Specific Query Fusion [7.719330752075467]
  We present an open-world object detection framework capable of discovering unseen objects while achieving favorable performance.<n>By adjusting queries from open-set models, we enable VL-SAM-V2 to be evaluated in the open-set or open-ended mode.<n> Experimental results on LVIS show that our method surpasses the previous open-set and open-ended methods, especially on rare objects.
  arXiv  Detail & Related papers  (2025-05-25T05:44:02Z)
• Multi-View Industrial Anomaly Detection with Epipolar Constrained Cross-View Fusion [15.819291772583393]
  We introduce an epipolar geometry-constrained attention module to guide cross-view fusion. To further enhance the potential of cross-view attention, we propose a pretraining strategy inspired by memory bank-based anomaly detection. We demonstrate that our framework outperforms existing methods on the state-of-the-art multi-view anomaly detection dataset.
  arXiv  Detail & Related papers  (2025-03-14T05:02:54Z)
• Regularized Contrastive Partial Multi-view Outlier Detection [76.77036536484114]
  We propose a novel method named Regularized Contrastive Partial Multi-view Outlier Detection (RCPMOD) In this framework, we utilize contrastive learning to learn view-consistent information and distinguish outliers by the degree of consistency. Experimental results on four benchmark datasets demonstrate that our proposed approach could outperform state-of-the-art competitors.
  arXiv  Detail & Related papers  (2024-08-02T14:34:27Z)
• Masked Two-channel Decoupling Framework for Incomplete Multi-view Weak Multi-label Learning [21.49630640829186]
  In this paper, we focus on the complex yet highly realistic task of incomplete multi-view weak multi-label learning. We propose a masked two-channel decoupling framework based on deep neural networks to solve this problem. Our model is fully adaptable to arbitrary view and label absences while also performing well on the ideal full data.
  arXiv  Detail & Related papers  (2024-04-26T11:39:50Z)
• Towards Generalized Multi-stage Clustering: Multi-view Self-distillation [10.368796552760571]
  Existing multi-stage clustering methods independently learn the salient features from multiple views and then perform the clustering task. This paper proposes a novel multi-stage deep MVC framework where multi-view self-distillation (DistilMVC) is introduced to distill dark knowledge of label distribution.
  arXiv  Detail & Related papers  (2023-10-29T03:35:34Z)
• Open Visual Knowledge Extraction via Relation-Oriented Multimodality Model Prompting [89.95541601837719]
  We take a first exploration to a new paradigm of open visual knowledge extraction. OpenVik consists of an open relational region detector to detect regions potentially containing relational knowledge. A visual knowledge generator that generates format-free knowledge by prompting the large multimodality model with the detected region of interest.
  arXiv  Detail & Related papers  (2023-10-28T20:09:29Z)
• Debunking Free Fusion Myth: Online Multi-view Anomaly Detection with Disentangled Product-of-Experts Modeling [25.02446577349165]
  Multi-view or even multi-modal data is appealing yet challenging for real-world applications. We propose dPoE, a novel multi-view variational autoencoder model that involves (1) a Product-of-Experts layer in tackling multi-view data, (2) a Total Correction discriminator in disentangling view-common and view-specific representations, and (3) a joint loss function in wrapping up all components.
  arXiv  Detail & Related papers  (2023-10-28T15:14:43Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• Multi-View Class Incremental Learning [57.14644913531313]
  Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. This paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views.
  arXiv  Detail & Related papers  (2023-06-16T08:13:41Z)
• Open Long-Tailed Recognition in a Dynamic World [82.91025831618545]
  Real world data often exhibits a long-tailed and open-ended (with unseen classes) distribution. A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes) We define Open Long-Tailed Recognition++ as learning from such naturally distributed data and optimizing for the classification accuracy over a balanced test set.
  arXiv  Detail & Related papers  (2022-08-17T15:22:20Z)
• Generative Partial Multi-View Clustering [133.36721417531734]
  We propose a generative partial multi-view clustering model, named as GP-MVC, to address the incomplete multi-view problem. First, multi-view encoder networks are trained to learn common low-dimensional representations, followed by a clustering layer to capture the consistent cluster structure across multiple views. Second, view-specific generative adversarial networks are developed to generate the missing data of one view conditioning on the shared representation given by other views.
  arXiv  Detail & Related papers  (2020-03-29T17:48:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.