Ensemble Distillation for Unsupervised Constituency Parsing

• URL: http://arxiv.org/abs/2310.01717v2
• Date: Fri, 26 Apr 2024 00:41:48 GMT
• Title: Ensemble Distillation for Unsupervised Constituency Parsing
• Authors: Behzad Shayegh, Yanshuai Cao, Xiaodan Zhu, Jackie C. K. Cheung, Lili Mou,
• Abstract summary: We investigate the unsupervised constituency parsing task, which organizes words and phrases of a sentence into a hierarchical structure without using linguistically annotated data. We propose a notion of "tree averaging," based on which we further propose a novel ensemble method for unsupervised parsing.
• Score: 40.96887945888518
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We investigate the unsupervised constituency parsing task, which organizes words and phrases of a sentence into a hierarchical structure without using linguistically annotated data. We observe that existing unsupervised parsers capture differing aspects of parsing structures, which can be leveraged to enhance unsupervised parsing performance. To this end, we propose a notion of "tree averaging," based on which we further propose a novel ensemble method for unsupervised parsing. To improve inference efficiency, we further distill the ensemble knowledge into a student model; such an ensemble-then-distill process is an effective approach to mitigate the over-smoothing problem existing in common multi-teacher distilling methods. Experiments show that our method surpasses all previous approaches, consistently demonstrating its effectiveness and robustness across various runs, with different ensemble components, and under domain-shift conditions.

Related papers

• Error Diversity Matters: An Error-Resistant Ensemble Method for Unsupervised Dependency Parsing [49.60415088899208]
  We propose an efficient ensemble-selection approach that considers error diversity and avoids error accumulation. Results demonstrate that our approach outperforms each individual model as well as previous ensemble techniques.
  arXiv  Detail & Related papers  (2024-12-16T08:23:50Z)
• Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsing [23.091613114955543]
  We propose to build an ensemble of different runs of the existing discontinuous by averaging the predicted trees. We then develop an efficient exact algorithm to tackle the task, which runs in a reasonable time for all samples. Results on three datasets show our method outperforms all baselines in all metrics.
  arXiv  Detail & Related papers  (2024-02-29T21:49:31Z)
• Causal Unsupervised Semantic Segmentation [60.178274138753174]
  Unsupervised semantic segmentation aims to achieve high-quality semantic grouping without human-labeled annotations. We propose a novel framework, CAusal Unsupervised Semantic sEgmentation (CAUSE), which leverages insights from causal inference.
  arXiv  Detail & Related papers  (2023-10-11T10:54:44Z)
• Unsupervised Chunking with Hierarchical RNN [62.15060807493364]
  This paper introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a two-layer Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Experiments on the CoNLL-2000 dataset reveal a notable improvement over existing unsupervised methods, enhancing phrase F1 score by up to 6 percentage points.
  arXiv  Detail & Related papers  (2023-09-10T02:55:12Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• Demystifying Unsupervised Semantic Correspondence Estimation [13.060538447838303]
  We explore semantic correspondence estimation through the lens of unsupervised learning. We thoroughly evaluate several recently proposed unsupervised methods across multiple challenging datasets. We introduce a new unsupervised correspondence approach which utilizes the strength of pre-trained features while encouraging better matches during training.
  arXiv  Detail & Related papers  (2022-07-11T17:59:51Z)
• FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment [93.09267863425492]
  We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable. We construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures.
  arXiv  Detail & Related papers  (2022-04-07T17:59:32Z)
• Rethinking Ensemble-Distillation for Semantic Segmentation Based Unsupervised Domain Adaptation [6.487749466672554]
  We propose a flexible ensemble-distillation framework for performing semantic segmentation based UDA. Our framework is designed to be robust against the output inconsistency and the performance variation of the members within the ensemble.
  arXiv  Detail & Related papers  (2021-04-29T08:47:24Z)
• Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation [39.95831985522991]
  We propose a framework integrated with more reliable supervision guided by semantic co-segmentation and data-augmentation. Our proposed methods achieve the state-of-the-art performance among unsupervised methods, and even compete on par with supervised methods.
  arXiv  Detail & Related papers  (2021-04-12T11:48:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.