PropNet: a White-Box and Human-Like Network for Sentence Representation

• URL: http://arxiv.org/abs/2502.10725v1
• Date: Sat, 15 Feb 2025 08:28:58 GMT
• Title: PropNet: a White-Box and Human-Like Network for Sentence Representation
• Authors: Fei Yang,
• Abstract summary: PropNet is a hierarchical network based on the propositions contained in a sentence. PropNet enables us to analyze and understand the human cognitive processes underlying STS benchmarks.
• Score: 3.994730279677248
• License:
• Abstract: Transformer-based embedding methods have dominated the field of sentence representation in recent years. Although they have achieved remarkable performance on NLP missions, such as semantic textual similarity (STS) tasks, their black-box nature and large-data-driven training style have raised concerns, including issues related to bias, trust, and safety. Many efforts have been made to improve the interpretability of embedding models, but these problems have not been fundamentally resolved. To achieve inherent interpretability, we propose a purely white-box and human-like sentence representation network, PropNet. Inspired by findings from cognitive science, PropNet constructs a hierarchical network based on the propositions contained in a sentence. While experiments indicate that PropNet has a significant gap compared to state-of-the-art (SOTA) embedding models in STS tasks, case studies reveal substantial room for improvement. Additionally, PropNet enables us to analyze and understand the human cognitive processes underlying STS benchmarks.

Related papers

• Language Model Meets Prototypes: Towards Interpretable Text Classification Models through Prototypical Networks [1.1711824752079485]
  dissertation focuses on developing intrinsically interpretable models when using LMs as encoders. I developed a novel white-box multi-head graph attention-based prototype network. I am working on extending the attention-based prototype network with contrastive learning to redesign an interpretable graph neural network.
  arXiv  Detail & Related papers  (2024-12-04T22:59:35Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• Hierarchical Invariance for Robust and Interpretable Vision Tasks at Larger Scales [54.78115855552886]
  We show how to construct over-complete invariants with a Convolutional Neural Networks (CNN)-like hierarchical architecture. With the over-completeness, discriminative features w.r.t. the task can be adaptively formed in a Neural Architecture Search (NAS)-like manner. For robust and interpretable vision tasks at larger scales, hierarchical invariant representation can be considered as an effective alternative to traditional CNN and invariants.
  arXiv  Detail & Related papers  (2024-02-23T16:50:07Z)
• Robust Text Classification: Analyzing Prototype-Based Networks [12.247144383314177]
  Prototype-Based Networks (PBNs) have been shown to be robust to noise for computer vision tasks. We study whether the robustness properties of PBNs transfer to text classification tasks under both targeted and static adversarial attack settings. We showcase how PBNs' interpretability can help us to understand PBNs' robustness properties.
  arXiv  Detail & Related papers  (2023-11-11T19:34:06Z)
• Evaluation and Improvement of Interpretability for Self-Explainable Part-Prototype Networks [43.821442711496154]
  Part-prototype networks have attracted broad research interest for their intrinsic interpretability and comparable accuracy to non-interpretable counterparts. We make the first attempt to quantitatively and objectively evaluate the interpretability of the part-prototype networks. We propose an elaborated part-prototype network with a shallow-deep feature alignment module and a score aggregation module to improve the interpretability of prototypes.
  arXiv  Detail & Related papers  (2022-12-12T14:59:11Z)
• Emergent Linguistic Structures in Neural Networks are Fragile [20.692540987792732]
  Large Language Models (LLMs) have been reported to have strong performance on natural language processing tasks. We propose a framework to assess the consistency and robustness of linguistic representations.
  arXiv  Detail & Related papers  (2022-10-31T15:43:57Z)
• SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers [61.48159785138462]
  This paper aims to improve the performance of text-to-dependence by exploring the intrinsic uncertainties in the neural network based approaches (called SUN) Extensive experiments on five benchmark datasets demonstrate that our method significantly outperforms competitors and achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-09-14T06:27:51Z)
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation [107.27377745720243]
  This paper proposes a self-ensembling generative adversarial network (SE-GAN) exploiting cross-domain data for semantic segmentation. In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN. Despite its simplicity, we find SE-GAN can significantly boost the performance of adversarial training and enhance the stability of the model.
  arXiv  Detail & Related papers  (2021-12-15T09:50:25Z)
• i-Algebra: Towards Interactive Interpretability of Deep Neural Networks [41.13047686374529]
  We present i-Algebra, a first-of-its-kind interactive framework for interpreting deep neural networks (DNNs) At its core is a library of atomic, composable operators, which explain model behaviors at varying input granularity, during different inference stages, and from distinct interpretation perspectives. We conduct user studies in a set of representative analysis tasks, including inspecting adversarial inputs, resolving model inconsistency, and cleansing contaminated data, all demonstrating its promising usability.
  arXiv  Detail & Related papers  (2021-01-22T19:22:57Z)
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation [51.29486247405601]
  We design a framework to generate counterfactuals for raw data instances with the proposed Attribute-Informed Perturbation (AIP) By utilizing generative models conditioned with different attributes, counterfactuals with desired labels can be obtained effectively and efficiently. Experimental results on real-world texts and images demonstrate the effectiveness, sample quality as well as efficiency of our designed framework.
  arXiv  Detail & Related papers  (2021-01-18T08:37:13Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.