Whodunit? Learning to Contrast for Authorship Attribution

• URL: http://arxiv.org/abs/2209.11887v1
• Date: Fri, 23 Sep 2022 23:45:08 GMT
• Title: Whodunit? Learning to Contrast for Authorship Attribution
• Authors: Bo Ai, Yuchen Wang, Yugin Tan, Samson Tan
• Abstract summary: Authorship attribution is the task of identifying the author of a given text. We propose to fine-tune pre-trained language representations using a combination of contrastive learning and supervised learning. We show that Contra-X advances the state-of-the-art on multiple human and machine authorship attribution benchmarks.
• Score: 22.37948005237967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Authorship attribution is the task of identifying the author of a given text. Most existing approaches use manually designed features that capture a dataset's content and style. However, this dataset-dependent approach yields inconsistent performance. Thus, we propose to fine-tune pre-trained language representations using a combination of contrastive learning and supervised learning (Contra-X). We show that Contra-X advances the state-of-the-art on multiple human and machine authorship attribution benchmarks, enabling improvements of up to 6.8%. We also show Contra-X to be consistently superior to cross-entropy fine-tuning across different data regimes. Crucially, we present qualitative and quantitative analyses of these improvements. Our learned representations form highly separable clusters for different authors. However, we find that contrastive learning improves overall accuracy at the cost of sacrificing performance for some authors. Resolving this tension will be an important direction for future work. To the best of our knowledge, we are the first to analyze the effect of combining contrastive learning with cross-entropy fine-tuning for authorship attribution.

Related papers

• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Inferring Latent Class Statistics from Text for Robust Visual Few-Shot Learning [4.300029426596762]
  We present a novel approach that leverages text-derived statistics to predict the mean and covariance of the visual feature distribution for each class. Our method shows that we can use text to predict the mean and covariance of the distribution offering promising improvements in few-shot learning scenarios.
  arXiv  Detail & Related papers  (2023-11-24T15:23:47Z)
• Time Series Contrastive Learning with Information-Aware Augmentations [57.45139904366001]
  A key component of contrastive learning is to select appropriate augmentations imposing some priors to construct feasible positive samples. How to find the desired augmentations of time series data that are meaningful for given contrastive learning tasks and datasets remains an open question. We propose a new contrastive learning approach with information-aware augmentations, InfoTS, that adaptively selects optimal augmentations for time series representation learning.
  arXiv  Detail & Related papers  (2023-03-21T15:02:50Z)
• PART: Pre-trained Authorship Representation Transformer [64.78260098263489]
  Authors writing documents imprint identifying information within their texts: vocabulary, registry, punctuation, misspellings, or even emoji usage. Previous works use hand-crafted features or classification tasks to train their authorship models, leading to poor performance on out-of-domain authors. We propose a contrastively trained model fit to learn textbfauthorship embeddings instead of semantics.
  arXiv  Detail & Related papers  (2022-09-30T11:08:39Z)
• Few-shot Text Classification with Dual Contrastive Consistency [31.141350717029358]
  In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification. We adopt supervised contrastive learning on few labeled data and consistency-regularization on vast unlabeled data.
  arXiv  Detail & Related papers  (2022-09-29T19:26:23Z)
• An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs [67.23285413610243]
  Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models. We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
  arXiv  Detail & Related papers  (2022-05-21T19:49:04Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning [6.6389732792316005]
  We extend the use of contrastive learning embeddings to sentiment analysis tasks. We show that fine-tuning on these embeddings provides an improvement over fine-tuning on BERT-based embeddings.
  arXiv  Detail & Related papers  (2021-12-02T08:29:26Z)
• Constructing Contrastive samples via Summarization for Text Classification with limited annotations [46.53641181501143]
  We propose a novel approach to constructing contrastive samples for language tasks using text summarization. We use these samples for supervised contrastive learning to gain better text representations with limited annotations. Experiments on real-world text classification datasets (Amazon-5, Yelp-5, AG News) demonstrate the effectiveness of the proposed contrastive learning framework.
  arXiv  Detail & Related papers  (2021-04-11T20:13:24Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.