PART: Pre-trained Authorship Representation Transformer

• URL: http://arxiv.org/abs/2209.15373v2
• Date: Fri, 09 May 2025 09:18:49 GMT
• Title: PART: Pre-trained Authorship Representation Transformer
• Authors: Javier Huertas-Tato, Alejandro Martin, David Camacho,
• Abstract summary: Authors writing documents imprint identifying information within their texts.<n>Previous works use hand-crafted features or classification tasks to train their authorship models.<n>We propose a contrastively trained model fit to learn textbfauthorship embeddings instead of semantics.
• Score: 52.623051272843426
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Authors writing documents imprint identifying information within their texts: vocabulary, registry, punctuation, misspellings, or even emoji usage. Previous works use hand-crafted features or classification tasks to train their authorship models, leading to poor performance on out-of-domain authors. Using stylometric representations is more suitable, but this by itself is an open research challenge. In this paper, we propose PART, a contrastively trained model fit to learn \textbf{authorship embeddings} instead of semantics. We train our model on ~1.5M texts belonging to 1162 literature authors, 17287 blog posters and 135 corporate email accounts; a heterogeneous set with identifiable writing styles. We evaluate the model on current challenges, achieving competitive performance. We also evaluate our model on test splits of the datasets achieving zero-shot 72.39\% accuracy when bounded to 250 authors, a 54\% and 56\% higher than RoBERTa embeddings. We qualitatively assess the representations with different data visualizations on the available datasets, observing features such as gender, age, or occupation of the author.

Related papers

• A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution [57.309390098903]
  Authorship attribution aims to identify the origin or author of a document. Large Language Models (LLMs) with their deep reasoning capabilities and ability to maintain long-range textual associations offer a promising alternative. Our results on the IMDb and blog datasets show an impressive 85% accuracy in one-shot authorship classification across ten authors.
  arXiv  Detail & Related papers  (2024-10-29T04:14:23Z)
• BookWorm: A Dataset for Character Description and Analysis [59.186325346763184]
  We define two tasks: character description, which generates a brief factual profile, and character analysis, which offers an in-depth interpretation. We introduce the BookWorm dataset, pairing books from the Gutenberg Project with human-written descriptions and analyses. Our findings show that retrieval-based approaches outperform hierarchical ones in both tasks.
  arXiv  Detail & Related papers  (2024-10-14T10:55:58Z)
• Capturing Style in Author and Document Representation [4.323709559692927]
  We propose a new architecture that learns embeddings for both authors and documents with a stylistic constraint. We evaluate our method on three datasets: a literary corpus extracted from the Gutenberg Project, the Blog Authorship and IMDb62.
  arXiv  Detail & Related papers  (2024-07-18T10:01:09Z)
• Understanding writing style in social media with a supervised contrastively pre-trained transformer [57.48690310135374]
  Online Social Networks serve as fertile ground for harmful behavior, ranging from hate speech to the dissemination of disinformation. We introduce the Style Transformer for Authorship Representations (STAR), trained on a large corpus derived from public sources of 4.5 x 106 authored texts. Using a support base of 8 documents of 512 tokens, we can discern authors from sets of up to 1616 authors with at least 80% accuracy.
  arXiv  Detail & Related papers  (2023-10-17T09:01:17Z)
• Self-Supervised Representation Learning for Online Handwriting Text Classification [0.8594140167290099]
  We propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages. To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods. The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification.
  arXiv  Detail & Related papers  (2023-10-10T14:07:49Z)
• Can Authorship Representation Learning Capture Stylistic Features? [5.812943049068866]
  We show that representations learned for a surrogate authorship prediction task are indeed sensitive to writing style. As a consequence, authorship representations may be expected to be robust to certain kinds of data shift, such as topic drift over time. Our findings may open the door to downstream applications that require stylistic representations, such as style transfer.
  arXiv  Detail & Related papers  (2023-08-22T15:10:45Z)
• How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning [23.274139396706264]
  Recent advancements in Deep Learning-based Handwritten Text Recognition (HTR) have led to models with remarkable performance on modern and historical manuscripts. Those models struggle to obtain the same performance when applied to manuscripts with peculiar characteristics, such as language, paper support, ink, and author handwriting. In this paper, we take into account large, real benchmark datasets and synthetic ones obtained with a styled Handwritten Text Generation model. We give a quantitative indication of the most relevant characteristics of such data for obtaining an HTR model able to effectively transcribe manuscripts in small collections with as little as five real fine-tuning lines
  arXiv  Detail & Related papers  (2023-05-04T07:00:28Z)
• Towards Writing Style Adaptation in Handwriting Recognition [0.0]
  We explore models with writer-dependent parameters which take the writer's identity as an additional input. We propose a Writer Style Block (WSB), an adaptive instance normalization layer conditioned on learned embeddings of the partitions. We show that our approach outperforms a baseline with no WSB in a writer-dependent scenario and that it is possible to estimate embeddings for new writers.
  arXiv  Detail & Related papers  (2023-02-13T12:36:17Z)
• Cloning Ideology and Style using Deep Learning [0.0]
  Research focuses on text generation based on the ideology and style of a specific author, and text generation on a topic that was not written by the same author in the past. Bi-LSTM model is used to make predictions at the character level, during the training corpus of a specific author is used along with the ground truth corpus. A pre-trained model is used to identify the sentences of ground truth having contradiction with the author's corpus to make our language model inclined.
  arXiv  Detail & Related papers  (2022-10-25T11:37:19Z)
• Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters [66.17039929803933]
  We propose a novel transfer learning framework which updates only $0.3%$ of model parameters to learn style specific attributes for response generation. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset.
  arXiv  Detail & Related papers  (2022-10-07T00:09:22Z)
• Sentiment analysis in tweets: an assessment study from classical to modern text representation models [59.107260266206445]
  Short texts published on Twitter have earned significant attention as a rich source of information. Their inherent characteristics, such as the informal, and noisy linguistic style, remain challenging to many natural language processing (NLP) tasks. This study fulfils an assessment of existing language models in distinguishing the sentiment expressed in tweets by using a rich collection of 22 datasets.
  arXiv  Detail & Related papers  (2021-05-29T21:05:28Z)
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation [50.01708049531156]
  We focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer. In detail, the input is a set of structured records and a reference text for describing another recordset. The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference.
  arXiv  Detail & Related papers  (2020-02-24T12:52:10Z)
• Authorship Attribution in Bangla literature using Character-level CNN [0.5243460995467893]
  We investigate the effectiveness of character-level signals in Authorship Attribution of Bangla Literature. Time and memory efficiency of the proposed model is much higher than the word level counterparts. It is seen that the performance is improved by up to 10% on pre-training.
  arXiv  Detail & Related papers  (2020-01-11T14:54:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.