STEB: Style Text Embedding Benchmark

• URL: http://arxiv.org/abs/2606.31741v1
• Date: Tue, 30 Jun 2026 14:35:02 GMT
• Title: STEB: Style Text Embedding Benchmark
• Abstract summary: Style Text Embedding Benchmark is an open-source benchmark to standardize the evaluation of style embeddings.<n>StEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of linguistic features.<n>We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated.
• Score: 1.4783646973333087
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of linguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. We open-source the STEB code base at: https://github.com/rrivera1849/STEB.

Related papers

• Skeletons Matter: Dynamic Data Augmentation for Text-to-Query [66.52311036179294]
  We formally define the Text-to-Query task paradigm, unifying semantic parsing tasks across various query languages.<n>We identify query skeletons as a shared optimization target of Text-to-Query tasks, and propose a general dynamic data augmentation framework.<n> Experiments on four Text-to-Query benchmarks demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-24T09:39:03Z)
• Text-based Aerial-Ground Person Retrieval [55.31140361809554]
  This work introduces Text-based Aerial-Ground Person Retrieval (TAG-PR)<n>It aims to retrieve person images from heterogeneous aerial and ground views with textual descriptions.
  arXiv  Detail & Related papers  (2025-11-11T15:49:04Z)
• Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings [25.966475857117175]
  In this work, we introduce ConTEB, a benchmark designed to evaluate retrieval models on their ability to leverage document-wide context.<n>Our results show that state-of-the-art embedding models struggle in retrieval scenarios where context is required.<n>We propose InSeNT, a novel contrastive post-training approach which combined with late chunking pooling enhances contextual representation learning.
  arXiv  Detail & Related papers  (2025-05-30T16:43:28Z)
• WritingBench: A Comprehensive Benchmark for Generative Writing [87.48445972563631]
  We present WritingBench, a benchmark designed to evaluate large language models (LLMs) across 6 core writing domains and 100, encompassing creative, persuasive, informative, and technical writing.<n>We propose a query-dependent evaluation framework that empowers LLMs to dynamically generate instance-specific assessment criteria.<n>This framework is complemented by a fine-tuned critic model for criteria-aware scoring, enabling evaluations in style, format and length.
  arXiv  Detail & Related papers  (2025-03-07T08:56:20Z)
• FaMTEB: Massive Text Embedding Benchmark in Persian Language [9.204800002382042]
  This paper introduces a comprehensive benchmark for Persian (Farsi) text embeddings built upon the Massive Text Embedding Benchmark (MTEB)<n>Our benchmark includes 63 datasets spanning seven different tasks.<n>We evaluate the performance of several Persian and multilingual embedding models in a range of tasks.
  arXiv  Detail & Related papers  (2025-02-17T09:05:21Z)
• PLATTER: A Page-Level Handwritten Text Recognition System for Indic Scripts [20.394597266150534]
  We present an end-to-end framework for Page-Level hAndwriTTen TExt Recognition (PLATTER)<n> Secondly, we demonstrate the usage of PLATTER to measure the performance of our language-agnostic HTD model.<n>Finally, we release a Corpus of Handwritten Indic Scripts (CHIPS), a meticulously curated, page-level Indic handwritten OCR dataset.
  arXiv  Detail & Related papers  (2025-02-10T05:50:26Z)
• Discourse Features Enhance Detection of Document-Level Machine-Generated Content [53.41994768824785]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>Existing MGC detectors often focus solely on surface-level information, overlooking implicit and structural features.<n>We introduce novel methodologies and datasets to overcome these challenges.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• MTEB: Massive Text Embedding Benchmark [6.023518635799927]
  It is unclear whether state-of-the-art embeddings on semantic textual similarity can be equally well applied to other tasks like clustering or reranking. Massive Text Embedding Benchmark (MTEB) spans 8 embedding tasks covering a total of 58 datasets and 112 languages.
  arXiv  Detail & Related papers  (2022-10-13T19:42:08Z)
• DeepStyle: User Style Embedding for Authorship Attribution of Short Texts [57.503904346336384]
  Authorship attribution (AA) is an important and widely studied research topic with many applications. Recent works have shown that deep learning methods could achieve significant accuracy improvement for the AA task. We propose DeepStyle, a novel embedding-based framework that learns the representations of users' salient writing styles.
  arXiv  Detail & Related papers  (2021-03-14T15:56:37Z)
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation [50.01708049531156]
  We focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer. In detail, the input is a set of structured records and a reference text for describing another recordset. The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference.
  arXiv  Detail & Related papers  (2020-02-24T12:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.