The Long Tail, Not the Front Page: Cold-Start Prediction of Crowd Highlight Salience

• URL: http://arxiv.org/abs/2606.11654v1
• Date: Wed, 10 Jun 2026 04:37:23 GMT
• Title: The Long Tail, Not the Front Page: Cold-Start Prediction of Crowd Highlight Salience
• Abstract summary: A social highlighter's most useful signal -- which passages a crowd of readers marks -- exists only for documents people have already read.<n>Can the aggregate crowd salience of a document be predicted from its text before its marks accumulate?<n>Prior work on this data found that zero-shot language models recover highlight locations worse than a trivial lead (position) baseline.<n>We ask whether a model trained on the highlight corpus can beat that baseline.
• Score: 1.8620637029128544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A social highlighter's most useful signal -- which passages a crowd of readers marks -- exists only for documents people have already read. Can the aggregate crowd salience of a document be predicted from its text before its marks accumulate? Prior work on this data found that zero-shot language models recover highlight locations worse than a trivial lead (position) baseline, so we ask whether a model trained on the highlight corpus can beat that baseline. Using a pre-registered ladder of models and a by-document cluster bootstrap, we find a small but robust edge: a logistic ranker over sentence embeddings and positional/contextual features beats the lead baseline by +0.044 average precision (95% CI [+0.029, +0.058]; clears a pre-registered margin delta=0.03 in 97% of resamples, and stable across pipeline re-runs). Two unsupervised extractive baselines (centroid, LexRank-style centrality) lose to lead, and the trained model beats them by +0.108, so the edge is not recovered by generic unsupervised proxies -- it reflects learning from real reader marks. In product terms, precision@3 rises from 0.25 to 0.39 (+55% relative) and the model beats lead on 69% of documents. An ablation attributes the edge to the raw embedding (+0.014) and training augmentation (+0.010), each with a positive CI. The edge is not a temporal-generalization failure, and we find no evidence that content drift or near-duplicate leakage explains it. A standardized regression shows the advantage is governed mainly by document popularity (lower popularity, larger edge) and by label reliability. It nearly vanishes only on the most popular content; there it is the lead baseline that strengthens, not the model that weakens. Because our evaluation conditions on documents that eventually accumulated readers, these results are a retrospective cold-start simulation.

Related papers

• Floor, Ceiling, and the Fusion Gap: How Much of Crowd Reading Attention Can Machines Predict? [1.8620637029128544]
  benchmark score means nothing without knowing what a trivial method achieves and what the best possible method could achieve.<n>We construct both bounds for a task with a rare kind of ground truth: predicting which sentences a crowd of readers -- highlighting for their own purposes, unpaid, uninstructed, and blind to each other -- marked in 120 web documents.
  arXiv  Detail & Related papers  (2026-08-03T05:09:49Z)
• Measuring Alignment With Reader Highlights Net of Position and Length [1.8620637029128544]
  On 120 web documents (at least 12 independent readers each), a language-model importance ranking keeps 38.4% of crowd-marked sentences against 19.9% of their matched neighbours.<n>A single human reader reaches +0.182 - indistinguishable from GPT-5.4 (+0.002 [-0.081, +0.088]) and below 5.
  arXiv  Detail & Related papers  (2026-07-30T06:24:38Z)
• Trait, Not State: The Durability of Reading Identity in Social Highlighting [1.8620637029128544]
  We ask the temporal question: is a reader's selection signature a trait or a state?<n>We freeze each reader's first six months of highlighting as a profile.<n>We track its own-vs-other advantage on its later selections at growing gaps.
  arXiv  Detail & Related papers  (2026-06-11T04:59:37Z)
• Factions Within, Uncertain Across: Within-Document Reader Sub-Groups in Social Highlighting [1.8620637029128544]
  We ask the group-level question on a co-readership platform using a margin-preserving curveball null.<n>Experiment 1 shows readers form strong sub-groups far beyond what shared salience, mark density, and sentence popularity predict.<n>Experiment 2 is that grouping a stable reader trait? Here we are honest about power.
  arXiv  Detail & Related papers  (2026-06-10T03:25:36Z)
• Selection, Not Salience: The Shape and Limits of Personalization in Social Highlighting [1.8620637029128544]
  We map the shape and limits of personalization across reading altitudes.<n>Measurable personalization appears primarily at the selection layer.<n>Going beyond the shared salience layer may be better approached by aggregating individuals.
  arXiv  Detail & Related papers  (2026-06-09T04:18:08Z)
• NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models [0.0]
  NumLeak is a measurement framework that combines API-boundary probes on production models with a white-box controlled validation on an open causal LM.<n>Top-tier frontier LLMs recall the Fama-French market excess return at 3-seed pooled Pearson r=0.97-0.99 while staying within 0.15 within-25bps on the five sibling factors.
  arXiv  Detail & Related papers  (2026-05-28T12:52:49Z)
• RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems [6.083097040417168]
  We introduce LR-bench, a benchmark curated from 2024-2025 AI/NLP manuscripts with five-level self-assessed familiarity ratings.<n>We also propose RATE, a reviewer-centric ranking framework that distills each reviewer's recent publications into compact keyword-based profiles.<n>Our approach consistently achieves state-of-the-art performance, outperforming strong embedding baselines by a clear margin.
  arXiv  Detail & Related papers  (2026-01-27T14:13:46Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence [56.09494651178128]
  Retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG)<n>We quantify the impact of biases, such as a preference for shorter documents, on retrievers like Dragon+ and Contriever.<n>We uncover major vulnerabilities, showing retrievers favor shorter documents, early positions, repeated entities, and literal matches, all while ignoring the answer's presence!
  arXiv  Detail & Related papers  (2025-03-06T23:23:13Z)
• Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
  We propose a novel method dubbed ShrinkMatch to learn uncertain samples. For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class. We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
  arXiv  Detail & Related papers  (2023-08-13T14:05:24Z)
• Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation [56.57532238195446]
  We propose a method named Ethicist for targeted training data extraction. To elicit memorization, we tune soft prompt embeddings while keeping the model fixed. We show that Ethicist significantly improves the extraction performance on a recently proposed public benchmark.
  arXiv  Detail & Related papers  (2023-07-10T08:03:41Z)
• On the Blind Spots of Model-Based Evaluation Metrics for Text Generation [79.01422521024834]
  We explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics. We design and synthesize a wide range of potential errors and check whether they result in a commensurate drop in the metric scores. Our experiments reveal interesting insensitivities, biases, or even loopholes in existing metrics.
  arXiv  Detail & Related papers  (2022-12-20T06:24:25Z)
• Out-of-Vocabulary Entities in Link Prediction [1.9036571490366496]
  Link prediction is often used as a proxy to evaluate the quality of embeddings. As benchmarks are crucial for the fair comparison of algorithms, ensuring their quality is tantamount to providing a solid ground for developing better solutions. We provide an implementation of an approach for spotting and removing such entities and provide corrected versions of the datasets WN18RR, FB15K-237, and YAGO3-10.
  arXiv  Detail & Related papers  (2021-05-26T12:58:18Z)
• Negative Margin Matters: Understanding Margin in Few-shot Classification [72.85978953262004]
  This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and state-of-the-art accuracy on three standard few-shot classification benchmarks.
  arXiv  Detail & Related papers  (2020-03-26T17:59:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.