Perceive, Refine, Reason: A Calibrated Pipeline for Measuring Indicators in Strategic Visual Communication on Social Media

• URL: http://arxiv.org/abs/2609.14699v1
• Date: Sun, 13 Sep 2026 17:54:01 GMT
• Title: Perceive, Refine, Reason: A Calibrated Pipeline for Measuring Indicators in Strategic Visual Communication on Social Media
• Abstract summary: We introduce Perceive, Refine, Reason (PRR), a calibrated pipeline that turns flexible vision-language detectors into auditable measurement instruments for social-scientific research.<n>PRR combines natural-language category prompts with pixel-level spatial refinement via the Segment Anything Model (SAM)<n>A complementary three-tier auditability framework applies learning to profile per-category reliability, support task-aligned configuration, and statistically correct prevalence estimates.
• Score: 2.927717169939877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual content shapes audience perception and opinion on social media, and computational social science increasingly relies on automated tools to analyze images at scale. Yet a measurement gap persists: existing tools rely on predefined categories or produce only coarse image-level labels, while measuring which specific objects appear in an image, how prominently, and where in the frame remains difficult at scale. We introduce Perceive, Refine, Reason (PRR), a calibrated pipeline that turns flexible vision-language detectors into auditable measurement instruments for social-scientific research. PRR combines natural-language category prompts with pixel-level spatial refinement via the Segment Anything Model (SAM) and a multimodal LLM arbitration layer whose reasoning chains externalize domain knowledge and lower the expertise threshold for human-in-the-loop validation. A complementary three-tier auditability framework applies quantification learning to profile per-category reliability, support task-aligned configuration, and statistically correct prevalence estimates. Across four vision-language detectors and nine sociological categories, the pipeline yields substantial precision gains over zero-shot baselines, including a 43.3-point improvement for the strongest backbone. Applying PRR to 103,920 Facebook images from U.S. legislators during the 2024 election cycle and linking detections to DW-NOMINATE ideology scores, we find that more conservative legislators display U.S. flags as larger visual elements, with a weaker tendency toward peripheral placement, a spatial pattern invisible to binary detection. PRR provides computational social scientists with a model-agnostic toolkit for accessible, spatially-grounded, and correctable visual measurement.

Related papers

• Latent Fact-Checking: Detecting Misinformation through Activation Engineering [1.3299946892361476]
  We introduce a misinformation detection framework grounded in activation engineering.<n>Our approach elicits a misinformation direction in the residual stream by contrasting activations from paired truthful and false statements.<n>We evaluate the method across 11 models on three fact-checking benchmarks: AVeriTeC, LIAR, and FACTors.
  arXiv  Detail & Related papers  (2026-08-05T18:00:14Z)
• Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models [61.89854422423702]
  Vision language models (VLMs) excel at many tasks but struggle with spatial reasoning when critical information is not directly observable.<n>We introduce Imaginative Perception Tokens (IPT), intermediate perceptual representations that externalize what a VLM would perceive under alternative spatial configurations.<n>IPT supervision consistently improves spatial reasoning and often outperforms textual chain of thought training.
  arXiv  Detail & Related papers  (2026-06-02T17:59:17Z)
• Beyond Activation Alignment: The Geometry of Neural Sensitivity [0.0]
  We introduce a framework based on local decodable information, which focuses on a representation's ability to discriminate small perturbations.<n>We compare these regularized signatures using a log-spectral distance on the manifold of symmetric positive definite (SPD) matrices.<n> Empirically, this framework enables the recovery of corresponding layers across independently trained artificial neural networks.
  arXiv  Detail & Related papers  (2026-05-04T23:21:24Z)
• ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization [62.03035862528452]
  ForgeryVCR is a framework that materializes imperceptible traces into explicit visual intermediates via Visual-Centric Reasoning.<n>ForgeryVCR achieves state-of-the-art (SOTA) performance in both detection and localization tasks.
  arXiv  Detail & Related papers  (2026-02-15T11:14:47Z)
• WESR: Scaling and Evaluating Word-level Event-Speech Recognition [59.21814194620928]
  Speech conveys not only linguistic information but also rich non-verbal vocal events such as laughing and crying.<n>We develop a refined taxonomy of 21 vocal events, with a new categorization into discrete (standalone) versus continuous (mixed with speech) types.<n>Based on the refined taxonomy, we introduce WESR-Bench, an expert-annotated evaluation set (900+ utterances) with a novel position-aware protocol.
  arXiv  Detail & Related papers  (2026-01-08T02:23:21Z)
• Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench [4.095423692230828]
  Measure-Bench is a benchmark on visual measurement reading covering both real-world and synthesized images of various types of measurements.<n>Our pipeline procedurally generates a specified type of gauge with controllable visual appearance.
  arXiv  Detail & Related papers  (2025-10-30T17:20:51Z)
• Reading a Ruler in the Wild [1.4785540163232234]
  Accurately converting pixel measurements into absolute real-world dimensions remains a fundamental challenge in computer vision.<n>We introduce RulerNet, a deep learning framework that robustly infers scale "in the wild"<n> Experiments show that RulerNet delivers accurate, consistent, and efficient scale estimates under challenging real-world conditions.
  arXiv  Detail & Related papers  (2025-07-09T17:35:58Z)
• VOCAL: Visual Odometry via ContrAstive Learning [25.50526976366299]
  We introduce VOCAL (Visual Odometry via ContrAstive Learning), a novel framework that reimagines visual odometry as a label ranking challenge.<n>By integrating Bayesian inference with a representation learning framework, VOCAL organizes visual features to mirror camera states.<n>This strategic alignment bolsters the interpretability of the learned features and ensures compatibility with multimodal data sources.
  arXiv  Detail & Related papers  (2025-06-30T20:26:13Z)
• Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs [65.93003087656754]
  VisFactor is a benchmark that digitizes 20 vision-centric subtests from a well-established cognitive psychology assessment.<n>We evaluate 20 frontier Multimodal Large Language Models (MLLMs) from GPT, Gemini, Claude, LLaMA, Qwen, and SEED families.<n>The best-performing model achieves a score of only 25.19 out of 100, with consistent failures on tasks such as mental rotation, spatial relation inference, and figure-ground discrimination.
  arXiv  Detail & Related papers  (2025-02-23T04:21:32Z)
• Metrics reloaded: Recommendations for image analysis validation [59.60445111432934]
  Metrics Reloaded is a comprehensive framework guiding researchers in the problem-aware selection of metrics. The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint. Based on the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics.
  arXiv  Detail & Related papers  (2022-06-03T15:56:51Z)
• Inter-class Discrepancy Alignment for Face Recognition [55.578063356210144]
  We propose a unified framework calledInter-class DiscrepancyAlignment(IDA) IDA-DAO is used to align the similarity scores considering the discrepancy between the images and its neighbors. IDA-SSE can provide convincing inter-class neighbors by introducing virtual candidate images generated with GAN.
  arXiv  Detail & Related papers  (2021-03-02T08:20:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.