Towards Detecting AI-Assisted Responses in Online Surveys

• URL: http://arxiv.org/abs/2609.17317v1
• Date: Tue, 15 Sep 2026 15:27:04 GMT
• Title: Towards Detecting AI-Assisted Responses in Online Surveys
• Abstract summary: We introduce an initial benchmark dataset, namely ASURRE, for AI-assisted survey participation.<n>We show that naive AI usage is readily detectable, whereas persona-grounded agents that mimic entire respondents push detector performance toward chance.
• Score: 10.251117966654645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The use of LLMs to complete online surveys impacts the validity of survey-based research, but detecting such usage remains underexplored. We introduce an initial benchmark dataset, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion. Controlled by these strategies, LLM-assisted survey responses are generated using multiple LLMs on three real-world surveys in different disciplines, paired with genuine human responses. Our evaluation of existing machine-generated text (MGT) detectors shows that naive AI usage is readily detectable, whereas persona-grounded agents that mimic entire respondents push detector performance toward chance. We further show that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces. While individual cues can be circumvented by targeted prompting, a simple few-shot, training-free aggregator over these cues improves mean AUROC by +0.14 over the best existing detector across agentic settings. Our project is available at https://github.com/mike-qz-wang/ASURRE.

Related papers

• A Penny for Your Prompts: Experiments Detecting and Mitigating LLM Usage by Survey Respondents [1.6643984790651822]
  Large language models are increasingly used by participants on crowdsourcing platforms when responding to surveys.<n>Our study aims to quantify the prevalence of this behavior and investigate methods to detect and prevent it.
  arXiv  Detail & Related papers  (2026-07-01T04:01:26Z)
• Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening [101.09427874773901]
  We present the first systematic study of prompt-injection attacks in a widely used application: LLM-based resume screening.<n>Our analysis is based on approximately 200K real-world resumes collected over multiple years by hireEZ.
  arXiv  Detail & Related papers  (2026-05-27T18:56:19Z)
• Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content [2.7048403457282633]
  Large language models (LLMs) are increasingly used by students to generate natural language responses and program code.<n>We investigate the extent to which LLMs can detect their own generated content across multiple educational task types.
  arXiv  Detail & Related papers  (2026-05-13T13:38:40Z)
• Learning to Retrieve from Agent Trajectories [72.8923565916533]
  We argue that retrieval models for agentic search should be trained directly from agent interaction data.<n>We introduce learning to retrieve from agent trajectories as a new training paradigm, where supervision is derived from multi-step agent interactions.<n>Our results highlight agent trajectories as a practical and scalable supervision source, pointing to a promising direction for retrieval in the era of agentic search.
  arXiv  Detail & Related papers  (2026-03-30T17:59:02Z)
• Transparent Reference-free Automated Evaluation of Open-Ended User Survey Responses [7.295969279816647]
  Open-ended survey responses provide valuable insights in marketing research.<n>Low-quality responses not only burden researchers with manual filtering but also risk leading to misleading conclusions.<n>We propose a two-stage evaluation framework specifically designed for human survey responses.
  arXiv  Detail & Related papers  (2025-10-03T08:37:33Z)
• Learning from Convenience Samples: A Case Study on Fine-Tuning LLMs for Survey Non-response in the German Longitudinal Election Study [0.6104510780984732]
  We fine-tune large language models to impute self-reported vote choice under both random and systematic nonresponse.<n>LLMs can recover both individual-level predictions and population-level distributions more accurately than zero-shot.<n>This suggests fine-tuned LLMs offer a promising strategy for researchers working with non-probability samples or systematic missingness.
  arXiv  Detail & Related papers  (2025-09-29T17:12:18Z)
• DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Models [60.713908578319256]
  We propose Direct Discrepancy Learning (DDL) to optimize the detector with task-oriented knowledge.<n>Built upon this, we introduce DetectAnyLLM, a unified detection framework that achieves state-of-the-art MGTD performance.<n>MIRAGE samples human-written texts from 10 corpora across 5 text-domains, which are then re-generated or revised using 17 cutting-edge LLMs.
  arXiv  Detail & Related papers  (2025-09-15T10:59:57Z)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents [66.29263282311258]
  We introduce a new benchmark AgentDAM that measures if AI web-navigation agents follow the privacy principle of data minimization''<n>Our benchmark simulates realistic web interaction scenarios end-to-end and is adaptable to all existing web navigation agents.
  arXiv  Detail & Related papers  (2025-03-12T19:30:31Z)
• LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing [0.0]
  We investigate whether providing respondents' prior information can replicate both statistical distributions and individual decision-making patterns.<n>We also introduce the concept of the LLM-Mirror, user personas generated by supplying respondent-specific information to the LLM.<n>Our findings show that: (1) PLS-SEM analysis shows LLM-generated responses align with human responses, (2) LLMs are capable of reproducing individual human responses, and (3) LLM-Mirror responses closely follow human responses at the individual level.
  arXiv  Detail & Related papers  (2024-12-04T09:39:56Z)
• MALT: Improving Reasoning with Multi-Agent LLM Training [67.76186488361685]
  MALT (Multi-Agent LLM Training) is a novel post-training strategy that divides the reasoning process into generation, verification, and refinement steps.<n>On MATH, GSM8K, and CSQA, MALT surpasses the same baseline LLM with a relative improvement of 15.66%, 7.42%, and 9.40% respectively.
  arXiv  Detail & Related papers  (2024-12-02T19:30:36Z)
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection [74.51523859064802]
  We introduce a new framework called Self-Reflective Retrieval-Augmented Generation (Self-RAG) Self-RAG enhances an LM's quality and factuality through retrieval and self-reflection. It significantly outperforms state-of-the-art LLMs and retrieval-augmented models on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-10-17T18:18:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.