Rally and WebScience: A Platform and Toolkit for Browser-Based Research on Technology and Society Problems

• URL: http://arxiv.org/abs/2211.02274v2
• Date: Wed, 30 Nov 2022 17:20:29 GMT
• Title: Rally and WebScience: A Platform and Toolkit for Browser-Based Research on Technology and Society Problems
• Abstract summary: Deploying studies in participant browsers and devices is a promising direction. Rally and sdkName are a platform and toolkit for browser-based research. We report results from a one-month pilot study on news engagement, analyzing 4,466,200 webpage visits from 1,817 participants.
• Score: 1.4511816961424293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Empirical technology and society research is in a methodological crisis. Problems increasingly involve closed platforms, targeted content, and context-specific behavior. Prevailing research methods, such as surveys, tasks, and web crawls, pose design and ecological validity limitations. Deploying studies in participant browsers and devices is a promising direction. These vantage points can observe individualized experiences and implement UI interventions in real settings. We survey scholarship that uses these methods, annotating 284 sampled papers. Our analysis demonstrates their potential, but also recurring implementation barriers and shortcomings. We then present Rally and sdkName, a platform and toolkit for browser-based research. These systems lower implementation barriers and advance the science of measuring online behavior. Finally, we evaluate Rally and sdkName against our design goals. We report results from a one-month pilot study on news engagement, analyzing 4,466,200 webpage visits from 1,817 participants. We also present observations from interviews with researchers using these systems.

Related papers

• AI for Auto-Research: Roadmap & User Guide [107.0834449839233]
  We present an end-to-end analysis of AI across the complete research lifecycle.<n>We identify a sharp, stage-dependent boundary between reliable assistance and unreliable autonomy.<n>We show that greater automation can obscure rather than eliminate failure modes.
  arXiv  Detail & Related papers  (2026-05-18T17:08:26Z)
• SoK: After Decades of Web Tracker Detection, What's Next? [3.013252147267835]
  We present the most comprehensive meta-science study on web tracker detection by systematizing and synthesizing the available knowledge.<n>We suggest a taxonomy, observe and evaluate trends, propose open research gaps, and recommendations.
  arXiv  Detail & Related papers  (2026-05-04T08:12:06Z)
• Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset [47.98539809308384]
  We analyze the Asta Interaction dataset, a large-scale resource comprising over 200,000 user queries and interaction logs.<n>We characterize query patterns, engagement behaviors, and how usage evolves with experience.<n>We release the anonymized dataset and analysis with a new query taxonomy to inform future designs of real-world AI research assistants.
  arXiv  Detail & Related papers  (2026-02-26T18:40:28Z)
• VIRENA: Virtual Arena for Research, Education, and Democratic Innovation [2.344992278528697]
  VIRENA (Virtual Arena) is a platform that enables controlled experimentation in realistic social media environments.<n>Multiple participants interact simultaneously in realistic replicas of feed-based platforms.<n>Large language model-powered AI agents participate alongside humans with personas and realistic behavior.
  arXiv  Detail & Related papers  (2026-02-12T17:46:52Z)
• ECHO: An Open Research Platform for Evaluation of Chat, Human Behavior, and Outcomes [6.989051035721272]
  ECHO is an open research platform designed to support mixed-method studies of human interaction with both conversational AI systems and Web search engines.<n>It enables researchers from varying disciplines to orchestrate end-to-end experimental that integrate consent and background surveys, chat-based and search-based information-seeking sessions, writing or judgment tasks, and pre-task evaluations within a unified, low-coding-load framework.
  arXiv  Detail & Related papers  (2026-02-10T21:10:38Z)
• SoK: Advances and Open Problems in Web Tracking [71.54586748169943]
  Web tracking is a pervasive and opaque practice that enables personalized advertising, and conversion tracking.<n>Web tracking is undergoing a once-in-a-generation transformation driven by shifts in the advertising industry, the adoption of anti-tracking countermeasures by browsers, and the growing enforcement of emerging privacy regulations.<n>This Systematization of Knowledge (SoK) aims to consolidate and synthesize this wide-ranging research, offering a comprehensive overview of the technical mechanisms, countermeasures, and regulations that shape the modern and rapidly evolving web tracking landscape.
  arXiv  Detail & Related papers  (2025-06-16T23:30:54Z)
• Sentiment Analysis in SemEval: A Review of Sentiment Identification Approaches [1.6874375111244329]
  We study top-ranking systems for each SemEval edition during the 2013-2021 period. We analyze the proposed systems marking the evolution of research trends with a focus on the main components of sentiment analysis systems. Our study shows an active use of preprocessing techniques, an evolution of features engineering and word representation from lexicon-based approaches to word embeddings, and the dominance of neural networks and transformers.
  arXiv  Detail & Related papers  (2025-03-13T15:25:23Z)
• A Survey on Web Testing: On the Rise of AI and Applications in Industry [1.5149438988761574]
  This paper presents a systematic literature survey focusing on web testing methodologies, tools, and trends from 2014 to 2024. Our results show that web testing research has been highly active, with ICST as the leading venue. Selenium is the most widely used tool, while industrial adoption and human studies remain comparatively limited.
  arXiv  Detail & Related papers  (2025-03-07T12:39:59Z)
• A Survey of Stance Detection on Social Media: New Directions and Perspectives [50.27382951812502]
  stance detection has emerged as a crucial subfield within affective computing. Recent years have seen a surge of research interest in developing effective stance detection methods. This paper provides a comprehensive survey of stance detection techniques on social media.
  arXiv  Detail & Related papers  (2024-09-24T03:06:25Z)
• Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
  Ontologies are widely used for representing domain knowledge and meta data. One straightforward solution is to integrate statistical analysis and machine learning. Numerous papers have been published on embedding, but a lack of systematic reviews hinders researchers from gaining a comprehensive understanding of this field.
  arXiv  Detail & Related papers  (2024-06-16T14:49:19Z)
• Cyberbullying Detection: Exploring Datasets, Technologies, and Approaches on Social Media Platforms [3.235558067839701]
  This paper presents a comprehensive systematic review of studies conducted on cyberbullying detection. It explores existing studies, proposed solutions, identified gaps, datasets, technologies, approaches, challenges, and recommendations.
  arXiv  Detail & Related papers  (2024-05-22T04:58:20Z)
• Keeping Teams in the Game: Predicting Dropouts in Online Problem-Based Learning Competition [0.4779196219827506]
  The study employs an online longitudinal problem-based learning (PBL) collaborative robotics competition as the testbed. The study aims to predict dropout behavior via the contributions of Discourse discussion forum 'activities' of participating teams, along with a self-reported Online Learning Strategies Questionnaire (OSLQ) The findings demonstrate the reliability of OSLQ with our substantial sample size and reveal promising results for predicting the dropout rate in online competition.
  arXiv  Detail & Related papers  (2023-12-27T00:08:19Z)
• Inclusion in Virtual Reality Technology: A Scoping Review [0.3626013617212666]
  We provide a scoping analysis of existing virtual reality research literature about inclusion. We categorize the literature based on target group into ability, gender, and age, followed by those that study community-based design of VR experiences. In the latter group, we focus mainly on Indigenous Peoples as a clearer and more important example.
  arXiv  Detail & Related papers  (2023-10-23T18:55:48Z)
• Resilience of Deep Learning applications: a systematic literature review of analysis and hardening techniques [3.265458968159693]
  The review is based on 220 scientific articles published between January 2019 and March 2024. The authors adopt a classifying framework to interpret and highlight research similarities and peculiarities.
  arXiv  Detail & Related papers  (2023-09-27T19:22:19Z)
• Machine Unlearning: A Survey [56.79152190680552]
  A special need has arisen where, due to privacy, usability, and/or the right to be forgotten, information about some specific samples needs to be removed from a model, called machine unlearning. This emerging technology has drawn significant interest from both academics and industry due to its innovation and practicality. No study has analyzed this complex topic or compared the feasibility of existing unlearning solutions in different kinds of scenarios. The survey concludes by highlighting some of the outstanding issues with unlearning techniques, along with some feasible directions for new research opportunities.
  arXiv  Detail & Related papers  (2023-06-06T10:18:36Z)
• Machine Learning for Detection and Mitigation of Web Vulnerabilities and Web Attacks [0.0]
  Cross-site scripting (XSS) and cross-site request forgery (CSRF) have been a great concern in the field of web security. Several ideas have been put forth that can be used to improve the performance of detecting these web vulnerabilities. Machine learning techniques have lately been used by researchers to defend against XSS and CSRF.
  arXiv  Detail & Related papers  (2023-04-27T18:27:26Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• On the Social and Technical Challenges of Web Search Autosuggestion Moderation [118.47867428272878]
  Autosuggestions are typically generated by machine learning (ML) systems trained on a corpus of search logs and document representations. While current search engines have become increasingly proficient at suppressing such problematic suggestions, there are still persistent issues that remain. We discuss several dimensions of problematic suggestions, difficult issues along the pipeline, and why our discussion applies to the increasing number of applications beyond web search.
  arXiv  Detail & Related papers  (2020-07-09T19:22:00Z)
• An Exploration of Embodied Visual Exploration [97.21890864063872]
  Embodied computer vision considers perception for robots in novel, unstructured environments. We present a taxonomy for existing visual exploration algorithms and create a standard framework for benchmarking them. We then perform a thorough empirical study of the four state-of-the-art paradigms using the proposed framework.
  arXiv  Detail & Related papers  (2020-01-07T17:40:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.