The Phantom Steering Effect in Q&A Websites
- URL: http://arxiv.org/abs/2002.06160v2
- Date: Fri, 21 Aug 2020 14:00:19 GMT
- Title: The Phantom Steering Effect in Q&A Websites
- Authors: Nicholas Hoernle and Gregory Kehne and Ariel D. Procaccia and Kobi Gal
- Abstract summary: Badges are commonly used in online platforms as incentives for promoting contributions.
This paper provides a new probabilistic model of user behavior in the presence of badges.
We find that steering is not as widely applicable as was previously understood.
- Score: 37.098578930642745
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Badges are commonly used in online platforms as incentives for promoting
contributions. It is widely accepted that badges "steer" people's behavior
toward increasing their rate of contributions before obtaining the badge. This
paper provides a new probabilistic model of user behavior in the presence of
badges. By applying the model to data from thousands of users on the Q&A site
Stack Overflow, we find that steering is not as widely applicable as was
previously understood. Rather, the majority of users remain apathetic toward
badges, while still providing a substantial number of contributions to the
site. An interesting statistical phenomenon, termed "Phantom Steering,"
accounts for the interaction data of these users and this may have contributed
to some previous conclusions about steering. Our results suggest that a small
population, approximately 20%, of users respond to the badge incentives.
Moreover, we conduct a qualitative survey of the users on Stack Overflow which
provides further evidence that the insights from the model reflect the true
behavior of the community. We argue that while badges might contribute toward a
suite of effective rewards in an online system, research into other aspects of
reward systems such as Stack Overflow reputation points should become a focus
of the community.
Related papers
- Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback.
The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied.
We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
arXiv Detail & Related papers (2024-04-19T16:45:50Z) - A Lot of Talk and a Badge: An Exploratory Analysis of Personal Achievements in GitHub [13.556531448699284]
GitHub introduced a new element through personal achievements, whereby badges are unlocked and displayed on developers' personal profile pages in recognition of their development activities.
We present an exploratory analysis using mixed methods to study the diffusion of personal badges in GitHub.
We find that most of the developers sampled own at least a badge, but we also observe an increasing number of users who choose to keep their profile private and opt out of displaying badges.
arXiv Detail & Related papers (2023-03-26T12:08:50Z) - Personalizing Intervened Network for Long-tailed Sequential User
Behavior Modeling [66.02953670238647]
Tail users suffer from significantly lower-quality recommendation than the head users after joint training.
A model trained on tail users separately still achieve inferior results due to limited data.
We propose a novel approach that significantly improves the recommendation performance of the tail users.
arXiv Detail & Related papers (2022-08-19T02:50:19Z) - Estimating Topic Exposure for Under-Represented Users on Social Media [25.963970325207892]
This work focuses on highlighting the contributions of the engagers in the observed data.
The first step in behavioral analysis of these users is to find the topics they are exposed to but did not engage with.
We propose a novel framework that aids in identifying these users and estimates their topic exposure.
arXiv Detail & Related papers (2022-08-07T19:37:41Z) - These Deals Won't Last! Longevity, Uniformity and Bias in Product Badge
Assignment in E-Commerce Platforms [5.582405594617256]
We try to answer questions such as: How long does a product retain a badge on a given platform?
We collect longitudinal data from several e-commerce platforms over 45 days, and find that although most of the badges are short-lived, there are several permanent badge assignments.
It is unclear how the badge assignments are done, and we find evidence that highly-rated products are missing out on badges compared to lower quality ones.
arXiv Detail & Related papers (2022-04-26T19:16:34Z) - Reputation Gaming in Stack Overflow [10.021057473471236]
This paper offers a comprehensive study of the reported types of reputation manipulation scenarios that might be exercised in Stack Overflow.
We found four different types of reputation fraud scenarios, such as voting rings where communities form to upvote each other repeatedly on similar posts.
We developed algorithms that enable platform managers to automatically identify these suspicious reputation gaming scenarios for review.
arXiv Detail & Related papers (2021-11-13T11:58:59Z) - Improving Conversational Question Answering Systems after Deployment
using Feedback-Weighted Learning [69.42679922160684]
We propose feedback-weighted learning based on importance sampling to improve upon an initial supervised system using binary user feedback.
Our work opens the prospect to exploit interactions with real users and improve conversational systems after deployment.
arXiv Detail & Related papers (2020-11-01T19:50:34Z) - Causal Understanding of Fake News Dissemination on Social Media [50.4854427067898]
We argue that it is critical to understand what user attributes potentially cause users to share fake news.
In fake news dissemination, confounders can be characterized by fake news sharing behavior that inherently relates to user attributes and online activities.
We propose a principled approach to alleviating selection bias in fake news dissemination.
arXiv Detail & Related papers (2020-10-20T19:37:04Z) - From Asking to Answering: Getting More Involved on Stack Overflow [3.1160359353275346]
This paper investigates issues on Stack Overflow, a popular question and answer community for computer programming.
We document evidence of a "leaky pipeline", specifically that there are many active users on the platform who never post an answer.
We find a user's individual features, such as their tenure, gender, and geographic location, have a significant relationship with their likelihood to post answers.
arXiv Detail & Related papers (2020-10-08T14:41:22Z) - Study of the usability of LinkedIn: a social media platform meant to
connect employers and employees [91.3755431537592]
This paper is assessing LinkedIn's usability using both user and expert evaluation.
The overall usability of LinkedIn application has been measured by using SUS (System Usability Scale)
arXiv Detail & Related papers (2020-06-06T18:19:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.