Reputation Gaming in Stack Overflow

• URL: http://arxiv.org/abs/2111.07101v2
• Date: Thu, 1 Aug 2024 09:02:31 GMT
• Title: Reputation Gaming in Stack Overflow
• Authors: Iren Mazloomzadeh, Gias Uddin, Foutse Khomh, Ashkan Sami,
• Abstract summary: This paper offers a comprehensive study of the reported types of reputation manipulation scenarios that might be exercised in Stack Overflow. We found four different types of reputation fraud scenarios, such as voting rings where communities form to upvote each other repeatedly on similar posts. We developed algorithms that enable platform managers to automatically identify these suspicious reputation gaming scenarios for review.
• Score: 10.021057473471236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stack Overflow incentive system awards users with reputation scores to ensure quality. The decentralized nature of the forum may make the incentive system prone to manipulation. This paper offers, for the first time, a comprehensive study of the reported types of reputation manipulation scenarios that might be exercised in Stack Overflow and the prevalence of such reputation gamers by a qualitative study of 1,697 posts from meta Stack Exchange sites. We found four different types of reputation fraud scenarios, such as voting rings where communities form to upvote each other repeatedly on similar posts. We developed algorithms that enable platform managers to automatically identify these suspicious reputation gaming scenarios for review. The first algorithm identifies isolated/semi-isolated communities where probable reputation frauds may occur mostly by collaborating with each other. The second algorithm looks for sudden unusual big jumps in the reputation scores of users. We evaluated the performance of our algorithms by examining the reputation history dashboard of Stack Overflow users from the Stack Overflow website. We observed that around 60-80% of users flagged as suspicious by our algorithms experienced reductions in their reputation scores by Stack Overflow.

Related papers

• Towards Better Answers: Automated Stack Overflow Post Updating [11.85319691188159]
  We introduce a novel framework, named Soup (Stack Overflow Updator for Post) for this task. Soup addresses two key tasks: Valid Comment-Edit Prediction (VCP) and Automatic Post Updating (APU)
  arXiv  Detail & Related papers  (2024-08-17T04:48:53Z)
• Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content [66.71102704873185]
  We test for user strategization by conducting a lab experiment and survey. We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes" Our findings suggest that platforms cannot ignore the effect of their algorithms on user behavior.
  arXiv  Detail & Related papers  (2024-05-09T07:36:08Z)
• A Tale of Two Communities: Exploring Academic References on Stack Overflow [1.2914230269240388]
  We find that Stack Overflow communities with different domains of interest engage with academic literature at varying frequencies and speeds. The contradicting patterns suggest that some disciplines may have diverged in their interests and development trajectories from the corresponding practitioner community.
  arXiv  Detail & Related papers  (2024-03-14T20:33:55Z)
• User Strategization and Trustworthy Algorithms [81.82279667028423]
  We show that user strategization can actually help platforms in the short term. We then show that it corrupts platforms' data and ultimately hurts their ability to make counterfactual decisions.
  arXiv  Detail & Related papers  (2023-12-29T16:09:42Z)
• Incentivizing Combinatorial Bandit Exploration [87.08827496301839]
  Consider a bandit algorithm that recommends actions to self-interested users in a recommendation system. Users are free to choose other actions and need to be incentivized to follow the algorithm's recommendations. While the users prefer to exploit, the algorithm can incentivize them to explore by leveraging the information collected from the previous users.
  arXiv  Detail & Related papers  (2022-06-01T13:46:25Z)
• Integrating Rankings into Quantized Scores in Peer Review [61.27794774537103]
  In peer review, reviewers are usually asked to provide scores for the papers. To mitigate this issue, conferences have started to ask reviewers to additionally provide a ranking of the papers they have reviewed. There are no standard procedure for using this ranking information and Area Chairs may use it in different ways. We take a principled approach to integrate the ranking information into the scores.
  arXiv  Detail & Related papers  (2022-04-05T19:39:13Z)
• Features that Predict the Acceptability of Java and JavaScript Answers on Stack Overflow [5.332217496693262]
  We studied the Stack Overflow dataset by analyzing questions and answers for the two most popular tags (Java and JavaScript) Our findings reveal that the length of code in answers, reputation of users, similarity of the text between questions and answers, and the time lag between questions and answers have the highest predictive power for differentiating accepted and unaccepted answers.
  arXiv  Detail & Related papers  (2021-01-08T03:09:38Z)
• Partial Bandit and Semi-Bandit: Making the Most Out of Scarce Users' Feedback [62.997667081978825]
  We present a novel approach for considering user feedback and evaluate it using three distinct strategies. Despite a limited number of feedbacks returned by users (as low as 20% of the total), our approach obtains similar results to those of state of the art approaches.
  arXiv  Detail & Related papers  (2020-09-16T07:32:51Z)
• Improving Quality of a Post's Set of Answers in Stack Overflow [2.0625936401496237]
  A large number of low-quality posts on Stack Overflow require improvement. We propose an approach to automate the identification process of such posts and boost their set of answers.
  arXiv  Detail & Related papers  (2020-05-30T19:40:19Z)
• Oblivion of Online Reputation: How Time Cues Improve Online Recruitment [77.34726150561087]
  This paper argues that exposing employers to the temporal context of job-seekers' reputation leads to better hiring decisions. An experimental lab study with 335 students shows that current reputation systems fall short of making them aware of obsolete reputation.
  arXiv  Detail & Related papers  (2020-05-13T13:23:05Z)
• The Phantom Steering Effect in Q&A Websites [37.098578930642745]
  Badges are commonly used in online platforms as incentives for promoting contributions. This paper provides a new probabilistic model of user behavior in the presence of badges. We find that steering is not as widely applicable as was previously understood.
  arXiv  Detail & Related papers  (2020-02-14T18:20:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.