Tradeoffs in Preventing Manipulation in Paper Bidding for Reviewer Assignment

• URL: http://arxiv.org/abs/2207.11315v1
• Date: Fri, 22 Jul 2022 19:58:17 GMT
• Title: Tradeoffs in Preventing Manipulation in Paper Bidding for Reviewer Assignment
• Authors: Steven Jecmen, Nihar B. Shah, Fei Fang, Vincent Conitzer
• Abstract summary: Despite the benefits of using bids, reliance on paper bidding can allow malicious reviewers to manipulate the paper assignment for unethical purposes. Several different approaches to preventing this manipulation have been proposed and deployed. In this paper, we enumerate certain desirable properties that algorithms for addressing bid manipulation should satisfy.
• Score: 89.38213318211731
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many conferences rely on paper bidding as a key component of their reviewer assignment procedure. These bids are then taken into account when assigning reviewers to help ensure that each reviewer is assigned to suitable papers. However, despite the benefits of using bids, reliance on paper bidding can allow malicious reviewers to manipulate the paper assignment for unethical purposes (e.g., getting assigned to a friend's paper). Several different approaches to preventing this manipulation have been proposed and deployed. In this paper, we enumerate certain desirable properties that algorithms for addressing bid manipulation should satisfy. We then offer a high-level analysis of various approaches along with directions for future investigation.

Related papers

• On the Detection of Reviewer-Author Collusion Rings From Paper Bidding [71.43634536456844]
  Collusion rings pose a major threat to the peer-review systems of computer science conferences. One approach to solve this problem would be to detect the colluding reviewers from their manipulated bids. No research has yet established that detecting collusion rings is even possible.
  arXiv  Detail & Related papers  (2024-02-12T18:12:09Z)
• A Dataset on Malicious Paper Bidding in Peer Review [84.68308372858755]
  Malicious reviewers strategically bid in order to unethically manipulate the paper assignment. A critical impediment towards creating and evaluating methods to mitigate this issue is the lack of publicly-available data on malicious paper bidding. We release a novel dataset, collected from a mock conference activity where participants were instructed to bid either honestly or maliciously.
  arXiv  Detail & Related papers  (2022-06-24T20:23:33Z)
• Auctions and Peer Prediction for Academic Peer Review [11.413240461538589]
  We propose a novel peer prediction mechanism (H-DIPP) building on recent work in the information elicitation literature. The revenue raised in the submission stage auction is used to pay reviewers based on the quality of their reviews in the reviewing stage.
  arXiv  Detail & Related papers  (2021-08-27T23:47:15Z)
• Making Paper Reviewing Robust to Bid Manipulation Attacks [44.34601846490532]
  Anecdotal evidence suggests that some reviewers bid on papers by "friends" or colluding authors. We develop a novel approach for paper bidding and assignment that is much more robust against such attacks. In addition to being more robust, the quality of our paper review assignments is comparable to that of current, non-robust assignment approaches.
  arXiv  Detail & Related papers  (2021-02-09T21:24:16Z)
• Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments [96.114824979298]
  Three important challenges in conference peer review are maliciously attempting to get assigned to certain papers and "torpedo reviewing" We present a framework that brings all these challenges under a common umbrella and present a (randomized) algorithm for reviewer assignment. Our algorithms can limit the chance that any malicious reviewer gets assigned to their desired paper to 50% while producing assignments with over 90% of the total optimal similarity.
  arXiv  Detail & Related papers  (2020-06-29T23:55:53Z)
• Context-Based Quotation Recommendation [60.93257124507105]
  We propose a novel context-aware quote recommendation system. It generates a ranked list of quotable paragraphs and spans of tokens from a given source document. We conduct experiments on a collection of speech transcripts and associated news articles.
  arXiv  Detail & Related papers  (2020-05-17T17:49:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.