Evolving Methods for Evaluating and Disseminating Computing Research

• URL: http://arxiv.org/abs/2007.01242v1
• Date: Thu, 2 Jul 2020 16:50:28 GMT
• Title: Evolving Methods for Evaluating and Disseminating Computing Research
• Authors: Benjamin Zorn, Tom Conte, Keith Marzullo, and Suresh Venkatasubramanian
• Abstract summary: Social and technical trends have significantly changed methods for evaluating and disseminating computing research. Traditional venues for reviewing and publishing, such as conferences and journals, worked effectively in the past. Many conferences have seen large increases in the number of submissions. Dis dissemination of research ideas has become dramatically through publication venues such as arXiv.org and social media networks.
• Score: 4.0318506932466445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social and technical trends have significantly changed methods for evaluating and disseminating computing research. Traditional venues for reviewing and publishing, such as conferences and journals, worked effectively in the past. Recently, trends have created new opportunities but also put new pressures on the process of review and dissemination. For example, many conferences have seen large increases in the number of submissions. Likewise, dissemination of research ideas has become dramatically through publication venues such as arXiv.org and social media networks. While these trends predate COVID-19, the pandemic could accelerate longer term changes. Based on interviews with leading academics in computing research, our findings include: (1) Trends impacting computing research are largely positive and have increased the participation, scope, accessibility, and speed of the research process. (2) Challenges remain in securing the integrity of the process, including addressing ways to scale the review process, avoiding attempts to misinform or confuse the dissemination of results, and ensuring fairness and broad participation in the process itself. Based on these findings, we recommend: (1) Regularly polling members of the computing research community, including program and general conference chairs, journal editors, authors, reviewers, etc., to identify specific challenges they face to better understand these issues. (2) An influential body, such as the Computing Research Association regularly issues a "State of the Computing Research Enterprise" report to update the community on trends, both positive and negative, impacting the computing research enterprise. (3) A deeper investigation, specifically to better understand the influence that social media and preprint archives have on computing research, is conducted.

Related papers

• SMP Challenge: An Overview and Analysis of Social Media Prediction Challenge [63.311045291016555]
  Social Media Popularity Prediction (SMPP) is a crucial task that involves automatically predicting future popularity values of online posts. This paper summarizes the challenging task, data, and research progress.
  arXiv  Detail & Related papers  (2024-05-17T02:36:14Z)
• Academic competitions [61.592427413342975]
  This chapter provides a survey of academic challenges in the context of machine learning and related fields. We review the most influential competitions in the last few years and analyze challenges per area of knowledge. The aims of scientific challenges, their goals, major achievements and expectations for the next few years are reviewed.
  arXiv  Detail & Related papers  (2023-12-01T01:01:04Z)
• "That's important, but...": How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations [12.947525301829835]
  We show that considering unintended consequences is generally seen as important but rarely practiced. Principal barriers are a lack of formal process and strategy as well as the academic practice that prioritizes fast progress and publications. We intend for our work to pave the way for routine explorations of the societal implications of technological innovations before, during, and after the research process.
  arXiv  Detail & Related papers  (2023-03-27T18:21:29Z)
• Research Trends and Applications of Data Augmentation Algorithms [77.34726150561087]
  We identify the main areas of application of data augmentation algorithms, the types of algorithms used, significant research trends, their progression over time and research gaps in data augmentation literature. We expect readers to understand the potential of data augmentation, as well as identify future research directions and open questions within data augmentation research.
  arXiv  Detail & Related papers  (2022-07-18T11:38:32Z)
• Fairness in Recommender Systems: Research Landscape and Future Directions [119.67643184567623]
  We review the concepts and notions of fairness that were put forward in the area in the recent past. We present an overview of how research in this field is currently operationalized. Overall, our analysis of recent works points to certain research gaps.
  arXiv  Detail & Related papers  (2022-05-23T08:34:25Z)
• Yes-Yes-Yes: Donation-based Peer Reviewing Data Collection for ACL Rolling Review and Beyond [58.71736531356398]
  We present an in-depth discussion of peer reviewing data, outline the ethical and legal desiderata for peer reviewing data collection, and propose the first continuous, donation-based data collection workflow. We report on the ongoing implementation of this workflow at the ACL Rolling Review and deliver the first insights obtained with the newly collected data.
  arXiv  Detail & Related papers  (2022-01-27T11:02:43Z)
• Identifying Causal Influences on Publication Trends and Behavior: A Case Study of the Computational Linguistics Community [10.791197825505755]
  We present mixed-method analyses to investigate causal influences of publication trends and behavior. Key findings highlight the transition to rapidly emerging methodologies in the research community. We anticipate this work to provide useful insights about publication trends and behavior.
  arXiv  Detail & Related papers  (2021-10-15T08:36:13Z)
• Learning to Drive on the Wrong Side of the Road: How American Computing Came to Rely on Conferences for Primary Publication [1.0660480034605242]
  This paper presents the first systematic investigation of the development of modern computing publications. It relies on semi-structured interviews with eight computing professors from diverse backgrounds to understand how researchers experienced changes in publication culture over time.
  arXiv  Detail & Related papers  (2021-09-14T04:59:09Z)
• Industry and Academic Research in Computer Vision [5.634825161148484]
  This work aims to study the dynamic between research in the industry and academia in computer vision. The results are demonstrated on a set of top-5 vision conferences that are representative of the field.
  arXiv  Detail & Related papers  (2021-07-10T20:09:52Z)
• Some Ethical Issues in the Review Process of Machine Learning Conferences [0.38073142980733]
  Recent successes in the Machine Learning community have led to a steep increase in the number of papers submitted to conferences. This increase made more prominent some of the issues that affect the current review process used by these conferences. We study the problem of reviewers' recruitment, infringements of the double-blind process, fraudulent behaviors, biases in numerical ratings, and the appendix phenomenon.
  arXiv  Detail & Related papers  (2021-06-01T21:22:41Z)
• A Survey on Causal Inference [64.45536158710014]
  Causal inference is a critical research topic across many domains, such as statistics, computer science, education, public policy and economics. Various causal effect estimation methods for observational data have sprung up.
  arXiv  Detail & Related papers  (2020-02-05T21:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.