Collaboration or Corporate Capture? Quantifying NLP's Reliance on Industry Artifacts and Contributions

• URL: http://arxiv.org/abs/2312.03912v2
• Date: Sat, 22 Jun 2024 15:27:04 GMT
• Title: Collaboration or Corporate Capture? Quantifying NLP's Reliance on Industry Artifacts and Contributions
• Authors: Will Aitken, Mohamed Abdalla, Karen Rudie, Catherine Stinson,
• Abstract summary: We surveyed 100 papers published at EMNLP 2022 to determine the degree to which researchers rely on industry models. Our work serves as a scaffold to enable future researchers to more accurately address whether collaboration with industry is still collaboration in the absence of an alternative.
• Score: 2.6746207141044582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Impressive performance of pre-trained models has garnered public attention and made news headlines in recent years. Almost always, these models are produced by or in collaboration with industry. Using them is critical for competing on natural language processing (NLP) benchmarks and correspondingly to stay relevant in NLP research. We surveyed 100 papers published at EMNLP 2022 to determine the degree to which researchers rely on industry models, other artifacts, and contributions to publish in prestigious NLP venues and found that the ratio of their citation is at least three times greater than what would be expected. Our work serves as a scaffold to enable future researchers to more accurately address whether: 1) Collaboration with industry is still collaboration in the absence of an alternative or 2) if NLP inquiry has been captured by the motivations and research direction of private corporations.

Related papers

• The Nature of NLP: Analyzing Contributions in NLP Papers [77.31665252336157]
  We quantitatively investigate what constitutes NLP research by examining research papers. Our findings reveal a rising involvement of machine learning in NLP since the early nineties. In post-2020, there has been a resurgence of focus on language and people.
  arXiv  Detail & Related papers  (2024-09-29T01:29:28Z)
• What Can Natural Language Processing Do for Peer Review? [173.8912784451817]
  In modern science, peer review is widely used, yet it is hard, time-consuming, and prone to error. Since the artifacts involved in peer review are largely text-based, Natural Language Processing has great potential to improve reviewing. We detail each step of the process from manuscript submission to camera-ready revision, and discuss the associated challenges and opportunities for NLP assistance.
  arXiv  Detail & Related papers  (2024-05-10T16:06:43Z)
• We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fields [30.550895983110806]
  Cross-field engagement of Natural Language Processing has declined. Less than 8% of NLP citations are to linguistics. Less than 3% of NLP citations are to math and psychology.
  arXiv  Detail & Related papers  (2023-10-23T12:42:06Z)
• Who should I Collaborate with? A Comparative Study of Academia and Industry Research Collaboration in NLP [5.929956715430167]
  The goal of our research was to investigate the effects of collaboration between academia and industry on Natural Language Processing (NLP) We created a pipeline to extract affiliations and citations from NLP papers and divided them into three categories: academia, industry, and hybrid (collaborations between academia and industry) Our empirical analysis found that there is a trend towards an increase in industry and academia-industry collaboration publications and that these types of publications tend to have a higher impact compared to those produced solely within academia.
  arXiv  Detail & Related papers  (2023-07-21T01:26:29Z)
• Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good [115.1507728564964]
  We introduce NLP4SG Papers, a scientific dataset with three associated tasks. These tasks help identify NLP4SG papers and characterize the NLP4SG landscape. We use state-of-the-art NLP models to address each of these tasks and use them on the entire ACL Anthology.
  arXiv  Detail & Related papers  (2023-05-09T14:16:25Z)
• The Elephant in the Room: Analyzing the Presence of Big Tech in Natural Language Processing Research [28.382353702576314]
  We use a corpus with comprehensive metadata of 78,187 NLP publications and 701 resumes of NLP publication authors. We find that industry presence among NLP authors has been steady before a steep increase over the past five years. A few companies account for most of the publications and provide funding to academic researchers through grants and internships.
  arXiv  Detail & Related papers  (2023-05-04T12:57:18Z)
• NLPeer: A Unified Resource for the Computational Study of Peer Review [58.71736531356398]
  We introduce NLPeer -- the first ethically sourced multidomain corpus of more than 5k papers and 11k review reports from five different venues. We augment previous peer review datasets to include parsed and structured paper representations, rich metadata and versioning information. Our work paves the path towards systematic, multi-faceted, evidence-based study of peer review in NLP and beyond.
  arXiv  Detail & Related papers  (2022-11-12T12:29:38Z)
• Geographic Citation Gaps in NLP Research [63.13508571014673]
  This work asks a series of questions on the relationship between geographical location and publication success. We first created a dataset of 70,000 papers from the ACL Anthology, extracted their meta-information, and generated their citation network. We show that not only are there substantial geographical disparities in paper acceptance and citation but also that these disparities persist even when controlling for a number of variables such as venue of publication and sub-field of NLP.
  arXiv  Detail & Related papers  (2022-10-26T02:25:23Z)
• Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold [88.83876819883653]
  We show through a manual classification of recent NLP research papers that this is indeed the case. We observe that NLP research often goes beyond the square one setup, focusing not only on accuracy, but also on fairness or interpretability, but typically only along a single dimension.
  arXiv  Detail & Related papers  (2022-06-20T13:04:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.