Towards Better User Studies in Computer Graphics and Vision

• URL: http://arxiv.org/abs/2206.11461v3
• Date: Mon, 24 Apr 2023 21:27:13 GMT
• Title: Towards Better User Studies in Computer Graphics and Vision
• Authors: Zoya Bylinskii, Laura Herman, Aaron Hertzmann, Stefanie Hutka, Yile Zhang
• Abstract summary: We argue that user research is underutilized in driving project direction and forecasting user needs and reception. We offer an overview of methodologies from user experience research (UXR), human-computer interaction (HCI) and applied perception.
• Score: 16.871125231593272
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online crowdsourcing platforms have made it increasingly easy to perform evaluations of algorithm outputs with survey questions like "which image is better, A or B?", leading to their proliferation in vision and graphics research papers. Results of these studies are often used as quantitative evidence in support of a paper's contributions. On the one hand we argue that, when conducted hastily as an afterthought, such studies lead to an increase of uninformative, and, potentially, misleading conclusions. On the other hand, in these same communities, user research is underutilized in driving project direction and forecasting user needs and reception. We call for increased attention to both the design and reporting of user studies in computer vision and graphics papers towards (1) improved replicability and (2) improved project direction. Together with this call, we offer an overview of methodologies from user experience research (UXR), human-computer interaction (HCI), and applied perception to increase exposure to the available methodologies and best practices. We discuss foundational user research methods (e.g., needfinding) that are presently underutilized in computer vision and graphics research, but can provide valuable project direction. We provide further pointers to the literature for readers interested in exploring other UXR methodologies. Finally, we describe broader open issues and recommendations for the research community.

Related papers

• Learning the Bitter Lesson: Empirical Evidence from 20 Years of CVPR Proceedings [1.3812010983144802]
  This study examines the alignment of emphConference on Computer Vision and Pattern Recognition (CVPR) research with the principles of the "bitter lesson" proposed by Rich Sutton. We analyze two decades of CVPR abstracts and titles using large language models (LLMs) to assess the field's embracement of these principles.
  arXiv  Detail & Related papers  (2024-10-12T21:06:13Z)
• Fairness and Bias Mitigation in Computer Vision: A Survey [61.01658257223365]
  Computer vision systems are increasingly being deployed in high-stakes real-world applications. There is a dire need to ensure that they do not propagate or amplify any discriminatory tendencies in historical or human-curated data. This paper presents a comprehensive survey on fairness that summarizes and sheds light on ongoing trends and successes in the context of computer vision.
  arXiv  Detail & Related papers  (2024-08-05T13:44:22Z)
• Embedding in Recommender Systems: A Survey [67.67966158305603]
  A crucial aspect is embedding techniques that covert the high-dimensional discrete features, such as user and item IDs, into low-dimensional continuous vectors. Applying embedding techniques captures complex entity relationships and has spurred substantial research. This survey covers embedding methods like collaborative filtering, self-supervised learning, and graph-based techniques.
  arXiv  Detail & Related papers  (2023-10-28T06:31:06Z)
• Literature Review: Computer Vision Applications in Transportation Logistics and Warehousing [58.720142291102135]
  Computer vision applications in transportation logistics and warehousing have a huge potential for process automation. We present a structured literature review on research in the field to help leverage this potential.
  arXiv  Detail & Related papers  (2023-04-12T17:33:41Z)
• Attention is All They Need: Exploring the Media Archaeology of the Computer Vision Research Paper [4.968848569103028]
  We study changes in computer vision over the past decade, as the deep learning revolution has driven unprecedented growth in the discipline. Our analysis focuses on the research attention economy: how research paper elements contribute towards advertising, measuring, and disseminating an increasingly commodified "contribution"
  arXiv  Detail & Related papers  (2022-09-22T17:42:44Z)
• Tag-Aware Document Representation for Research Paper Recommendation [68.8204255655161]
  We propose a hybrid approach that leverages deep semantic representation of research papers based on social tags assigned by users. The proposed model is effective in recommending research papers even when the rating data is very sparse.
  arXiv  Detail & Related papers  (2022-09-08T09:13:07Z)
• Vision-Centric BEV Perception: A Survey [92.98068828762833]
  Vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia. The rapid advancements in deep learning have led to the proposal of numerous methods for addressing vision-centric BEV perception challenges. This paper compiles and organizes up-to-date knowledge, offering a systematic review and summary of prevalent algorithms.
  arXiv  Detail & Related papers  (2022-08-04T17:53:17Z)
• Knowledge Graph Induction enabling Recommending and Trend Analysis: A Corporate Research Community Use Case [11.907821975089064]
  We present a use case of how a corporate research community, IBM Research, utilizes Semantic Web technologies to induce a unified Knowledge Graph. We identify a set of common patterns for exploiting the induced knowledge and exposed them as APIs. Those patterns were born out of user research which identified the most valuable use cases or user pain points to be alleviated.
  arXiv  Detail & Related papers  (2022-07-11T20:51:28Z)
• Enhancing Reading Strategies by Exploring A Theme-based Approach to Literature Surveys [5.004814662623872]
  We have designed a methodology that allows users to visually and thematically explore corpora, while developing personalised holistic reading strategies. Using in-depth semi-structured interviews and stimulated recall, we found that users: (i) selected papers that they otherwise would not have read, (ii) developed a more coherent reading strategy, and (iii) understood the thematic structure and relationships between papers more effectively.
  arXiv  Detail & Related papers  (2021-02-10T10:36:45Z)
• Open Source Software for Efficient and Transparent Reviews [0.11179881480027788]
  ASReview is an open source machine learning-aided pipeline applying active learning. We demonstrate by means of simulation studies that ASReview can yield far more efficient reviewing than manual reviewing.
  arXiv  Detail & Related papers  (2020-06-22T11:57:10Z)
• A Survey on Knowledge Graph-Based Recommender Systems [65.50486149662564]
  We conduct a systematical survey of knowledge graph-based recommender systems. We focus on how the papers utilize the knowledge graph for accurate and explainable recommendation. We introduce datasets used in these works.
  arXiv  Detail & Related papers  (2020-02-28T02:26:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.