Does mapping elites illuminate search spaces? A large-scale user study of MAP--Elites applied to human--AI collaborative design

• URL: http://arxiv.org/abs/2402.07911v1
• Date: Tue, 30 Jan 2024 08:54:46 GMT
• Title: Does mapping elites illuminate search spaces? A large-scale user study of MAP--Elites applied to human--AI collaborative design
• Authors: Sean P. Walton, Ben J. Evans, Alma A. M. Rahat, James Stovold, Jakub Vincalek
• Abstract summary: The tool investigated is based on an evolutionary algorithm attempting to design a virtual car to travel as far as possible in a fixed time. Participants were able to design their own cars, make recommendations to the algorithm and view sets of recommendations from the algorithm.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Two studies of a human-AI collaborative design tool were carried out in order to understand the influence design recommendations have on the design process. The tool investigated is based on an evolutionary algorithm attempting to design a virtual car to travel as far as possible in a fixed time. Participants were able to design their own cars, make recommendations to the algorithm and view sets of recommendations from the algorithm. The algorithm-recommended sets were designs which had been previously tested; some sets were simply randomly picked and other sets were picked using MAP-Elites. In the first study 808 design sessions were recorded as part of a science outreach program, each with analytical data of how each participant used the tool. To provide context to this quantitative data, a smaller double-blind lab study was also carried out with 12 participants. In the lab study the same quantitative data from the large scale study was collected alongside responses to interview questions. Although there is some evidence that the MAP-Elites provide higher-quality individual recommendations, neither study provides convincing evidence that these recommendations have a more positive influence on the design process than simply a random selection of designs. In fact, it seems that providing a combination of MAP-Elites and randomly selected recommendations is beneficial to the process. Furthermore, simply viewing recommendations from the MAP-Elites had a positive influence on engagement in the design task and the quality of the final design produced. Our findings are significant both for researchers designing new mixed-initiative tools, and those who wish to evaluate existing tools. Most significantly, we found that metrics researchers currently use to evaluate the success of human-AI collaborative algorithms do not measure the full influence these algorithms have on the design process.

Related papers

• On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy [0.1909020214605419]
  This paper presents a review of 65 user studies evaluating XAI systems across different domains and application contexts.<n>We propose objectives for the human-centered design (design goals) of XAI systems.<n>For AI novices, the relevant extended design goals include responsible use, acceptance, and usability.
  arXiv  Detail & Related papers  (2025-10-14T06:52:43Z)
• Human-AI Co-Creation: A Framework for Collaborative Design in Intelligent Systems [0.6526824510982802]
  This paper explores the emergent paradigm of human-AI co-creation, where AI is not merely used for automation or efficiency gains, but actively participates in ideation, visual conceptualization, and decision-making.<n>Specifically, we investigate the use of large language models (LLMs) like GPT-4 and multimodal diffusion models such as Stable Diffusion as creative agents that engage designers in iterative cycles of proposal, critique, and revision.
  arXiv  Detail & Related papers  (2025-07-22T04:29:33Z)
• SITE: towards Spatial Intelligence Thorough Evaluation [121.1493852562597]
  Spatial intelligence (SI) represents a cognitive ability encompassing the visualization, manipulation, and reasoning about spatial relationships.<n>We introduce SITE, a benchmark dataset towards SI Thorough Evaluation.<n>Our approach to curating the benchmark combines a bottom-up survey about 31 existing datasets and a top-down strategy drawing upon three classification systems in cognitive science.
  arXiv  Detail & Related papers  (2025-05-08T17:45:44Z)
• Navigating Intelligence: A Survey of Google OR-Tools and Machine Learning for Global Path Planning in Autonomous Vehicles [49.1574468325115]
  Global path planning is essential for an autonomous mining sampling robot named ROMIE. Q-Learning stands out as the optimal strategy, demonstrating superior efficiency by deviating only 1.2% on average from the optimal solutions across our datasets.
  arXiv  Detail & Related papers  (2025-03-05T10:12:22Z)
• AI Automatons: AI Systems Intended to Imitate Humans [54.19152688545896]
  There is a growing proliferation of AI systems designed to mimic people's behavior, work, abilities, likenesses, or humanness.<n>The research, design, deployment, and availability of such AI systems have prompted growing concerns about a wide range of possible legal, ethical, and other social impacts.
  arXiv  Detail & Related papers  (2025-03-04T03:55:38Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  AI technology can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals.<n>The responsible use of AI and its participation in human-AI teams increasingly shows the need for AI alignment.<n>A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Characterizing and modeling harms from interactions with design patterns in AI interfaces [0.19116784879310028]
  We argue that design features of interfaces with adaptive AI systems can have cascading impacts, driven by feedback loops. We propose Design-Enhanced Control of AI systems (DECAI) to structure and facilitate impact assessments of AI interface designs.
  arXiv  Detail & Related papers  (2024-04-17T13:30:45Z)
• randomHAR: Improving Ensemble Deep Learners for Human Activity Recognition with Sensor Selection and Reinforcement Learning [4.5830802802139585]
  The general idea behind randomHAR is training a series of deep learning models with the same architecture on randomly selected sensor data. In contrast to existing work, this approach optimize the ensemble process rather than the architecture of the constituent models. The experiment demonstrates that the proposed approach outperforms the state-of-the-art method, ensembleLSTM.
  arXiv  Detail & Related papers  (2023-07-15T10:51:03Z)
• Human-AI Coevolution [48.74579595505374]
  Coevolution AI is a process in which humans and AI algorithms continuously influence each other. This paper introduces Coevolution AI as the cornerstone for a new field of study at the intersection between AI and complexity science.
  arXiv  Detail & Related papers  (2023-06-23T18:10:54Z)
• Beyond Prompts: Exploring the Design Space of Mixed-Initiative Co-Creativity Systems [11.427320283363326]
  We conduct a human participant study with 185 participants to understand how users want to interact with differently configured MI-CC systems. We find that MI-CC systems with more extensive coverage of the design space are rated higher or on par on a variety of creative and goal-completion metrics.
  arXiv  Detail & Related papers  (2023-05-03T22:32:37Z)
• A Reinforcement Learning-assisted Genetic Programming Algorithm for Team Formation Problem Considering Person-Job Matching [70.28786574064694]
  A reinforcement learning-assisted genetic programming algorithm (RL-GP) is proposed to enhance the quality of solutions. The hyper-heuristic rules obtained through efficient learning can be utilized as decision-making aids when forming project teams.
  arXiv  Detail & Related papers  (2023-04-08T14:32:12Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• Compactness Score: A Fast Filter Method for Unsupervised Feature Selection [66.84571085643928]
  We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features. Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
  arXiv  Detail & Related papers  (2022-01-31T13:01:37Z)
• The Impact of Visualizing Design Gradients for Human Designers [7.976180564784953]
  This paper introduces a mixed-initiative tool employing Exhaustive PCG ( EPCG) for puzzle level design. We run an online human subject study in which individuals use the tool with an EPCG component turned on or off. Our analysis of the results demonstrates that, although a majority of users did not prefer the tool, it made the level design process significantly easier.
  arXiv  Detail & Related papers  (2021-10-07T05:41:09Z)
• A Field Guide to Federated Optimization [161.3779046812383]
  Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms.
  arXiv  Detail & Related papers  (2021-07-14T18:09:08Z)
• An AI-Assisted Design Method for Topology Optimization Without Pre-Optimized Training Data [68.8204255655161]
  An AI-assisted design method based on topology optimization is presented, which is able to obtain optimized designs in a direct way. Designs are provided by an artificial neural network, the predictor, on the basis of boundary conditions and degree of filling as input data.
  arXiv  Detail & Related papers  (2020-12-11T14:33:27Z)
• Deep Learning of Individual Aesthetics [5.837881923712394]
  We investigate the relationship between image measures, such as complexity, and human aesthetic evaluation. We use dimension reduction methods to visualise both genotype and phenotype space in order to support the exploration of new territory in a generative system. We integrate this classification and discovery system into a software tool for evolving complex generative art and design.
  arXiv  Detail & Related papers  (2020-09-24T03:04:28Z)
• Evaluating Mixed-Initiative Procedural Level Design Tools using a Triple-Blind Mixed-Method User Study [0.0]
  A tool which generates levels using interactive evolutionary optimisation was designed for this study. The tool identifies level design patterns in an initial hand-designed map and uses that information to drive an interactive optimisation algorithm. A rigorous user study was designed which compared the experiences of designers using the mixed-initiative tool to designers who were given a tool which provided completely random level suggestions.
  arXiv  Detail & Related papers  (2020-05-15T11:40:53Z)
• Learning to Complement Humans [67.38348247794949]
  A rising vision for AI in the open world centers on the development of systems that can complement humans for perceptual, diagnostic, and reasoning tasks. We demonstrate how an end-to-end learning strategy can be harnessed to optimize the combined performance of human-machine teams.
  arXiv  Detail & Related papers  (2020-05-01T20:00:23Z)
• Multi-Task Learning for Dense Prediction Tasks: A Survey [87.66280582034838]
  Multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint. We provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision.
  arXiv  Detail & Related papers  (2020-04-28T09:15:50Z)
• Distributed and Democratized Learning: Philosophy and Research Challenges [80.39805582015133]
  We propose a novel design philosophy called democratized learning (Dem-AI) Inspired by the societal groups of humans, the specialized groups of learning agents in the proposed Dem-AI system are self-organized in a hierarchical structure to collectively perform learning tasks more efficiently. We present a reference design as a guideline to realize future Dem-AI systems, inspired by various interdisciplinary fields.
  arXiv  Detail & Related papers  (2020-03-18T08:45:10Z)
• Application of Deep Learning in Generating Desired Design Options: Experiments Using Synthetic Training Dataset [5.564299196293697]
  This study applies a method using Deep Learning (DL) algorithms towards generating demanded design options. An object recognition problem is investigated to initially predict the label of unseen sample images based on training dataset consisting of different types of synthetic 2D shapes. In the next step, the algorithm is trained to generate a window/wall pattern for desired light/shadow performance based on the spatial daylight autonomy (sDA) metrics.
  arXiv  Detail & Related papers  (2019-12-28T01:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.