Second SIGIR Workshop on Simulations for Information Access (Sim4IA 2025)

• URL: http://arxiv.org/abs/2505.11687v1
• Date: Fri, 16 May 2025 20:48:59 GMT
• Title: Second SIGIR Workshop on Simulations for Information Access (Sim4IA 2025)
• Abstract summary: Simulations in information access (IA) have recently gained interest, as shown by various tutorials and workshops around that topic.<n>Building on recent developments in methods and toolkits, the second iteration of our Sim4IA workshop aims to again bring together researchers and practitioners to form an interactive and engaging forum for discussions on the future perspectives of the field.
• Score: 13.554066857924898
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simulations in information access (IA) have recently gained interest, as shown by various tutorials and workshops around that topic. Simulations can be key contributors to central IA research and evaluation questions, especially around interactive settings when real users are unavailable, or their participation is impossible due to ethical reasons. In addition, simulations in IA can help contribute to a better understanding of users, reduce complexity of evaluation experiments, and improve reproducibility. Building on recent developments in methods and toolkits, the second iteration of our Sim4IA workshop aims to again bring together researchers and practitioners to form an interactive and engaging forum for discussions on the future perspectives of the field. An additional aim is to plan an upcoming TREC/CLEF campaign.

Related papers

• A Survey on LLM-based Conversational User Simulation [135.70167715992912]
  Large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation.<n>We introduce a novel taxonomy covering user granularity and simulation objectives.<n>We aim to keep the research community informed of the latest advancements in conversational user simulation.
  arXiv  Detail & Related papers  (2026-04-27T20:22:42Z)
• Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Prediction [18.30483927706278]
  We present Sim4IA-Bench, a simulation benchmark suit for the prediction of the next queries and utterances.<n>Our dataset comprises 160 real-world search sessions from the CORE search engine.<n>Sim4IA-Bench provides a basis for evaluating and comparing user simu- lation approaches.
  arXiv  Detail & Related papers  (2025-11-12T13:44:12Z)
• Dyna-Mind: Learning to Simulate from Experience for Better AI Agents [62.21219817256246]
  We argue that current AI agents need ''vicarious trial and error'' - the capacity to mentally simulate alternative futures before acting.<n>We introduce Dyna-Mind, a two-stage training framework that explicitly teaches (V)LM agents to integrate such simulation into their reasoning.
  arXiv  Detail & Related papers  (2025-10-10T17:30:18Z)
• The Indispensable Role of User Simulation in the Pursuit of AGI [37.789218939871105]
  We argue that realistic simulators provide the necessary environments for scalable evaluation, data generation for interactive learning, and fostering the adaptive capabilities central to Artificial General Intelligence (AGI)<n>This article elaborates on the critical role of user simulation for AGI, explores the interdisciplinary nature of building realistic simulators, identifies key challenges including those posed by large language models, and proposes a future research agenda.
  arXiv  Detail & Related papers  (2025-09-23T18:12:45Z)
• YuLan-OneSim: Towards the Next Generation of Social Simulator with Large Language Models [50.86336063222539]
  We introduce a novel social simulator called YuLan-OneSim.<n>Users can simply describe and refine their simulation scenarios through natural language interactions with our simulator.<n>We implement 50 default simulation scenarios spanning 8 domains, including economics, sociology, politics, psychology, organization, demographics, law, and communication.
  arXiv  Detail & Related papers  (2025-05-12T14:05:17Z)
• Evaluating Contrastive Feedback for Effective User Simulations [2.8089969618577997]
  This study explores whether the underlying principles of contrastive training techniques can be applied beneficially in the area of prompt engineering for user simulations.<n>The primary objective of this study is to analyze how different modalities of contextual information influence the effectiveness of user simulations.
  arXiv  Detail & Related papers  (2025-05-05T11:02:31Z)
• ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning [78.42927884000673]
  ExACT is an approach to combine test-time search and self-learning to build o1-like models for agentic applications.<n>We first introduce Reflective Monte Carlo Tree Search (R-MCTS), a novel test time algorithm designed to enhance AI agents' ability to explore decision space on the fly.<n>Next, we introduce Exploratory Learning, a novel learning strategy to teach agents to search at inference time without relying on any external search algorithms.
  arXiv  Detail & Related papers  (2024-10-02T21:42:35Z)
• Report on the Workshop on Simulations for Information Access (Sim4IA 2024) at SIGIR 2024 [33.22176229332443]
  This paper is a report of the Workshop on Simulations for Information Access (Sim4IA) workshop at SIGIR 2024. Key takeaways were user simulation's importance in academia and industry, the possible bridging of online and offline evaluation, and the issues of organizing a companion shared task around user simulations for information access.
  arXiv  Detail & Related papers  (2024-09-26T16:32:10Z)
• Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI [129.08019405056262]
  Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial Intelligence (AGI) MLMs andWMs have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilities. In this survey, we give a comprehensive exploration of the latest advancements in Embodied AI.
  arXiv  Detail & Related papers  (2024-07-09T14:14:47Z)
• GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Tasks [0.0]
  This paper is the first examination regarding the use of large-scale pre-trained language models for scientific simulations. The first task is devoted to explaining the structure of a conceptual model to promote the engagement of participants. The second task focuses on summarizing simulation outputs, so that model users can identify a preferred scenario. The third task seeks to broaden accessibility to simulation platforms by conveying the insights of simulation visualizations via text.
  arXiv  Detail & Related papers  (2023-06-21T15:42:36Z)
• User Simulation for Evaluating Information Access Systems [38.48048183731099]
  evaluating the effectiveness of interactive intelligent systems is a complex scientific challenge. This book provides a thorough understanding of user simulation techniques designed specifically for evaluation. It covers both general frameworks for designing user simulators, and specific models and algorithms for simulating user interactions with search engines, recommender systems, and conversational assistants.
  arXiv  Detail & Related papers  (2023-06-14T14:54:06Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• Synthetic Data-Based Simulators for Recommender Systems: A Survey [55.60116686945561]
  This survey aims at providing a comprehensive overview of the recent trends in the field of modeling and simulation. We start with the motivation behind the development of frameworks implementing the simulations -- simulators. We provide a new consistent classification of existing simulators based on their functionality, approbation, and industrial effectiveness.
  arXiv  Detail & Related papers  (2022-06-22T19:33:21Z)
• Game and Simulation Design for Studying Pedestrian-Automated Vehicle Interactions [1.3764085113103217]
  We first present contemporary tools in the field and then propose the design and development of a new application that facilitates pedestrian point of view research. We conduct a three-step user experience experiment where participants answer questions before and after using the application in various scenarios.
  arXiv  Detail & Related papers  (2021-09-30T15:26:18Z)
• A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
  This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
  arXiv  Detail & Related papers  (2020-11-17T22:24:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.