CausalSim: A Causal Framework for Unbiased Trace-Driven Simulation

• URL: http://arxiv.org/abs/2201.01811v4
• Date: Fri, 5 May 2023 23:14:14 GMT
• Title: CausalSim: A Causal Framework for Unbiased Trace-Driven Simulation
• Authors: Abdullah Alomar, Pouya Hamadanian, Arash Nasr-Esfahany, Anish Agarwal, Mohammad Alizadeh, Devavrat Shah
• Abstract summary: CausalSim is a causal framework for unbiased trace-driven simulation. It learns a causal model of the system dynamics and latent factors capturing the underlying system conditions during trace collection. It reduces errors by 53% and 61% on average compared to expert-designed and supervised learning baselines.
• Score: 25.620312665350028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present CausalSim, a causal framework for unbiased trace-driven simulation. Current trace-driven simulators assume that the interventions being simulated (e.g., a new algorithm) would not affect the validity of the traces. However, real-world traces are often biased by the choices algorithms make during trace collection, and hence replaying traces under an intervention may lead to incorrect results. CausalSim addresses this challenge by learning a causal model of the system dynamics and latent factors capturing the underlying system conditions during trace collection. It learns these models using an initial randomized control trial (RCT) under a fixed set of algorithms, and then applies them to remove biases from trace data when simulating new algorithms. Key to CausalSim is mapping unbiased trace-driven simulation to a tensor completion problem with extremely sparse observations. By exploiting a basic distributional invariance property present in RCT data, CausalSim enables a novel tensor completion method despite the sparsity of observations. Our extensive evaluation of CausalSim on both real and synthetic datasets, including more than ten months of real data from the Puffer video streaming system shows it improves simulation accuracy, reducing errors by 53% and 61% on average compared to expert-designed and supervised learning baselines. Moreover, CausalSim provides markedly different insights about ABR algorithms compared to the biased baseline simulator, which we validate with a real deployment.

Related papers

• GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator [55.02281855589641]
  GauSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels. We leverage continuum mechanics, modeling each kernel as a continuous piece of matter to account for realistic deformations without idealized assumptions. GauSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• Accelerated zero-order SGD under high-order smoothness and overparameterized regime [79.85163929026146]
  We present a novel gradient-free algorithm to solve convex optimization problems. Such problems are encountered in medicine, physics, and machine learning. We provide convergence guarantees for the proposed algorithm under both types of noise.
  arXiv  Detail & Related papers  (2024-11-21T10:26:17Z)
• NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking [65.24988062003096]
  We present NAVSIM, a framework for benchmarking vision-based driving policies. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights.
  arXiv  Detail & Related papers  (2024-06-21T17:59:02Z)
• Addressing Misspecification in Simulation-based Inference through Data-driven Calibration [43.811367860375825]
  Recent work has demonstrated that model misspecification can harm simulation-based inference's reliability. This work introduces robust posterior estimation (ROPE), a framework that overcomes model misspecification with a small real-world calibration set of ground truth parameter measurements.
  arXiv  Detail & Related papers  (2024-05-14T16:04:39Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study [61.64685376882383]
  Counterfactual learning to rank (CLTR) has attracted extensive attention in the IR community for its ability to leverage massive logged user interaction data to train ranking models. This paper investigates the robustness of existing CLTR models in complex and diverse situations. We find that the DLA models and IPS-DCM show better robustness under various simulation settings than IPS-PBM and PRS with offline propensity estimation.
  arXiv  Detail & Related papers  (2024-04-04T10:54:38Z)
• Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
  We present SIM-FSVGD for learning robot dynamics from data. We use low-fidelity physical priors to regularize the training of neural network models. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
  arXiv  Detail & Related papers  (2024-03-25T11:29:32Z)
• CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discovery [14.092834149864514]
  This study introduces the CausalTime pipeline to generate time-series that highly resemble the real data. The pipeline starts from real observations in a specific scenario and produces a matching benchmark dataset. In the experiments, we validate the fidelity of the generated data through qualitative and quantitative experiments, followed by a benchmarking of existing TSCD algorithms.
  arXiv  Detail & Related papers  (2023-10-03T02:29:19Z)
• Truncated Marginal Neural Ratio Estimation [5.438798591410838]
  We present a neural simulator-based inference algorithm which simultaneously offers simulation efficiency and fast empirical posterior testability. Our approach is simulation efficient by simultaneously estimating low-dimensional marginal posteriors instead of the joint posterior. By estimating a locally amortized posterior our algorithm enables efficient empirical tests of the robustness of the inference results.
  arXiv  Detail & Related papers  (2021-07-02T18:00:03Z)
• Training robust anomaly detection using ML-Enhanced simulations [1.370633147306388]
  Simulations can provide edge conditions for anomaly detection which may be sparse or non-existent in real-world data. Our approach enhances simulations using neural networks trained on real-world data to create outputs that are more realistic and variable than traditional simulations.
  arXiv  Detail & Related papers  (2020-08-27T12:28:07Z)
• Continuous Optimization Benchmarks by Simulation [0.0]
  Benchmark experiments are required to test, compare, tune, and understand optimization algorithms. Data from previous evaluations can be used to train surrogate models which are then used for benchmarking. We show that the spectral simulation method enables simulation for continuous optimization problems.
  arXiv  Detail & Related papers  (2020-08-14T08:50:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.