A Comprehensive Evaluation Framework for Synthetic Trip Data Generation in Public Transport

• URL: http://arxiv.org/abs/2510.24375v1
• Date: Tue, 28 Oct 2025 12:52:47 GMT
• Title: A Comprehensive Evaluation Framework for Synthetic Trip Data Generation in Public Transport
• Authors: Yuanyuan Wu, Zhenlin Qin, Zhenliang Ma,
• Abstract summary: Synthetic data offers a promising solution to the privacy and accessibility challenges of using smart card data in public transport research.<n>We propose a framework that systematically evaluates synthetic trip data across three complementary dimensions and three hierarchical levels.<n>Results show that synthetic data do not inherently guarantee privacy and there is no "one-size-fits-all" model.
• Score: 7.409483754602669
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthetic data offers a promising solution to the privacy and accessibility challenges of using smart card data in public transport research. Despite rapid progress in generative modeling, there is limited attention to comprehensive evaluation, leaving unclear how reliable, safe, and useful synthetic data truly are. Existing evaluations remain fragmented, typically limited to population-level representativeness or record-level privacy, without considering group-level variations or task-specific utility. To address this gap, we propose a Representativeness-Privacy-Utility (RPU) framework that systematically evaluates synthetic trip data across three complementary dimensions and three hierarchical levels (record, group, population). The framework integrates a consistent set of metrics to quantify similarity, disclosure risk, and practical usefulness, enabling transparent and balanced assessment of synthetic data quality. We apply the framework to benchmark twelve representative generation methods, spanning conventional statistical models, deep generative networks, and privacy-enhanced variants. Results show that synthetic data do not inherently guarantee privacy and there is no "one-size-fits-all" model, the trade-off between privacy and representativeness/utility is obvious. Conditional Tabular generative adversarial network (CTGAN) provide the most balanced trade-off and is suggested for practical applications. The RPU framework provides a systematic and reproducible basis for researchers and practitioners to compare synthetic data generation techniques and select appropriate methods in public transport applications.

Related papers

• Empirical Evaluation of Structured Synthetic Data Privacy Metrics: Novel experimental framework [34.56525983543448]
  Synthetic data generation is gaining traction as a privacy enhancing technology.<n>The concept of data privacy remains elusive, making it challenging for practitioners to evaluate and benchmark the degree of privacy protection offered by synthetic data.
  arXiv  Detail & Related papers  (2025-12-18T08:09:28Z)
• A Style-Based Profiling Framework for Quantifying the Synthetic-to-Real Gap in Autonomous Driving Datasets [9.788200709163064]
  This paper introduces a profile extraction and discovery framework for characterizing the style profiles underlying both synthetic and real image datasets.<n>Our framework combines Gram matrix-based style extraction with metric learning optimized for intra-class compactness and inter-class separation to extract style embeddings.
  arXiv  Detail & Related papers  (2025-10-11T13:09:41Z)
• A Survey on Tabular Data Generation: Utility, Alignment, Fidelity, Privacy, and Beyond [53.56796220109518]
  Different use cases demand synthetic data to comply with different requirements to be useful in practice.<n>Four types of requirements are reviewed: utility of the synthetic data, alignment of the synthetic data with domain-specific knowledge, statistical fidelity of the synthetic data distribution compared to the real data distribution, and privacy-preserving capabilities.<n>We discuss future directions for the field, along with opportunities to improve the current evaluation methods.
  arXiv  Detail & Related papers  (2025-03-07T21:47:11Z)
• Opinion: Revisiting synthetic data classifications from a privacy perspective [42.12937192948916]
  Synthetic data is emerging as a cost-effective solution to meet the increasing data demands of AI development.<n>Traditional classification of synthetic data types does not reflect the ever-increasing methods to generate synthetic data.<n>We make a case for an alternative approach to grouping synthetic data types that better reflect privacy perspectives.
  arXiv  Detail & Related papers  (2025-03-05T13:54:13Z)
• Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data [104.30479583607918]
  2nd FRCSyn-onGoing challenge is based on the 2nd Face Recognition Challenge in the Era of Synthetic Data (FRCSyn), originally launched at CVPR 2024.<n>We focus on exploring the use of synthetic data both individually and in combination with real data to solve current challenges in face recognition.
  arXiv  Detail & Related papers  (2024-12-02T11:12:01Z)
• Reconsidering utility: unveiling the limitations of synthetic mobility data generation algorithms in real-life scenarios [49.1574468325115]
  We evaluate the utility of five state-of-the-art synthesis approaches in terms of real-world applicability. We focus on so-called trip data that encode fine granular urban movements such as GPS-tracked taxi rides. One model fails to produce data within reasonable time and another generates too many jumps to meet the requirements for map matching.
  arXiv  Detail & Related papers  (2024-07-03T16:08:05Z)
• Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark [56.8042116967334]
  Synthetic data serves as an alternative in training machine learning models. ensuring that synthetic data mirrors the complex nuances of real-world data is a challenging task. This paper explores the potential of integrating data-centric AI techniques to guide the synthetic data generation process.
  arXiv  Detail & Related papers  (2023-10-25T20:32:02Z)
• Statistical properties and privacy guarantees of an original distance-based fully synthetic data generation method [0.0]
  This work shows the technical feasibility of generating publicly releasable synthetic data using a multi-step framework. By successfully assessing the quality of data produced using a novel multi-step synthetic data generation framework, we showed the technical and conceptual soundness of the Open-CESP initiative.
  arXiv  Detail & Related papers  (2023-10-10T12:29:57Z)
• Auditing and Generating Synthetic Data with Controllable Trust Trade-offs [54.262044436203965]
  We introduce a holistic auditing framework that comprehensively evaluates synthetic datasets and AI models. It focuses on preventing bias and discrimination, ensures fidelity to the source data, assesses utility, robustness, and privacy preservation. We demonstrate the framework's effectiveness by auditing various generative models across diverse use cases.
  arXiv  Detail & Related papers  (2023-04-21T09:03:18Z)
• Enabling Synthetic Data adoption in regulated domains [1.9512796489908306]
  The switch from a Model-Centric to a Data-Centric mindset is putting emphasis on data and its quality rather than algorithms. In particular, the sensitive nature of the information in highly regulated scenarios needs to be accounted for. A clever way to bypass such a conundrum relies on Synthetic Data: data obtained from a generative process, learning the real data properties.
  arXiv  Detail & Related papers  (2022-04-13T10:53:54Z)
• Representative & Fair Synthetic Data [68.8204255655161]
  We present a framework to incorporate fairness constraints into the self-supervised learning process. We generate a representative as well as fair version of the UCI Adult census data set. We consider representative & fair synthetic data a promising future building block to teach algorithms not on historic worlds, but rather on the worlds that we strive to live in.
  arXiv  Detail & Related papers  (2021-04-07T09:19:46Z)
• Holdout-Based Fidelity and Privacy Assessment of Mixed-Type Synthetic Data [0.0]
  AI-based data synthesis has seen rapid progress over the last several years, and is increasingly recognized for its promise to enable privacy-respecting data sharing. We introduce and demonstrate a holdout-based empirical assessment framework for quantifying the fidelity as well as the privacy risk of synthetic data solutions.
  arXiv  Detail & Related papers  (2021-04-01T17:30:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.