What FID Hides: Detecting, Ranking, and Diagnosing Deviations in Generative Evaluation

• URL: http://arxiv.org/abs/2608.24881v1
• Date: Tue, 25 Aug 2026 17:58:46 GMT
• Title: What FID Hides: Detecting, Ranking, and Diagnosing Deviations in Generative Evaluation
• Abstract summary: Generative models are commonly ranked by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID)<n>ZID detects a broad range of departures, and its score tracks increasing severity along the corresponding sweeps.
• Score: 6.078726431806193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative models are commonly ranked by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID), yet FID's first-two-moment summary can miss distributional differences, and a reported scalar gap alone is not a calibrated test against sampling variation. FID's moment restriction has concrete consequences: on ImageNet, visually unrecognizable images optimized only to match the reference Inception mean and covariance obtain FID $24.7$ versus $58.6$ for held-out real images (lower is better). Moreover, FID and KID are scalar discrepancies that are unchanged when the two samples are exchanged and therefore do not encode the direction of a dispersion change: under-dispersion, as can occur in mode collapse, versus over-dispersion. We introduce \textbf{ZID} (\emph{Z-resolved Integrated Diagnostic}), which combines six standardized location- and dispersion-sensitive arms from a rank graph (RISE) and Gaussian kernels (GPK at two bandwidths). Rather than asking one scalar to serve incompatible roles, ZID reports three linked outputs: an index for ranking departure magnitude, a permutation $p$-value for testing distributional equality, and a signed dispersion readout for diagnosis. In controlled experiments, ZID detects a broad range of departures, and its score tracks increasing severity along the corresponding sweeps, including cases in which FID is flat or reversed. On DiT-XL/2 and SiT-XL/2 guidance sweeps, ZID detects departure from real data, and its signed readout labels the high-guidance diversity collapse as under-dispersion.

Related papers

• Statistical Symmetry Release for Equivariant Quantum Learning [12.042012916861829]
  Hard symmetry constraints reduce model complexity, but can also erase label information.<n> Statistical symmetry release determines when finite data and quantum measurements justify relaxing such a constraint.
  arXiv  Detail & Related papers  (2026-09-10T12:38:27Z)
• Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data [0.5213778368155993]
  DIFFINT is an autoencoder whose latent bottleneck is structured as a set of soft, axis-aligned interval memberships.<n>It is the only interpretable detector in the statistically-tied leading cluster of seven methods.
  arXiv  Detail & Related papers  (2026-09-03T14:05:27Z)
• OmniJudge or OmniBias? Diagnosing Multimodal Judges through Balanced, Decoupled Lenses [15.379006872674106]
  We introduce D3- Omni, a balanced and decoupled benchmark for diagnosing fine-grained multimodal understanding.<n>Rather than re-generating outputs, which may leak information across dimensions, we fix verified fully positive seeds and derive negatives.<n>The resulting D3 design is Dual-balanced, which helps alleviate negative-sample scarcity and per-dimension label imbalance.
  arXiv  Detail & Related papers  (2026-08-25T07:18:35Z)
• Change Detection in Probability Flow ODE: Online Testing in Diffusion Latent Spaces [0.0]
  A growing range of sequential data tasks, such as identifying trend reversals in financial markets, cannot be fully addressed without detection of distributional shifts in time-ordered data.<n>We consider a sequential change-point detection problem where the conditional density switches at an unknown time, yet neither the pre- nor post-change distribution admits a closed-form.<n>A conditional diffusion model, trained on pre-change-point data with a frozen context encoder, defines a deterministicection via the probability flow ODE.
  arXiv  Detail & Related papers  (2026-08-24T05:09:24Z)
• Separating Covariate Shift from Mechanism Change with Two Discriminators: CJSD, a Conditional Discrepancy with an Exact Covariate-Concept Decomposition [0.0]
  Conditional Jensen-Shannon Discrepancy (CJSD)<n>On a ten-measure battery over 202 dataset pairs, only the two conditional-information estimators -- CJSD and a kNN plug-in for the same estimand -- fail.
  arXiv  Detail & Related papers  (2026-08-20T10:48:08Z)
• Label-Free Target-Domain Adaptation for Unconstrained Event-Image Feature Matching via Dual-Stage Distillation [30.789763956644958]
  Cross-modal matching methods are largely restricted by their reliance on either matching labels or strictly aligned hardware.<n>We propose a novel two-stage training paradigm to address this problem.<n>Our proposed method achieves state-of-the-art performance on both MVSEC and TUM-VIE pose estimation tasks.
  arXiv  Detail & Related papers  (2026-07-11T02:23:11Z)
• The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation [55.62875959637507]
  We treat FID as a random variable on a two-axis panel of training and generation seeds.<n>We measure its variance directly on several hundred SiT networks trained on class-conditional ImageNet 256x256.
  arXiv  Detail & Related papers  (2026-06-18T17:49:32Z)
• Unsupervised Domain Shift Detection with Interpretable Subspace Attribution [48.553902624178995]
  Domain shifts are subtle differences in the probability distributions of datasets.<n>We identify these shifts using an algorithm designed to detect localised density anomalies in high-dimensional feature spaces.<n>We trace the domain shift to a small set of features, making the shift interpretable.
  arXiv  Detail & Related papers  (2026-05-15T12:58:00Z)
• The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detector [70.43599299422813]
  Existing methods fuse multiple annotations using a simple voting process, ignoring the inherent ambiguity of edges and labeling bias of annotators. We propose a novel uncertainty-aware edge detector (UAED), which employs uncertainty to investigate the subjectivity and ambiguity of diverse annotations. UAED achieves superior performance consistently across multiple edge detection benchmarks.
  arXiv  Detail & Related papers  (2023-03-21T13:14:36Z)
• SIOD: Single Instance Annotated Per Category Per Image for Object Detection [67.64774488115299]
  We propose the Single Instance annotated Object Detection (SIOD), requiring only one instance annotation for each existing category in an image. Degraded from inter-task (WSOD) or inter-image (SSOD) discrepancies to the intra-image discrepancy, SIOD provides more reliable and rich prior knowledge for mining the rest of unlabeled instances. Under the SIOD setting, we propose a simple yet effective framework, termed Dual-Mining (DMiner), which consists of a Similarity-based Pseudo Label Generating module (SPLG) and a Pixel-level Group Contrastive Learning module (PGCL)
  arXiv  Detail & Related papers  (2022-03-29T08:49:51Z)
• The KFIoU Loss for Rotated Object Detection [115.334070064346]
  In this paper, we argue that one effective alternative is to devise an approximate loss who can achieve trend-level alignment with SkewIoU loss. Specifically, we model the objects as Gaussian distribution and adopt Kalman filter to inherently mimic the mechanism of SkewIoU. The resulting new loss called KFIoU is easier to implement and works better compared with exact SkewIoU.
  arXiv  Detail & Related papers  (2022-01-29T10:54:57Z)
• Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search [90.30623718137244]
  We propose a novel deep hashing method for scalable multi-label image search. A new rank-consistency objective is applied to align the similarity orders from two spaces. A powerful loss function is designed to penalize the samples whose semantic similarity and hamming distance are mismatched.
  arXiv  Detail & Related papers  (2021-02-02T13:46:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.