Raising the Ceiling: Better Empirical Fixation Densities for Saliency Benchmarking

• URL: http://arxiv.org/abs/2605.03885v1
• Date: Tue, 05 May 2026 15:45:00 GMT
• Title: Raising the Ceiling: Better Empirical Fixation Densities for Saliency Benchmarking
• Abstract summary: Empirical fixation densities, spatial distributions estimated from human eye-tracking data, are foundational to saliency benchmarking.<n>We propose a principled mixture model that combines an adaptive-bandwidth Ubuntu based on Abramson's method, center bias and uniform components, and a state-of-the-art saliency model.<n>Our method yields substantially higher interobserver consistency estimates across multiple benchmarks, with median per-image gains of 5-15% in log-likelihood and up to 2 percentage points in AUC.
• Score: 7.44553254894651
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Empirical fixation densities, spatial distributions estimated from human eye-tracking data, are foundational to saliency benchmarking. They directly shape benchmark conclusions, leaderboard rankings, failure case analyses, and scientific claims about human visual behavior. Yet the standard estimation method, fixed-bandwidth isotropic Gaussian KDE, has gone essentially unchanged for decades. This matters now more than ever: as the field shifts toward sample-level evaluation (failure case analysis, inverse benchmarking, per-image model comparison), reliable per-image density estimates become critical. We propose a principled mixture model that combines an adaptive-bandwidth KDE based on Abramson's method, center bias and uniform components, and a state-of-the-art saliency model, to capture different spatial and semantic types of interobserver consistency, and optimize all parameters per image via leave-one-subject-out cross-validation. Our method yields substantially higher interobserver consistency estimates across multiple benchmarks, with median per-image gains of 5-15% in log-likelihood and up to 2 percentage points in AUC. For the most affected images -- precisely those most relevant to failure case analysis -- improvements exceed 25%. We leverage these improved estimates to identify and analyze remaining failure cases of state-of-the-art saliency models, demonstrating that significant headroom for model improvement remains. More broadly, our findings highlight that empirical fixation densities should not be treated as fixed ground truths but as evolving estimates that improve with better methodology.

Related papers

• The Good, the Bad, and the Brittle: Benchmarking Robustness and Generalisation of Histopathology Foundation Models [0.1274452325287335]
  We benchmarked twelve pathology foundation models (PFMs) and ResNet baselines.<n>PFMs consistently outperform CNNs in both robustness and domain generalisation, yet model scaling shows diminishing returns.<n> NR-Kfold analysis reveals systematic accuracy loss and increased variability when training-test similarity is broken.
  arXiv  Detail & Related papers  (2026-07-05T16:49:09Z)
• K-Sort Eval: Efficient Preference Evaluation for Visual Generation via Corrected VLM-as-a-Judge [51.93484138861584]
  The rapid development of visual generative models raises the need for more scalable and human-aligned evaluation methods.<n>We propose K-Sort Eval, a reliable and efficient VLM-based evaluation framework that integrates posterior correction and dynamic matching.<n>Experiments show that K-Sort Eval delivers evaluation results consistent with K-Sort Arena, typically requiring fewer than 90 model runs.
  arXiv  Detail & Related papers  (2026-02-10T05:07:46Z)
• The Flaw of Averages: Quantifying Uniformity of Performance on Benchmarks [32.00464870277127]
  We study benchmark reliability from a distributional perspective and introduce benchmark harmony.<n>High harmony is a desirable benchmark property, indicating that the aggregate metric reflects uniform competence across models.<n>By recommending that harmony should be reported alongside accuracy, we reframe evaluation from simple performance averages to a more robust, distributionally reliable measurement of performance.
  arXiv  Detail & Related papers  (2025-09-30T02:14:30Z)
• No-Reference Image Contrast Assessment with Customized EfficientNet-B0 [3.4527546378946]
  No reference image quality assessment NR IQA models struggled to accurately evaluate contrast distortions under diverse real world conditions.<n>In this study, we proposed a deep learning based framework for blind contrast quality assessment.<n>Models are modified with a contrast-aware regression head and trained end to end using targeted data augmentations.
  arXiv  Detail & Related papers  (2025-09-26T06:54:37Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• Solving Inverse Problems with FLAIR [68.87167940623318]
  We present FLAIR, a training-free variational framework that leverages flow-based generative models as prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• A Meaningful Perturbation Metric for Evaluating Explainability Methods [55.09730499143998]
  We introduce a novel approach, which harnesses image generation models to perform targeted perturbation.<n> Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity.<n>This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results.
  arXiv  Detail & Related papers  (2025-04-09T11:46:41Z)
• Average-DICE: Stationary Distribution Correction by Regression [7.193870502672509]
  Off-policy policy evaluation (OPE) has long suffered from stationary state distribution mismatch.<n>We introduce AVG-DICE, a computationally simple Monte Carlo estimator for the density ratio.<n>In our experiments, AVG-DICE is at least as accurate as state-of-the-art estimators and sometimes offers orders-of-magnitude improvements.
  arXiv  Detail & Related papers  (2025-03-03T23:14:02Z)
• A Simple and Generalist Approach for Panoptic Segmentation [57.94892855772925]
  We propose a simple generalist framework based on a deep encoder - shallow decoder architecture with per-pixel prediction.<n>We show that this is due to imbalance during training and propose a novel method for reducing it.<n>Our method achieves panoptic quality (PQ) of 55.1 on the challenging MS-COCO dataset.
  arXiv  Detail & Related papers  (2024-08-29T13:02:12Z)
• Unmasking Bias in Diffusion Model Training [40.90066994983719]
  Denoising diffusion models have emerged as a dominant approach for image generation. They still suffer from slow convergence in training and color shift issues in sampling. In this paper, we identify that these obstacles can be largely attributed to bias and suboptimality inherent in the default training paradigm.
  arXiv  Detail & Related papers  (2023-10-12T16:04:41Z)
• GAN-based Image Compression with Improved RDO Process [20.00340507091567]
  We present a novel GAN-based image compression approach with improved rate-distortion optimization process. To achieve this, we utilize the DISTS and MS-SSIM metrics to measure perceptual degeneration in color, texture, and structure. The proposed method outperforms the existing GAN-based methods and the state-of-the-art hybrid (i.e., VVC)
  arXiv  Detail & Related papers  (2023-06-18T03:21:11Z)
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models [76.76860707897413]
  Current best practice advocates for a large T to ensure that the forward dynamics brings the diffusion sufficiently close to a known and simple noise distribution. We show how an auxiliary model can be used to bridge the gap between the ideal and the simulated forward dynamics, followed by a standard reverse diffusion process.
  arXiv  Detail & Related papers  (2022-06-10T15:09:46Z)
• Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization [67.99082021804145]
  We propose an explicit solution to the COO problem, called Detail Enhanced Contrastive Loss (DECLoss) DECLoss utilizes the clustering property of contrastive learning to directly reduce the variance of the potential high-resolution distribution. We evaluate DECLoss on multiple super-resolution benchmarks and demonstrate that it improves the perceptual quality of PSNR-oriented models.
  arXiv  Detail & Related papers  (2022-01-04T08:30:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.