EXHIB: A Benchmark for Realistic and Diverse Evaluation of Function Similarity in the Wild

• URL: http://arxiv.org/abs/2604.01554v1
• Date: Thu, 02 Apr 2026 02:58:55 GMT
• Title: EXHIB: A Benchmark for Realistic and Diverse Evaluation of Function Similarity in the Wild
• Abstract summary: We introduce EXHIB, a benchmark comprising five realistic datasets collected from the wild.<n>We evaluate 9 representative models spanning multiple BFSD paradigms on EXHIB.<n>Our results show that robustness to low- and mid-level binary variations does not generalize to high-level semantic differences.
• Score: 9.69274678999646
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Binary Function Similarity Detection (BFSD) is a core problem in software security, supporting tasks such as vulnerability analysis, malware classification, and patch provenance. In the past few decades, numerous models and tools have been developed for this application; however, due to the lack of a comprehensive universal benchmark in this field, researchers have struggled to compare different models effectively. Existing datasets are limited in scope, often focusing on a narrow set of transformations or types of binaries, and fail to reflect the full diversity of real-world applications. We introduce EXHIB, a benchmark comprising five realistic datasets collected from the wild, each highlighting a distinct aspect of the BFSD problem space. We evaluate 9 representative models spanning multiple BFSD paradigms on EXHIB and observe performance degradations of up to 30% on firmware and semantic datasets compared to standard settings, revealing substantial generalization gaps. Our results show that robustness to low- and mid-level binary variations does not generalize to high-level semantic differences, underscoring a critical blind spot in current BFSD evaluation practices.

Related papers

• Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation [47.29900218955283]
  Mutual information estimation is a central problem in machine learning and statistics.<n>Existing benchmarks evaluate estimators on simplified, low-dimensional distributions, leaving their performance on complex, realistic data largely unexplored.<n>We address this gap with a comprehensive benchmarking framework grounded in a unified copula-theoretic perspective.
  arXiv  Detail & Related papers  (2026-07-03T16:55:55Z)
• Real-time Appearance-based Gaze Estimation for Open Domains [15.195617974731766]
  We propose a robust AGE framework that enhances generalization without requiring additional human-annotated data.<n>To mitigate the impact of anisotropic inter-dataset label deviation, we reformulate gaze regression as a multi-task learning problem.<n>Our MobileNet-based lightweight model generalization achieves performance competitive with the state-of-the-art (SOTA) UniGaze-H.
  arXiv  Detail & Related papers  (2026-03-27T19:33:18Z)
• BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing? [61.247730037229815]
  We introduce BeyondSWE, a comprehensive benchmark that broadens existing evaluations along two axes - resolution scope and knowledge scope.<n>To investigate the role of external knowledge, we develop SearchSWE, a framework that integrates deep search with coding abilities.<n>This work offers both a realistic, challenging evaluation benchmark and a flexible framework to advance research toward more capable code agents.
  arXiv  Detail & Related papers  (2026-03-03T17:52:01Z)
• Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era [110.83702639978469]
  Real-IAD Variety is the largest and most diverse IAD benchmark, comprising 198,960 high-resolution images across 160 distinct object categories.<n>Its diversity is ensured through comprehensive coverage of 28 industries, 24 material types, and 22 color variations.<n>Real-IAD Variety will be made publicly available to facilitate innovation in this critical field.
  arXiv  Detail & Related papers  (2025-11-01T12:58:02Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• UniFault: A Fault Diagnosis Foundation Model from Bearing Data [23.797786758616223]
  Existing machine fault diagnosis models are operation-specific with limited generalization across diverse datasets.<n>We introduce UniFault, a foundation model for fault diagnosis that systematically addresses these issues.<n>UniFault is pretrained on over 6.9 million samples spanning diverse FD datasets, enabling superior few-shot performance.
  arXiv  Detail & Related papers  (2025-04-02T05:34:27Z)
• Deep Incomplete Multi-view Clustering with Distribution Dual-Consistency Recovery Guidance [69.58609684008964]
  We propose BURG, a novel method for incomplete multi-view clustering with distriBution dUal-consistency Recovery Guidance.<n>We treat each sample as a distinct category and perform cross-view distribution transfer to predict the distribution space of missing views.<n>To compensate for the lack of reliable category information, we design a dual-consistency guided recovery strategy that includes intra-view alignment guided by neighbor-aware consistency and cross-view alignment guided by prototypical consistency.
  arXiv  Detail & Related papers  (2025-03-14T02:27:45Z)
• Weak-to-Strong Diffusion with Reflection [56.39451539396458]
  We propose Weak-to-Strong Diffusion (W2SD) to bridge the gap between an ideal model and a strong model.<n>W2SD steers latent variables along sampling trajectories toward regions of the real data distribution.<n>Extensive experiments demonstrate that W2SD significantly improves human preference, aesthetic quality, and prompt adherence.
  arXiv  Detail & Related papers  (2025-02-01T16:00:08Z)
• Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
  Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification. In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction. Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
  arXiv  Detail & Related papers  (2024-10-24T09:09:20Z)
• Understanding the AI-powered Binary Code Similarity Detection [41.39226562321616]
  AI-powered binary code similarity detection (BinSD) has been widely applied to program analysis. It is difficult to quantitatively understand to what extent the BinSD problem has been solved, especially in realworld applications. We present a systematic evaluation of state-of-the-art AI-powered BinSD approaches.
  arXiv  Detail & Related papers  (2024-10-10T02:13:01Z)
• Benchmark Granularity and Model Robustness for Image-Text Retrieval [44.045767657945895]
  We show how dataset granularity and query perturbations affect retrieval performance and robustness.<n>We show that richer captions consistently enhance retrieval, especially in text-to-image tasks.<n>Our results highlight variation in model robustness and a dataset-dependent relationship between caption granularity and sensitivity perturbation.
  arXiv  Detail & Related papers  (2024-07-21T18:08:44Z)
• Robustness Analysis on Foundational Segmentation Models [28.01242494123917]
  In this work, we perform a robustness analysis of Visual Foundation Models (VFMs) for segmentation tasks. We benchmark seven state-of-the-art segmentation architectures using 2 different datasets. Our findings reveal several key insights: VFMs exhibit vulnerabilities to compression-induced corruptions, despite not outpacing all of unimodal models in robustness, multimodal models show competitive resilience in zero-shot scenarios, and VFMs demonstrate enhanced robustness for certain object categories.
  arXiv  Detail & Related papers  (2023-06-15T16:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.