PROBE-Web: An Interactive System for Probing Evaluation Landscapes of Knowledge Graph Completion Models

• URL: http://arxiv.org/abs/2606.08926v1
• Date: Mon, 08 Jun 2026 02:08:23 GMT
• Title: PROBE-Web: An Interactive System for Probing Evaluation Landscapes of Knowledge Graph Completion Models
• Abstract summary: PROBE-Web is an interactive system for probing diverse evaluation landscapes for KGC models.<n>It enables users to evaluate flexibly KGC models by adjusting two critical perspectives: (P1) predictive sharpness and (P2) popularity-bias robustness.
• Score: 0.5271072635937927
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge graph completion (KGC) models are commonly evaluated using rank-based metrics such as MRR and Hits@K, despite different users often requiring different evaluation perspectives. In this demo, we present PROBE-Web, an interactive system for probing diverse evaluation landscapes for KGC models. PROBE-Web enables users to flexibly evaluate KGC models by adjusting two critical perspectives: (P1) predictive sharpness and (P2) popularity-bias robustness. Through a user-friendly GUI, users easily evaluate multiple KGC models and analyze their strengths and weaknesses. PROBE-Web provides four key functionalities: (1) conventional evaluation toolkit, (2) flexible perspective-aware evaluation, (3) explainable case studies, and (4) evaluation landscape exploration. We believe that PROBE-Web can help users better understand KGC models aligning with their objectives.

Related papers

• Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Models [74.1961755473754]
  Existing evaluation methods rely on rigid pipelines that overlook specific user needs and provide numerical results without clear explanations.<n>We propose the Evaluation Agent framework, which employs human-like strategies for efficient, dynamic, multi-round evaluations.<n>We show that Evaluation Agent reduces evaluation time to 10% of traditional methods while delivering comparable results.
  arXiv  Detail & Related papers  (2026-08-10T14:42:10Z)
• DualEval: Joint Model-Item Calibration for Unified LLM Evaluation [67.99119926012686]
  DualEval is a latent model-item calibration framework that represents models and evaluation items in a shared space.<n>We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries.
  arXiv  Detail & Related papers  (2026-06-24T22:40:46Z)
• Generalized Rank-based Evaluation for Knowledge Graph Completion: Perspectives, Framework, and Analyses [2.7836206220202757]
  We introduce two important perspectives for KGC evaluation that are overlooked by existing evaluation metrics.<n>We propose a generalized evaluation framework, PROBE, that estimates the score of each prediction based on a desired level of predictive sharpness.<n>We show that PROBE better maintains such consistency, providing a more reliable estimate of intrinsic model performance than existing metrics.
  arXiv  Detail & Related papers  (2026-06-08T02:00:38Z)
• How Sharp and Bias-Robust is a Model? Dual Evaluation Perspectives on Knowledge Graph Completion [0.5271072635937927]
  Knowledge graph completion (KGC) aims to predict missing facts from the observed KG.<n>Existing metrics overlook two key perspectives for KGC evaluation.<n>We propose a novel evaluation framework (PROBE)<n> Experiments on real-world KGs reveal that existing metrics tend to over- or under-estimate the accuracy of KGC models.
  arXiv  Detail & Related papers  (2025-12-06T04:49:29Z)
• Constantly Improving Image Models Need Constantly Improving Benchmarks [109.39018167487103]
  We present ECHO, a framework for constructing benchmarks directly from real-world evidence of model use.<n>Applying this framework to GPT-4o Image Gen, we construct a dataset of over 31,000 prompts curated from social media posts.
  arXiv  Detail & Related papers  (2025-10-16T17:59:30Z)
• MEF: A Systematic Evaluation Framework for Text-to-Image Models [21.006921005280493]
  Current evaluations rely on either ELO for overall ranking or MOS for dimension-specific scoring.<n>We introduce the Magic Evaluation Framework (MEF), a systematic and practical approach for evaluating T2I models.<n>We release our evaluation framework and make Magic-Bench-377 fully open-source to advance research in the evaluation of visual generative models.
  arXiv  Detail & Related papers  (2025-09-22T15:32:42Z)
• Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models [51.067146460271466]
  Evaluation of visual generative models can be time-consuming and computationally expensive.<n>We propose the Evaluation Agent framework, which employs human-like strategies for efficient, dynamic, multi-round evaluations.<n>It offers four key advantages: 1) efficiency, 2) promptable evaluation tailored to diverse user needs, 3) explainability beyond single numerical scores, and 4) scalability across various models and tools.
  arXiv  Detail & Related papers  (2024-12-10T18:52:39Z)
• CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolution [74.41064280094064]
  textbfJudger-1 is the first open-source textbfall-in-one judge LLM. CompassJudger-1 is a general-purpose LLM that demonstrates remarkable versatility. textbfJudgerBench is a new benchmark that encompasses various subjective evaluation tasks.
  arXiv  Detail & Related papers  (2024-10-21T17:56:51Z)
• Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform [64.76104135495576]
  We propose a novel anonymous crowd-sourcing evaluation platform, BingJian, for large language models. Through this platform, users have the opportunity to submit their questions, testing the models on a personalized and potentially broader range of capabilities.
  arXiv  Detail & Related papers  (2024-03-13T07:31:20Z)
• MMBench: Is Your Multi-modal Model an All-around Player? [114.45702807380415]
  We propose MMBench, a benchmark for assessing the multi-modal capabilities of vision-language models. MMBench is meticulously curated with well-designed quality control schemes. MMBench incorporates multiple-choice questions in both English and Chinese versions.
  arXiv  Detail & Related papers  (2023-07-12T16:23:09Z)
• Evaluating how interactive visualizations can assist in finding samples where and how computer vision models make mistakes [1.76602679361245]
  We present two interactive visualizations in the context of Sprite, a system for creating Computer Vision (CV) models. We study how these visualizations help Sprite's users identify (evaluate) and select (plan) images where a model is struggling and can lead to improved performance.
  arXiv  Detail & Related papers  (2023-05-19T14:43:00Z)
• KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models [76.01814380927507]
  KGxBoard is an interactive framework for performing fine-grained evaluation on meaningful subsets of the data. In our experiments, we highlight the findings with the use of KGxBoard, which would have been impossible to detect with standard averaged single-score metrics.
  arXiv  Detail & Related papers  (2022-08-23T15:11:45Z)
• Exploiting Behavioral Consistence for Universal User Representation [11.290137806288191]
  We focus on developing universal user representation model. The obtained universal representations are expected to contain rich information. We propose Self-supervised User Modeling Network (SUMN) to encode behavior data into the universal representation.
  arXiv  Detail & Related papers  (2020-12-11T06:10:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.