Picid: A Modular Evaluation Infrastructure for Reproducible PHM Across Tasks and Domains

• URL: http://arxiv.org/abs/2605.28345v1
• Date: Wed, 27 May 2026 11:50:52 GMT
• Title: Picid: A Modular Evaluation Infrastructure for Reproducible PHM Across Tasks and Domains
• Abstract summary: We introduce picid, a modular evaluation infrastructure that formalizes the PHM evaluation pipeline as an explicit, executable, and reproducible protocol.<n>picid enforces deterministic, leakage-safe dataset construction while remaining flexible across diverse PHM settings.<n>We demonstrate picid through an empirical evaluation of thirteen models on twelve datasets spanning batteries, bearings, turbofan engines, hydraulics, filtration systems, and buildings.
• Score: 13.382059615065167
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Progress in Prognostics and Health Management (PHM) is hindered by the lack of standardized and reusable evaluation practices across tasks, datasets, and application domains. Reported results are often difficult to reproduce and compare, as key protocol choices, such as data splits, preprocessing, label alignment, temporal windowing, and metrics, are often implicit or implemented ad hoc. We introduce \picid, a modular evaluation infrastructure that formalizes the PHM evaluation pipeline as an explicit, executable, and reproducible protocol. Through well-defined abstractions, \picid enforces deterministic, leakage-safe dataset construction while remaining flexible across diverse PHM settings. The framework supports fault detection, diagnostics, and prognostics through a unified interface and can be extended to new datasets and model classes without violating protocol invariants. By standardizing data contracts and evaluation boundaries, \picid also enables fair cross-task comparisons across diagnostics (classification) and prognostics (regression), allowing identical model families to be evaluated consistently across heterogeneous settings. We demonstrate \picid through an empirical evaluation of thirteen models on twelve datasets spanning batteries, bearings, turbofan engines, hydraulics, filtration systems, and buildings. This work establishes a reusable foundation for standardized, fair and reproducible evaluation in PHM.

Related papers

• MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks [1.0230975056328007]
  We present MEDSEGBENCHMARKER (MSB), a configuration-driven framework for controlled benchmarking of 2D datasets.<n>MSB exports sample- and class-level pixel counts and predictions together with the evaluation context.<n>We demonstrate MSB in a case study involving three heterogeneous 2D datasets and multiple vision models evaluated at 256- and 512-pixel input resolutions.
  arXiv  Detail & Related papers  (2026-08-30T09:20:38Z)
• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• TimeLAVA: Learning-Agnostic Data Valuation for Time Series [48.76711740818533]
  TimeLAVA is a learning-agnostic framework that values temporal segments by their marginal contribution to minimize distributional discrepancy between evaluated and reference data.<n>We provide theoretical guarantees linking valuation to model-agnostic generalization and prove bounded sensitivity to contamination.
  arXiv  Detail & Related papers  (2026-06-17T06:16:16Z)
• Towards Unified and Data-Efficient Prognostics and Health Management with Tabular Foundation Models [13.382059615065167]
  Data-driven Prognostics and Health Management (PHM) uses time-varying condition-monitoring data to diagnose system states and estimate remaining useful life in engineered assets.<n>These tasks are central to maintenance planning, but industrial PHM data are often fragmented, partially observed, and poorly labeled.<n>We propose a framework for applying Tabular Foundation Models to industrial time series using in-context learning.
  arXiv  Detail & Related papers  (2026-06-03T22:00:29Z)
• CellScientist: Dual-Space Hierarchical Orchestration for Closed-Loop Refinement of Virtual Cell Models [62.281480231694]
  Virtual Cell Modeling (VCM) requires models that not only predict perturbation responses, but also support targeted revision when predictions fail.<n>Current LLM-assisted modeling face a refinement-routing problem: prediction discrepancies are observed through executable implementations.<n>We propose CellScientist, a dual-space hierarchical framework that couples a high-level hypothesis space with a low-level executable implementation space.
  arXiv  Detail & Related papers  (2026-05-08T06:40:24Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• bioLeak: Leakage-Aware Modeling and Diagnostics for Machine Learning in R [0.0]
  BioLeak is an R package for constructing leakage-aware resampling and auditing fitted models for common leakage mechanisms.<n>The implementation supports binary classification, multiclass classification, regression, and survival analysis, with task-specific metrics and S4 containers for splits, fits, audits, and inflation summaries.
  arXiv  Detail & Related papers  (2026-04-13T04:01:31Z)
• UniDial-EvalKit: A Unified Toolkit for Evaluating Multi-Faceted Conversational Abilities [70.79422099851506]
  We present UniDial-EvalKit (UDE), a unified evaluation toolkit for assessing interactive AI systems.<n>UDE standardizes heterogeneous data formats into a universal schema, streamlines complex evaluation pipelines through a modular architecture, and aligns metric calculations under a consistent scoring interface.
  arXiv  Detail & Related papers  (2026-03-24T13:01:31Z)
• PIDSMaker: Building and Evaluating Provenance-based Intrusion Detection Systems [0.5830619388189558]
  PIDSMaker is an open-source framework for developing and evaluating PIDSs under consistent visualization protocols.<n>It consolidates eight state-of-the-art systems into a modular architecture with standardized preprocessing and ground-truth labels.<n>A YAML-based configuration interface supports rapid prototyping by composing components across systems without code changes.
  arXiv  Detail & Related papers  (2026-01-30T13:45:33Z)
• Procrustean Bed for AI-Driven Retrosynthesis: A Unified Framework for Reproducible Evaluation [0.0]
  RetroCast is a unified evaluation suite that standardizes heterogeneous model outputs into a common schema.<n>We evaluate leading search-based and sequence-based algorithms on a new suite of standardized benchmarks.
  arXiv  Detail & Related papers  (2025-12-08T01:26:39Z)
• Generating Highly Structured Test Inputs Leveraging Constraint-Guided Graph Refinement [4.121384394709256]
  This study investigates whether test inputs for structured domains can be unified through a graph-based representation.<n>We will evaluate the effectiveness of this approach in enhancing input validity and semantic preservation across eight AI systems.
  arXiv  Detail & Related papers  (2025-07-28T18:54:04Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective.<n>The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning.<n>The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Concurrent Misclassification and Out-of-Distribution Detection for Semantic Segmentation via Energy-Based Normalizing Flow [0.0]
  Recent semantic segmentation models accurately classify test-time examples that are similar to a training dataset distribution. We propose a generative model for concurrent in-distribution misclassification (IDM) and OOD detection that relies on a normalizing flow framework. FlowEneDet achieves promising results on Cityscapes, Cityscapes-C, FishyScapes and SegmentMeIfYouCan benchmarks in IDM/OOD detection when applied to pretrained DeepLabV3+ and SegFormer semantic segmentation models.
  arXiv  Detail & Related papers  (2023-05-16T17:02:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.