Mining Subscenario Refactoring Opportunities in Behaviour-Driven Software Test Suites: ML Classifiers and LLM-Judge Baselines

• URL: http://arxiv.org/abs/2605.14568v1
• Date: Thu, 14 May 2026 08:38:04 GMT
• Title: Mining Subscenario Refactoring Opportunities in Behaviour-Driven Software Test Suites: ML Classifiers and LLM-Judge Baselines
• Abstract summary: Behaviour-Driven Development (BDD) software test suites accumulate duplicated step subsequences.<n>Three published patterns are available (within-file Background, within-repo reusable-scenario invocation, cross-organisational shared higher-level step)<n>No prior work automates which recurring subsequences are worth extracting or which mechanism applies.
• Score: 1.9537983097153042
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context. Behaviour-Driven Development (BDD) software test suites accumulate duplicated step subsequences. Three published refactoring patterns are available (within-file Background, within-repo reusable-scenario invocation, cross-organisational shared higher-level step), but no prior work automates which recurring subsequences are worth extracting or which mechanism applies. Objective. Rank recurring step subsequences ("slices") by refactoring suitability (extraction-worthy), pre-map each to one of the three patterns, and quantify prevalence across the public BDD ecosystem. Method. Every contiguous L-step window (L in [2, 18]) in a 339-repository / 276-upstream-owner Gherkin corpus is keyed by paraphrase-robust cluster identifiers and counted under three scopes. Sentence-BERT (SBERT) / Uniform Manifold Approximation and Projection (UMAP) / Hierarchical Density-Based Clustering (HDBSCAN) recovers paraphrase-equivalent slices. Three authors label a stratified 200-slice pool against a written rubric. An eXtreme Gradient Boosting (XGBoost) extraction-worthy classifier trained under 5-fold cross-validation is compared with a tuned rule baseline and two open-weight Large Language Model (LLM) judges. Results. The miner produces 5,382,249 slices collapsing to 692,020 recurring patterns. Three-author Fleiss' kappa = 0.56 (extraction-worthy) and 0.79 (mechanism). The classifier reaches out-of-fold F1 = 0.891 (95% CI [0.852, 0.927]), outperforming both the rule baseline (F1 = 0.836, p = 0.017) and the better LLM judge (F1 = 0.728, p < 1e-4). 75.0%, 59.5%, and 11.7% of scenarios carry a within-file Background, within-repo reusable-scenario, or cross-organisational shared-step candidate. Conclusion. Paraphrase-robust subscenario discovery yields a corpus-wide census of BDD refactoring opportunities; pipeline, classifier predictions, labelled pool, and rubric are released under Apache-2.0.

Related papers

• Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations [4.342406076796542]
  We present a three-stage, chunk-side-only semantic chunking pipeline.<n>The title chain reuses the document's own header hierarchy instead of a generated summary.<n>The pipeline improves MRR@5 from 0.374 to 0.463 on the full set and from 0.828 to 0.925 on the answerable subset.
  arXiv  Detail & Related papers  (2026-08-01T19:01:46Z)
• BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents [10.767742708000897]
  Stepwise group-based RL is an attractive way to train long-horizon LLM agents without a learned critic.<n>We show that current agentic variants violate this assumption through a state-action credit mismatch.<n>We introduce BiPACE, a drop-in advantage estimator that fixes both sides without adding a critic, auxiliary loss, or extra rollouts.
  arXiv  Detail & Related papers  (2026-06-24T08:32:42Z)
• Cross-Dialect Generalization Without Retraining: Benchmarks and Evaluation of Schema-Derived Constrained Decoding for MLIR [0.0]
  Multi-Level Intermediate Representation (MLIR) underlies modern ML compiler infrastructure (TensorFlow, JAX/StableHLO, PyTorch Inductor, IREE)<n>MLIR is also by design: new dialects ship per application domain, so a fine-tuned model per dialect does not scale.<n>We release four natural-language-to-MLIR benchmarks across three dialects.
  arXiv  Detail & Related papers  (2026-05-14T02:08:13Z)
• Reducing Maintenance Burden in Behaviour-Driven Development: A Paraphrase-Robust Duplicate-Step Detector with a 1.1M-Step Open Benchmark [1.9537983097153042]
  Behaviour-Driven Development suites accumulate step-duplicate duplication with documented maintenance cost.<n>We release the largest cross-organisational BDD step corpus to date.
  arXiv  Detail & Related papers  (2026-04-22T11:44:05Z)
• Layer-wise MoE Routing Locality under Shared-Prefix Code Generation: Token-Identity Decomposition and Compile-Equivalent Fork Redundancy [0.0]
  In LLM-based code generation, multiple code candidates are often generated in parallel from the same prompt.<n>We study Qwen3.5-35B-A3B-FP8 by performing tree-search-based branching generation from a shared prefix.<n>We show that diversity in top-P search, including beam search, poses a significant challenge.
  arXiv  Detail & Related papers  (2026-04-19T00:56:08Z)
• HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy [54.91468501159335]
  Cross-document relation extraction (RE) aims to identify relations between the head tail entities located in different documents.<n>We propose an inference strategy that improves reliability through multi-view verification at each level.
  arXiv  Detail & Related papers  (2026-04-09T07:55:27Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• OpenSanctions Pairs: Large-Scale Entity Matching with LLMs [0.9131359219276399]
  We release OpenSanctions Pairs, a large-scale entity matching benchmark derived from real-world international sanctions aggregation and analyst deduplication.<n>The dataset contains 755,540 labeled pairs spanning 293 heterogeneous sources across 31 countries.<n>Off-the-shelf LLMs substantially outperform the production rule-based baseline.
  arXiv  Detail & Related papers  (2026-02-24T06:25:49Z)
• Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models [96.0074341403456]
  Inference-time compute has re-emerged as a practical way to improve LLM reasoning.<n>Most test-time scaling (TTS) algorithms rely on autoregressive decoding.<n>We propose Prism, an efficient TTS framework for dLLMs.
  arXiv  Detail & Related papers  (2026-02-02T09:14:51Z)
• Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation [19.671138538152213]
  We create a probabilistic framework for cross-tokenizer likelihood scoring.<n>Our approach improves GSM8K accuracy by more than 2% over the current state of the art.
  arXiv  Detail & Related papers  (2025-12-16T22:49:42Z)
• Lookahead Tree-Based Rollouts for Enhanced Trajectory-Level Exploration in Reinforcement Learning with Verifiable Rewards [48.321707628011005]
  Lookahead Tree-Based Rollouts (LATR) is a novel rollout strategy designed to explicitly promote trajectory-level diversity.<n>LATR accelerates policy learning by 131% on average and improves final pass@1 performance by 4.2%.
  arXiv  Detail & Related papers  (2025-10-28T11:12:02Z)
• LLM-guided Hierarchical Retrieval [54.73080745446999]
  LATTICE is a hierarchical retrieval framework that enables an LLM to reason over and navigate large corpora with logarithmic search complexity.<n>A central challenge in such LLM-guided search is that the model's relevance judgments are noisy, context-dependent, and unaware of the hierarchy.<n>Our framework achieves state-of-the-art zero-shot performance on the reasoning-intensive BRIGHT benchmark.
  arXiv  Detail & Related papers  (2025-10-15T07:05:17Z)
• Revisiting Foreground and Background Separation in Weakly-supervised Temporal Action Localization: A Clustering-based Approach [48.684550829098534]
  Weakly-supervised temporal action localization aims to localize action instances in videos with only video-level action labels. We propose a novel clustering-based F&B separation algorithm. We evaluate our method on three benchmarks: THUMOS14, ActivityNet v1.2 and v1.3.
  arXiv  Detail & Related papers  (2023-12-21T18:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.