Reducing Maintenance Burden in Behaviour-Driven Development: A Paraphrase-Robust Duplicate-Step Detector with a 1.1M-Step Open Benchmark

• URL: http://arxiv.org/abs/2604.20462v2
• Date: Sun, 26 Apr 2026 22:53:22 GMT
• Title: Reducing Maintenance Burden in Behaviour-Driven Development: A Paraphrase-Robust Duplicate-Step Detector with a 1.1M-Step Open Benchmark
• Abstract summary: Behaviour-Driven Development suites accumulate step-duplicate duplication with documented maintenance cost.<n>We release the largest cross-organisational BDD step corpus to date.
• Score: 1.9537983097153042
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Context. Behaviour-Driven Development (BDD) suites in Gherkin accumulate step-text duplication with documented maintenance cost. Prior detectors either require runnable tests or are single-organisation, leaving a gap: a static, paraphrase-robust, step-level detector and a public benchmark to calibrate it. Objective. We release (i) the largest cross-organisational BDD step corpus to date, (ii) a labelled pair-level calibration benchmark, and (iii) a four-strategy detector with a consolidation-savings model linking clusters to ISO/IEC 25010 maintainability sub-characteristics. Method. The corpus contains 347 public GitHub repositories, 23,667 .feature files, and 1,113,616 Gherkin steps, SPDX-tagged. The detector layers exact hashing, normalised Levenshtein, sentence-transformer cosine, and a Levenshtein-banded hybrid. Calibration uses 1,020 manually labelled step pairs under a released rubric (60-pair overlap, Fleiss kappa = 0.84). We report precision, recall, and F1 with bootstrap 95% CIs under the primary rubric and a score-free relabelling, and benchmark against SourcererCC-style and NiCad-style lexical baselines. Results. Step-weighted exact-duplicate rate is 80.2%; median-repository rate is 58.6% (Spearman rho = 0.51). The top hybrid cluster has 20,737 occurrences across 2,245 files. Near-exact reaches F1 = 0.822 on score-free labels; semantic F1 = 0.906 under the primary rubric reflects a disclosed stratification artefact. Lexical baselines reach F1 = 0.761 and 0.799. The savings model estimates 893,357 corpus-wide eliminable step occurrences; on the median repository 62.5% of step lines are eliminable.

Related papers

• SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology [47.16406021922537]
  We introduce router, a lightweight GLiClass-based router that assigns a suitability score to each inference-time model label without autore generation.<n>For task generation, we construct a task structure with 23 families, 115 task types, 345 routable subtypes, 1,173 synthetic examples, and an axis of 30 domains.<n>We then train the Qwen3 decoder on these tasks, while explicitly separating learned request prediction from per-task policies for attributes such as eligibility, cost, cache reuse, safety, and sovereignty.
  arXiv  Detail & Related papers  (2026-09-02T08:40:07Z)
• Provenance, Not Behaviour: A Serialisation Artifact in Edge-IIoTset and a Leakage-Free Benchmark for Precision-Agriculture Intrusion Detection [0.0]
  Edge-IIoTset is the benchmark for machine-learning intrusion detection in the Internet of Things.<n>We show that much of that performance is not intrusion detection.<n>We produce AgriEdge: 1,276,122 rows, five devices with full attribution, and no column separating the classes above 0.0288.
  arXiv  Detail & Related papers  (2026-08-16T14:30:45Z)
• TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenance [8.657685524975246]
  We introduce a training-free fingerprint based on the top-$k$ vocabulary projections of late hidden states elicited by 250 fixed knowledge probes.<n>We evaluate the method on 32 open-weight models from nine families (0.6B--32B) with documented relationships.
  arXiv  Detail & Related papers  (2026-08-08T13:56:39Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Towards Persistent Case-Based Memory for Autonomous Data Science: A CBR-Augmented R&D-Agent with a Locally Deployable Small Language Model [0.0]
  We present CBR-augmented R&D-Agent, integrating a persistent CBR layer into Microsoft's R&D-Agent framework with a custom backend for Gemma 4 31B Dense.<n>Cases are stored as structured records with executable code snapshots and quality metadata.<n>Heuristic reuse detection across 108 retrieval events shows high semantic relevance.
  arXiv  Detail & Related papers  (2026-06-03T12:56:11Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Mining Subscenario Refactoring Opportunities in Behaviour-Driven Software Test Suites: ML Classifiers and LLM-Judge Baselines [1.9537983097153042]
  Behaviour-Driven Development (BDD) software test suites accumulate duplicated step subsequences.<n>Three published patterns are available (within-file Background, within-repo reusable-scenario invocation, cross-organisational shared higher-level step)<n>No prior work automates which recurring subsequences are worth extracting or which mechanism applies.
  arXiv  Detail & Related papers  (2026-05-14T08:38:04Z)
• OCRR: A Benchmark for Online Correction Recovery under Distribution Shift [0.0]
  OCRR (Online Correction Recovery Rate) is a benchmark that streams a corpus through a classification system and applies oracle or corrections to wrong predictions.<n>It beats the next-best published continual-learning baseline by 32.6 percentage points at equal memory budget.<n>It also beats LoRA-on-DeBERTa-v3-large by 84.6 percentage points on retention.
  arXiv  Detail & Related papers  (2026-05-04T20:51:07Z)
• AnomalyGen: Enhancing Log-Based Anomaly Detection with Code-Guided Data Augmentation [42.87177529900358]
  AnomalyGen is a novel framework that augments training data by synthesizing labeled log sequences from source code.<n> Evaluations on HDFS and Zookeeper across 12 diverse anomaly detection models show AnomalyGen consistently improves performance.
  arXiv  Detail & Related papers  (2026-04-13T07:24:43Z)
• Detecting Media Clones in Cultural Repositories Using a Positive Unlabeled Learning Approach [0.0]
  We formulate curator-in-the-loop duplicate discovery in the AtticPOT repository as a Positive-Unlabeled (PU) learning problem.<n>We train a lightweight per-query Clone on augmented views of the anchor and score the unlabeled repository with an interpretable threshold on the latent l norm.<n>The system proposes candidates for curator verification, uncovering cross-record duplicates that were not verified a priori.
  arXiv  Detail & Related papers  (2026-04-05T11:41:29Z)
• BibTeX Citation Hallucinations in Scientific Publishing Agents: Evaluation and Mitigation [34.429649156970015]
  Prior evaluations tested base models without search, which does not reflect current practice.<n>Three search-enabled frontier models generate Bib entries scored on nine fields and a six-way error taxonomy.<n>Overall accuracy is 83.6%, but only 50.9% of entries are fully correct.
  arXiv  Detail & Related papers  (2026-04-03T16:30:58Z)
• Structurally Human, Semantically Biased: Detecting LLM-Generated References with Embeddings and GNNs [7.158582263989492]
  We build paired citation graphs, ground truth and GPT-4o-generated (from parametric knowledge) for 10,000 papers from SciSciNet.<n>We show that GPT barely separates from ground truth (RF accuracy $approx$ 0.60) despite cleanly rejecting the random baseline.<n> embeddings sharply increase separability: RF on aggregated embeddings reaches $approx$ 0.83, and GNNs with embedding node features achieve 93% test accuracy on GPT vs. ground truth.
  arXiv  Detail & Related papers  (2026-01-28T15:37:31Z)
• A Two-Stage Strategy for Mitosis Detection Using Improved YOLO11x Proposals and ConvNeXt Classification [5.1547008655164195]
  MIDOG 2025 Track 1 requires mitosis detection in whole-slideimages (WSIs) containing non-tumor, inflamed, and necrotic re-gions.<n>Due to the complicated and heterogeneous context, there are often false positives and falsenegatives, thus degrading the detection F1-score.<n>We propose a two-stage framework to generate mitosis candidates and filter out false positives.<n>On the MIDOG 2025 Track 1 preliminary test set, thealgorithm scores an F1 score of 0.7587.
  arXiv  Detail & Related papers  (2025-09-01T15:46:28Z)
• Certified Robustness Under Bounded Levenshtein Distance [55.54271307451233]
  We propose the first method for computing the Lipschitz constant of convolutional classifiers with respect to the Levenshtein distance.<n>Our method, LipsLev, is able to obtain $38.80$% and $13.93$% verified accuracy at distance $1$ and $2$ respectively.
  arXiv  Detail & Related papers  (2025-01-23T13:58:53Z)
• Hybrid-Segmentor: A Hybrid Approach to Automated Fine-Grained Crack Segmentation in Civil Infrastructure [52.2025114590481]
  We introduce Hybrid-Segmentor, an encoder-decoder based approach that is capable of extracting both fine-grained local and global crack features. This allows the model to improve its generalization capabilities in distinguish various type of shapes, surfaces and sizes of cracks. The proposed model outperforms existing benchmark models across 5 quantitative metrics (accuracy 0.971, precision 0.804, recall 0.744, F1-score 0.770, and IoU score 0.630), achieving state-of-the-art status.
  arXiv  Detail & Related papers  (2024-09-04T16:47:16Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)
• Appeal: Allow Mislabeled Samples the Chance to be Rectified in Partial Label Learning [55.4510979153023]
  In partial label learning (PLL), each instance is associated with a set of candidate labels among which only one is ground-truth. To help these mislabeled samples "appeal," we propose the first appeal-based framework.
  arXiv  Detail & Related papers  (2023-12-18T09:09:52Z)
• Conformal Predictor for Improving Zero-shot Text Classification Efficiency [37.745518881553416]
  We reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% respectively. With a suitable CP for each dataset, we reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% respectively.
  arXiv  Detail & Related papers  (2022-10-23T05:19:50Z)
• A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments [70.1864008701113]
  Bots are used in Github repositories to automate repetitive activities that are part of the distributed software development process. This paper proposes a ground-truth dataset, based on a manual analysis with high interrater agreement, of pull request and issue comments in 5,000 distinct Github accounts. We propose an automated classification model to detect bots, taking as main features the number of empty and non-empty comments of each account, the number of comment patterns, and the inequality between comments within comment patterns.
  arXiv  Detail & Related papers  (2020-10-07T09:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.