Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs

• URL: http://arxiv.org/abs/2605.06305v1
• Date: Thu, 07 May 2026 14:08:22 GMT
• Title: Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs
• Abstract summary: This paper investigates whether Large Language Models can support taxonomy-agnostic annotation of explicitly transmitted PII values in HTTP message bodies.<n>We introduce a multi-stage LLM-based pipeline that combines deterministic pre-processing with label-level classification, targeted instance-level value annotation, and output validation.<n>Results show that the pipeline accurately detects PII types and extracts corresponding values for concrete PII.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated privacy audits of web and mobile applications often analyse outbound HTTP traffic to detect Personally Identifiable Information (PII) leakage. However, existing learning-based detectors typically depend on scarce, manually labelled traffic and are tightly coupled to fixed label taxonomies, limiting transferability across domains and evolving definitions of PII. This paper investigates whether Large Language Models (LLMs) can support taxonomy-agnostic annotation of explicitly transmitted PII values in HTTP message bodies when the taxonomy is provided at runtime. We introduce a multi-stage LLM-based pipeline that combines deterministic pre-processing with label-level classification, targeted instance-level value annotation, and output validation. To enable controlled evaluation and exemplar-based prompting without relying on sensitive real-user captures, we further propose an LLM-based generator for synthetic HTTP traffic with manually validated, taxonomy-derived PII annotations. We evaluate the approach across three taxonomies spanning different PII domains and granularity levels. Results show that the pipeline accurately detects PII types and extracts corresponding values for concrete PII taxonomies. Overall, our findings position LLMs as a promising foundation for flexible, taxonomy-agnostic traffic annotation and for creating labelled data under evolving privacy taxonomies.

Related papers

• CAPID: Context-Aware PII Detection for Question-Answering Systems [2.538582648751871]
  We propose CAPID, a practical approach that fine-tunes a locally owned small language model (SLM) that filters sensitive information before it is passed to LLMs for QA.<n>Existing datasets do not capture the context-dependent relevance of PII needed to train such a model effectively.<n>Our experiments show that relevance-aware PII detection with a fine-tuned SLM substantially outperforms existing baselines in span, relevance and type accuracy.
  arXiv  Detail & Related papers  (2026-02-10T18:41:31Z)
• Large Language Models as Automatic Annotators and Annotation Adjudicators for Fine-Grained Opinion Analysis [3.186130813218338]
  In this work, we use a declarative annotation pipeline to identify fine-grained opinion spans in text.<n>We show that LLMs can serve as automatic annotators and adjudicators, achieving high Inter-Annotator Agreement across individual LLM-based annotators.<n>This reduces the cost and human effort needed to create these fine-grained opinion-annotated datasets.
  arXiv  Detail & Related papers  (2026-01-23T14:52:56Z)
• EReLiFM: Evidential Reliability-Aware Residual Flow Meta-Learning for Open-Set Domain Generalization under Noisy Labels [85.78886153628663]
  Open-Set Domain Generalization aims to enable deep learning models to recognize unseen categories in new domains.<n>Label noise hinders open-set domain generalization by corrupting source-domain knowledge.<n>We propose Evidential Reliability-Aware Residual Flow Meta-Learning (EReLiFM) to bridge domain gaps.
  arXiv  Detail & Related papers  (2025-10-14T16:23:11Z)
• You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs [50.54173262572369]
  Large language models (LLMs) are increasingly deployed in specialized domains such as finance, medicine, and agriculture.<n>We study label-free test-time adaptation for language models and present SyTTA, an inference-time framework that adapts models on-the-fly without additional supervision.
  arXiv  Detail & Related papers  (2025-10-11T14:00:39Z)
• SCAN: Structured Capability Assessment and Navigation for LLMs [54.54085382131134]
  textbfSCAN (Structured Capability Assessment and Navigation) is a practical framework that enables detailed characterization of Large Language Models.<n>SCAN incorporates four key components:.<n>TaxBuilder, which extracts capability-indicating tags from queries to construct a hierarchical taxonomy;.<n>RealMix, a query synthesis and filtering mechanism that ensures sufficient evaluation data for each capability tag;.<n>A PC$2$-based (Pre-Comparison-derived Criteria) LLM-as-a-Judge approach achieves significantly higher accuracy compared to classic LLM-as-a-Judge method
  arXiv  Detail & Related papers  (2025-05-10T16:52:40Z)
• Permissive Information-Flow Analysis for Large Language Models [21.563132267220073]
  Large Language Models (LLMs) are rapidly becoming commodity components of larger software systems.<n>This poses natural security and privacy problems: poisoned data retrieved from one component can change the model's behavior and compromise the entire system.<n>We propose a novel, more permissive approach to propagate information flow labels through LLM queries.
  arXiv  Detail & Related papers  (2024-10-04T00:25:43Z)
• TnT-LLM: Text Mining at Scale with Large Language Models [24.731544646232962]
  Large Language Models (LLMs) automate the process of end-to-end label generation and assignment with minimal human effort. We show that TnT-LLM generates more accurate and relevant label when compared against state-of-the-art baselines. We also share our practical experiences and insights on the challenges and opportunities of using LLMs for large-scale text mining in real-world applications.
  arXiv  Detail & Related papers  (2024-03-18T18:45:28Z)
• Exploiting Contextual Target Attributes for Target Sentiment Classification [53.30511968323911]
  Existing PTLM-based models for TSC can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. We present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes.
  arXiv  Detail & Related papers  (2023-12-21T11:45:28Z)
• ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks [91.55895047448249]
  This paper presents ReEval, an LLM-based framework using prompt chaining to perturb the original evidence for generating new test cases. We implement ReEval using ChatGPT and evaluate the resulting variants of two popular open-domain QA datasets. Our generated data is human-readable and useful to trigger hallucination in large language models.
  arXiv  Detail & Related papers  (2023-10-19T06:37:32Z)
• A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning [12.24664973838839]
  Since test suites are costly to execute, industry needs methods for test case prioritisation ( TCP) Recent TCP methods use machine learning (ML) to exploit the information known about the system under test (SUT) and its test cases. This paper analyses two decades of TCP research, and presents a taxonomy of 91 information attributes that have been used.
  arXiv  Detail & Related papers  (2022-01-16T13:13:29Z)
• Group-aware Label Transfer for Domain Adaptive Person Re-identification [179.816105255584]
  Unsupervised Adaptive Domain (UDA) person re-identification (ReID) aims at adapting the model trained on a labeled source-domain dataset to a target-domain dataset without any further annotations. Most successful UDA-ReID approaches combine clustering-based pseudo-label prediction with representation learning and perform the two steps in an alternating fashion. We propose a Group-aware Label Transfer (GLT) algorithm, which enables the online interaction and mutual promotion of pseudo-label prediction and representation learning.
  arXiv  Detail & Related papers  (2021-03-23T07:57:39Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.