Co-Scraper: query-aware DOM Pruning and Reusable Scraper Synthesis for Lightweight Web Data Extraction

• URL: http://arxiv.org/abs/2606.14821v1
• Date: Fri, 12 Jun 2026 12:37:40 GMT
• Title: Co-Scraper: query-aware DOM Pruning and Reusable Scraper Synthesis for Lightweight Web Data Extraction
• Abstract summary: Co-Scraper is a framework capable of handling the hierarchical complexity of long HTML documents.<n>On the test set of SWDE, Co-Scraper achieves state-of-the-art performance with an F1 score of 94.78% and a reuse success rate of 90.39%.
• Score: 27.756601321209832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The abundant and heterogeneous nature of web content necessitates automated information extraction, and generating scrapers that can be reused across similar web pages offers an effective solution for scalable data extraction. In this work, we propose Co-Scraper, a two-stage framework capable of handling the hierarchical complexity of long HTML documents. By integrating a query-aware DOM pruning mechanism with stable extraction strategy induction, Co-Scraper can effectively transforms web content into executable programmatic wrappers using a fine-tuned Qwen3-8B model. On the test set of SWDE, Co-Scraper achieves state-of-the-art performance with an F1 score of 94.78% and a reuse success rate of 90.39%. This framework significantly enhances the accuracy and resilience of data extraction, providing a highly efficient approach for web data acquisition tasks.

Related papers

• BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes [54.96895469970198]
  BrowserForge generates web interaction data at scale by driving many browser sandboxes in parallel over the open web.<n> BrowserForge couples three components: an open-web sourcing stage that exposes the agent to hundreds of thousands of real, openly reachable websites.<n>A rule-plus-model cleaning pipeline removes failed runs and rewrites the surviving reasoning into a single unified chain-of-thought style.
  arXiv  Detail & Related papers  (2026-08-25T17:35:42Z)
• Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pretraining [78.36592534300839]
  We show that for structured content such as tables and code blocks, extractor choice can significantly impact downstream task performance.<n>This suggests a simple intervention: by taking a Union over different extractors, we can increase the token yield of DCLM-Baseline by up to 71%.
  arXiv  Detail & Related papers  (2026-02-23T06:41:57Z)
• An Index-based Approach for Efficient and Effective Web Content Extraction [38.40209116782093]
  We introduce Index-based Web Content Extraction.<n>We partition HTML into structure-aware, addressable segments, and extract only the positional indices of content relevant to a given query.<n>This method decouples extraction latency from content length, enabling rapid, query-relevant extraction.
  arXiv  Detail & Related papers  (2025-12-07T03:18:19Z)
• Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM [35.10225876152952]
  We present Dripper, an efficient HTML main content extraction framework powered by lightweight language models.<n>We introduce a controlled decoding mechanism that strictly constrains the output space through logits processors.<n>Dripper achieves state-of-the-art performance across all evaluation benchmarks and outperforms all baseline methods.
  arXiv  Detail & Related papers  (2025-11-28T12:04:46Z)
• AICC: Parse HTML Finer, Make Models Better -- A 7.3T AI-Ready Corpus Built by a Model-Based HTML Parser [54.623900859999424]
  We introduce MinerU-HTML, a novel extraction pipeline that reformulates content extraction as a sequence labeling problem.<n>On MainWebBench, our benchmark of 7,887 annotated web pages, MinerU-HTML 81.8% ROUGE-N F1 compared to Trafilatura's 63.6%, with exceptional structured element preservation.
  arXiv  Detail & Related papers  (2025-11-20T14:15:23Z)
• Development of an Automated Web Application for Efficient Web Scraping: Design and Implementation [0.0]
  This paper presents the design and implementation of a user-friendly, automated web application that simplifies and optimize the web scraping process for non-technical users.<n>The application breaks down the complex task of web scraping into three main stages: fetching, extraction, and execution.<n>This automated tool not only enhances the efficiency of web scraping but also democratizes access to data extraction by empowering users of all technical levels to gather and manage data tailored to their needs.
  arXiv  Detail & Related papers  (2025-10-22T04:56:00Z)
• SCRIBES: Web-Scale Script-Based Semi-Structured Data Extraction with Reinforcement Learning [48.376164461507244]
  We introduce SCRIBES (SCRIpt-Based Semi-Structured Content Extraction at Web-Scale), a novel reinforcement learning framework.<n>Instead of processing each page individually, SCRIBES generates reusable extraction scripts that can be applied to groups of structurally similar webpages.<n> Experiments show that our approach outperforms strong baselines by over 13% in script quality and boosts downstream question answering accuracy by more than 4% for GPT-4o.
  arXiv  Detail & Related papers  (2025-10-02T09:27:15Z)
• Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction [83.0216122783429]
  Web Reconstruction (WebR) is a fully automated framework for synthesizing high-quality instruction-tuning (IT) data directly from raw web documents.<n>We show that datasets generated by WebR outperform state-of-the-art baselines by up to 16.65% across four instruction-following benchmarks.
  arXiv  Detail & Related papers  (2025-04-22T04:07:13Z)
• AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation [54.17246674188208]
  Web scraping is a powerful technique that extracts data from websites, enabling automated data collection, enhancing data analysis capabilities, and minimizing manual data entry efforts. Existing methods, wrappers-based methods suffer from limited adaptability and scalability when faced with a new website. We introduce the paradigm of generating web scrapers with large language models (LLMs) and propose AutoScraper, a two-stage framework that can handle diverse and changing web environments more efficiently.
  arXiv  Detail & Related papers  (2024-04-19T09:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.