Should we trust web-scraped data?

• URL: http://arxiv.org/abs/2308.02231v1
• Date: Fri, 4 Aug 2023 10:07:31 GMT
• Title: Should we trust web-scraped data?
• Abstract summary: Web scraping refers to the use of automated computer programs to access websites and download their content. This article describes three sources of sampling bias in web-scraped data.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing adoption of econometric and machine-learning approaches by empirical researchers has led to a widespread use of one data collection method: web scraping. Web scraping refers to the use of automated computer programs to access websites and download their content. The key argument of this paper is that na\"ive web scraping procedures can lead to sampling bias in the collected data. This article describes three sources of sampling bias in web-scraped data. More specifically, sampling bias emerges from web content being volatile (i.e., being subject to change), personalized (i.e., presented in response to request characteristics), and unindexed (i.e., abundance of a population register). In a series of examples, I illustrate the prevalence and magnitude of sampling bias. To support researchers and reviewers, this paper provides recommendations on anticipating, detecting, and overcoming sampling bias in web-scraped data.

Related papers

• BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers [13.838654196169612]
  We introduce BiasEdit, a framework that automatically detects bias attributes from the original dataset and edits them to construct a debiased dataset.<n>Our method operates without manual annotations and can leverage off-the-shelf vision-induced and editing models.
  arXiv  Detail & Related papers  (2026-05-27T13:17:58Z)
• Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models [107.24906866038431]
  We propose REWIRE, REcycling the Web with guIded REwrite, to enrich low-quality documents so that they could become useful for training.<n>We show that mixing high-quality raw texts and our rewritten texts lead to 1.0, 1.3 and 2.5 percentage points improvement respectively across 22 diverse tasks.
  arXiv  Detail & Related papers  (2025-06-05T07:12:12Z)
• Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence [56.09494651178128]
  Retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG) We show that retrievers often rely on superficial patterns like over-prioritizing document beginnings, shorter documents, repeated entities, and literal matches. We show that these biases have direct consequences for downstream applications like RAG, where retrieval-preferred documents can mislead LLMs.
  arXiv  Detail & Related papers  (2025-03-06T23:23:13Z)
• Multi-Record Web Page Information Extraction From News Websites [83.88591755871734]
  In this paper, we focus on the problem of extracting information from web pages containing many records. To address this gap, we created a large-scale, open-access dataset specifically designed for list pages. Our dataset contains 13,120 web pages with news lists, significantly exceeding existing datasets in both scale and complexity.
  arXiv  Detail & Related papers  (2025-02-20T15:05:00Z)
• Beyond time delays: How web scraping distorts measures of online news consumption [0.4706932040794696]
  We investigate discrepancies between content obtained directly during user interaction with a website (in-situ) and content scraped using the URLs of participants' logged visits (ex-situ) with various time delays. We find substantial disparities between the methodologies, uncovering that errors are not uniformly distributed across news categories. Our research emphasizes the need for data collection methods that capture web content directly in the user's environment.
  arXiv  Detail & Related papers  (2024-11-30T13:36:48Z)
• How Unique is Whose Web Browser? The role of demographics in browser fingerprinting among US users [50.699390248359265]
  Browser fingerprinting can be used to identify and track users across the Web, even without cookies. This technique and resulting privacy risks have been studied for over a decade. We provide a first-of-its-kind dataset to enable further research.
  arXiv  Detail & Related papers  (2024-10-09T14:51:58Z)
• BiasScanner: Automatic Detection and Classification of News Bias to Strengthen Democracy [4.248837664338829]
  BiasScanner aims to strengthen democracy by supporting news consumers with scrutinizing news articles they are reading online. It contains a server-side pre-trained large language model to identify biased sentences of news articles and a front-end Web browser plug-in.
  arXiv  Detail & Related papers  (2024-07-15T15:42:22Z)
• AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation [54.17246674188208]
  Web scraping is a powerful technique that extracts data from websites, enabling automated data collection, enhancing data analysis capabilities, and minimizing manual data entry efforts. Existing methods, wrappers-based methods suffer from limited adaptability and scalability when faced with a new website. We introduce the paradigm of generating web scrapers with large language models (LLMs) and propose AutoScraper, a two-stage framework that can handle diverse and changing web environments more efficiently.
  arXiv  Detail & Related papers  (2024-04-19T09:59:44Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)
• Cleaner Pretraining Corpus Curation with Neural Web Scraping [39.97459187762505]
  This paper presents a simple, fast, and effective Neural web Scraper (NeuScraper) to help extract primary and clean text contents from webpages. Experimental results show that NeuScraper surpasses the baseline scrapers by achieving more than a 20% improvement.
  arXiv  Detail & Related papers  (2024-02-22T16:04:03Z)
• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• Unsupervised Bias Detection in College Student Newspapers [0.0]
  This paper introduces a framework for scraping complex archive sites and generates a dataset of 14 student papers with 23,154 entries. The advantages of this approach are that it is less comparative than reconstruction bias and requires less labelled data than generating keyword sentiment. Results are calculated on politically charged words as well as control words to show how conclusions can be drawn.
  arXiv  Detail & Related papers  (2023-09-11T06:51:09Z)
• Harnessing the Power of Text-image Contrastive Models for Automatic Detection of Online Misinformation [50.46219766161111]
  We develop a self-learning model to explore the constrastive learning in the domain of misinformation identification. Our model shows the superior performance of non-matched image-text pair detection when the training data is insufficient.
  arXiv  Detail & Related papers  (2023-04-19T02:53:59Z)
• Representation Bias in Data: A Survey on Identification and Resolution Techniques [26.142021257838564]
  Data-driven algorithms are only as good as the data they work with, while data sets, especially social data, often fail to represent minorities adequately. Representation Bias in data can happen due to various reasons ranging from historical discrimination to selection and sampling biases in the data acquisition and preparation methods. This paper reviews the literature on identifying and resolving representation bias as a feature of a data set, independent of how consumed later.
  arXiv  Detail & Related papers  (2022-03-22T16:30:22Z)
• Correcting Exposure Bias for Link Recommendation [31.799185352323807]
  Exposure bias can arise when users are systematically underexposed to certain relevant items. We propose estimators that leverage known exposure probabilities to mitigate this bias. Our methods lead to greater diversity in the recommended papers' fields of study.
  arXiv  Detail & Related papers  (2021-06-13T16:51:41Z)
• Hidden Biases in Unreliable News Detection Datasets [60.71991809782698]
  We show that selection bias during data collection leads to undesired artifacts in the datasets. We observed a significant drop (>10%) in accuracy for all models tested in a clean split with no train/test source overlap. We suggest future dataset creation include a simple model as a difficulty/bias probe and future model development use a clean non-overlapping site and date split.
  arXiv  Detail & Related papers  (2021-04-20T17:16:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.