Hidden Errors in Big Data: The Case of Property Records

• URL: http://arxiv.org/abs/2607.28827v1
• Date: Thu, 30 Jul 2026 20:36:06 GMT
• Title: Hidden Errors in Big Data: The Case of Property Records
• Abstract summary: We audit two prominent brokered property datasets in Cook County, IL.<n>For 1-2% of matched sales, broker-provided sale prices differ from ground truth sale prices by more than 5%.<n>We show that misreporting is highly consistent between brokers.
• Score: 7.339927315483204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Big data are the foundation for an increasing share of academic research and AI models deployed in both the public and private sectors, prompting substantial growth over time in reliance on brokered datasets. Brokered property records, which are ubiquitous in studies of gentrification, inequality, and the property tax in the U.S. and serve as inputs to property valuation models, are one notable example. In this paper, we audit two prominent brokered property datasets, finding errors in these data which bias key measures of economic inequality. First, we document that for 1-2% of matched sales in Cook County, IL, from 2018-2021, broker-provided sale prices differ from ground truth sale prices by more than 5%. Moreover, missing data and conceptual differences in the reporting of deed and property characteristics lead to coverage errors ranging from 12 to 15% of transactions. Second, we show that misreporting is highly consistent between brokers: more often than not, brokers make identical reporting errors for the same transactions. Third, to illustrate the significance of these errors, we measure their impact on estimates of property tax regressivity, finding that they drive significant wedges between estimates depending on the data source. These findings generalize to two other large counties in the U.S., and highlight the crucial importance of open administrative data and transparency from brokers regarding data provenance and lineage.

Related papers

• Tradeoffs are Domain Dependent: Improving Accuracy and Fairness in Property Tax Assessments [7.885461849343874]
  Algorithmic fairness research often assumes a tradeoff between fairness and accuracy. Yet this tradeoff may not be universal.<n>We test this assumption in the context of U.S. property tax assessment.
  arXiv  Detail & Related papers  (2026-05-14T16:20:32Z)
• Estimating Misreporting in the Presence of Genuine Modification: A Causal Perspective [11.438018798893113]
  In settings where ML models are used to inform the allocation of resources, agents might have an incentive to strategically change their features to secure better outcomes.<n>We propose a causally-motivated approach to identify and quantify how much an agent misreports on average by distinguishing deceptive changes in their features from genuine modification.
  arXiv  Detail & Related papers  (2025-05-29T19:06:30Z)
• Assessing Judging Bias in Large Reasoning Models: An Empirical Study [99.86300466350013]
  Large Reasoning Models (LRMs) like DeepSeek-R1 and OpenAI-o1 have demonstrated remarkable reasoning capabilities.<n>We present a benchmark comparing judging biases between LLMs and LRMs across both subjective preference-alignment datasets and objective fact-based datasets.
  arXiv  Detail & Related papers  (2025-04-14T07:14:27Z)
• Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
  We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods.<n>We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results.<n>We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
  arXiv  Detail & Related papers  (2024-07-19T14:53:18Z)
• The Impact of Differential Feature Under-reporting on Algorithmic Fairness [86.275300739926]
  We present an analytically tractable model of differential feature under-reporting. We then use to characterize the impact of this kind of data bias on algorithmic fairness. Our results show that, in real world data settings, under-reporting typically leads to increasing disparities.
  arXiv  Detail & Related papers  (2024-01-16T19:16:22Z)
• A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing [1.850972250657274]
  Decision-makers often observe the occurrence of events through a reporting process. We show how to overcome this challenge by leveraging the fact that events are spatially correlated.
  arXiv  Detail & Related papers  (2023-12-18T23:40:56Z)
• Quantifying Spatial Under-reporting Disparities in Resident Crowdsourcing [5.701305404173138]
  We develop a method to identify reporting delays without using external ground-truth data. We apply our method to over 100,000 resident reports made in New York City and to over 900,000 reports made in Chicago.
  arXiv  Detail & Related papers  (2022-04-19T02:54:16Z)
• Competency Problems: On Finding and Removing Artifacts in Language Data [50.09608320112584]
  We argue that for complex language understanding tasks, all simple feature correlations are spurious. We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account.
  arXiv  Detail & Related papers  (2021-04-17T21:34:10Z)
• Domain Adaptative Causality Encoder [52.779274858332656]
  We leverage the characteristics of dependency trees and adversarial learning to address the tasks of adaptive causality identification and localisation. We present a new causality dataset, namely MedCaus, which integrates all types of causality in the text.
  arXiv  Detail & Related papers  (2020-11-27T04:14:55Z)
• On Disentangled Representations Learned From Correlated Data [59.41587388303554]
  We bridge the gap to real-world scenarios by analyzing the behavior of the most prominent disentanglement approaches on correlated data. We show that systematically induced correlations in the dataset are being learned and reflected in the latent representations. We also demonstrate how to resolve these latent correlations, either using weak supervision during training or by post-hoc correcting a pre-trained model with a small number of labels.
  arXiv  Detail & Related papers  (2020-06-14T12:47:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.