Denoising the US Census: Succinct Block Hierarchical Regression

• URL: http://arxiv.org/abs/2603.10099v1
• Date: Tue, 10 Mar 2026 17:39:25 GMT
• Title: Denoising the US Census: Succinct Block Hierarchical Regression
• Abstract summary: We introduce BlueDown, a new post-processing method that produces more accurate, consistent estimates.<n>We obtain especially large accuracy improvements for aggregates at the county and tract levels on evaluation metrics proposed by the US Census Bureau.
• Score: 48.50026671993762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The US Census Bureau Disclosure Avoidance System (DAS) balances confidentiality and utility requirements for the decennial US Census (Abowd et al., 2022). The DAS was used in the 2020 Census to produce demographic datasets critically used for legislative apportionment and redistricting, federal and state funding allocation, municipal and infrastructure planning, and scientific research. At the heart of DAS is TopDown, a heuristic post-processing method that combines billions of private noisy measurements across six geographic levels in order to produce new estimates that are consistent, more accurate, and satisfy certain structural constraints on the data. In this work, we introduce BlueDown, a new post-processing method that produces more accurate, consistent estimates while satisfying the same privacy guarantees and structural constraints. We obtain especially large accuracy improvements for aggregates at the county and tract levels on evaluation metrics proposed by the US Census Bureau. From a technical perspective, we develop a new algorithm for generalized least-squares regression that leverages the hierarchical structure of the measurements and that is statistically optimal among linear unbiased estimators. This reduces the computational dependence on the number of geographic regions measured from matrix multiplication time, which would be infeasible for census-scale data, to linear time. We incorporate the additional structural constraints by combining this regression algorithm with an optimization routine that extends TDA to support correlated measurements. We further improve the efficiency of our algorithm using succinct linear-algebraic operations that exploit symmetries in the structure of the measurements and constraints. We believe our hierarchical regression and succinct operations to be of independent interest.

Related papers

• A Sieve-Accelerated Quadrature Method for Exact Privacy Accounting in the 2020 U.S. Decennial Census [5.387472623136637]
  In 2020, the U.S. Census Bureau adopted differential privacy for the Decennial Census by injecting integer-valued Gaussian noise into published census tabulations.<n>Exactly evaluating the privacy guarantees of these data releases would enable the Bureau to determine the absolute minimum noise required to satisfy a given privacy budget.<n>We introduce a computationally efficient and mathematically rigorous quadrature method to evaluate the exact privacy profile of practical, large-scale census releases.
  arXiv  Detail & Related papers  (2026-06-29T06:19:51Z)
• Aligning Data-Driven Predictors with Allocation: A Decision-Focused Approach to Survival Analysis [52.10284705210197]
  We show that machine learning predictors can yield arbitrarily poor outcomes when used for allocation.<n>We introduce a decision-focused learning approach based on optimizing normalized discounted cumulative gain (NDCG)<n>We propose a bootstrapping approach to optimize the NDCG of existing survival models.
  arXiv  Detail & Related papers  (2026-06-01T12:03:22Z)
• Best Linear Unbiased Estimate from Privatized Contingency Tables [6.17477133700348]
  In differential privacy (DP) mechanisms, some quantities can be estimated in multiple ways by combining different privatized values.<n>The DP 2020 Decennial Census products published by the U.S. Census Bureau consist of such redundant noisy counts.<n>We show that the minimum variance processing is a linear projection.
  arXiv  Detail & Related papers  (2024-09-06T16:27:34Z)
• Least Squares Estimation For Hierarchical Data [0.7366405857677227]
  The U.S. Census Bureau's 2020 Disclosure Avoidance System (DAS) bases its output on noisy measurements.<n>This paper describes an algorithm that can leverage a hierarchical structure of the input data in order to compute very high dimensional least squares estimates.
  arXiv  Detail & Related papers  (2024-04-19T20:18:16Z)
• Large-Scale OD Matrix Estimation with A Deep Learning Method [70.78575952309023]
  The proposed method integrates deep learning and numerical optimization algorithms to infer matrix structure and guide numerical optimization. We conducted tests to demonstrate the good generalization performance of our method on a large-scale synthetic dataset.
  arXiv  Detail & Related papers  (2023-10-09T14:30:06Z)
• Federated Offline Reinforcement Learning [55.326673977320574]
  We propose a multi-site Markov decision process model that allows for both homogeneous and heterogeneous effects across sites. We design the first federated policy optimization algorithm for offline RL with sample complexity. We give a theoretical guarantee for the proposed algorithm, where the suboptimality for the learned policies is comparable to the rate as if data is not distributed.
  arXiv  Detail & Related papers  (2022-06-11T18:03:26Z)
• Spatially Adaptive Online Prediction of Piecewise Regular Functions [12.18340575383456]
  We consider the problem of estimating piecewise regular functions in an online setting. We propose a modified version of a recently developed online learning algorithm called the sleeping experts aggregation algorithm.
  arXiv  Detail & Related papers  (2022-03-30T18:15:56Z)
• Online Forgetting Process for Linear Regression Models [18.336825781223034]
  Motivated by the EU's "Right To Be Forgotten" regulation, we initiate a study of statistical data deletion problems. We propose a deletion-aware algorithm textttFIFD-OLS for the low dimensional case, and witness a catastrophic rank swinging phenomenon. As a remedy, we propose the textttFIFD-Adaptive Ridge algorithm with a novel online regularization scheme.
  arXiv  Detail & Related papers  (2020-12-03T02:57:53Z)
• Privacy Preserving Recalibration under Domain Shift [119.21243107946555]
  We introduce a framework that abstracts out the properties of recalibration problems under differential privacy constraints. We also design a novel recalibration algorithm, accuracy temperature scaling, that outperforms prior work on private datasets.
  arXiv  Detail & Related papers  (2020-08-21T18:43:37Z)
• Magnify Your Population: Statistical Downscaling to Augment the Spatial Resolution of Socioeconomic Census Data [48.7576911714538]
  We present a new statistical downscaling approach to derive fine-scale estimates of key socioeconomic attributes. For each selected socioeconomic variable, a Random Forest model is trained on the source Census units and then used to generate fine-scale gridded predictions. As a case study, we apply this method to Census data in the United States, downscaling the selected socioeconomic variables available at the block group level, to a grid of 300 spatial resolution.
  arXiv  Detail & Related papers  (2020-06-23T16:52:18Z)
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data [59.50904660420082]
  Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model.
  arXiv  Detail & Related papers  (2020-05-22T23:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.