GeoReg: Weight-Constrained Few-Shot Regression for Socio-Economic Estimation using LLM

• URL: http://arxiv.org/abs/2507.13323v1
• Date: Thu, 17 Jul 2025 17:42:29 GMT
• Title: GeoReg: Weight-Constrained Few-Shot Regression for Socio-Economic Estimation using LLM
• Authors: Kyeongjin Ahn, Sungwon Han, Seungeon Lee, Donghyun Ahn, Hyoshin Kim, Jungwon Kim, Jihee Kim, Sangyoon Park, Meeyoung Cha,
• Abstract summary: This research introduces GeoReg a regression model that integrates diverse data sources to estimate socio-economic indicators.<n> Experiments across three countries at different stages of development demonstrate that our model outperforms baselines in estimating socio-economic indicators.
• Score: 12.740560201477798
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Socio-economic indicators like regional GDP, population, and education levels, are crucial to shaping policy decisions and fostering sustainable development. This research introduces GeoReg a regression model that integrates diverse data sources, including satellite imagery and web-based geospatial information, to estimate these indicators even for data-scarce regions such as developing countries. Our approach leverages the prior knowledge of large language model (LLM) to address the scarcity of labeled data, with the LLM functioning as a data engineer by extracting informative features to enable effective estimation in few-shot settings. Specifically, our model obtains contextual relationships between data features and the target indicator, categorizing their correlations as positive, negative, mixed, or irrelevant. These features are then fed into the linear estimator with tailored weight constraints for each category. To capture nonlinear patterns, the model also identifies meaningful feature interactions and integrates them, along with nonlinear transformations. Experiments across three countries at different stages of development demonstrate that our model outperforms baselines in estimating socio-economic indicators, even for low-income countries with limited data availability.

Related papers

• Exploring Scaling Laws for EHR Foundation Models [17.84205864956449]
  We present the first empirical investigation of scaling laws for EHR foundation models.<n>We identify consistent scaling patterns, including parabolic IsoFLOPs curves and power-law relationships between compute, model parameters, data size, and clinical utility.
  arXiv  Detail & Related papers  (2025-05-29T01:05:11Z)
• Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning [77.120955854093]
  We show that data diversity can be a strong predictor of generalization in language models.<n>We introduce G-Vendi, a metric that quantifies diversity via the entropy of model-induced gradients.<n>We present Prismatic Synthesis, a framework for generating diverse synthetic data.
  arXiv  Detail & Related papers  (2025-05-26T16:05:10Z)
• Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy [104.48511402784763]
  Performance Law for SR models aims to theoretically investigate and model the relationship between model performance and data quality.<n>We propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics.
  arXiv  Detail & Related papers  (2024-11-30T10:56:30Z)
• GeoSEE: Regional Socio-Economic Estimation With a Large Language Model [17.31652821477571]
  We present GeoSEE, a method that can estimate various socio-economic indicators using a unified pipeline powered by a large language model (LLM) The system then computes target indicators via in-context learning after aggregating results from selected modules in the format of natural language-based texts. Our method outperforms other predictive models in both unsupervised and low-shot contexts.
  arXiv  Detail & Related papers  (2024-06-14T07:50:22Z)
• TrajPRed: Trajectory Prediction with Region-based Relation Learning [11.714283460714073]
  We propose a region-based relation learning paradigm for predicting human trajectories in traffic scenes. Social interactions are modeled by relating the temporal changes of local joint information from a global perspective. We integrate multi-goal estimation and region-based relation learning to model the two stimuli, social interactions, and goals, in a prediction framework.
  arXiv  Detail & Related papers  (2024-04-10T12:31:43Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Learning Economic Indicators by Aggregating Multi-Level Geospatial Information [20.0397537179667]
  This research presents a deep learning model to predict economic indicators via aggregating traits observed from multiple levels of geographical units. Our new multi-level learning model substantially outperforms strong baselines in predicting key indicators such as population, purchasing power, and energy consumption. We discuss the multi-level model's implications for measuring inequality, which is the essential first step in policy and social science research on inequality and poverty.
  arXiv  Detail & Related papers  (2022-05-03T13:05:39Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Predicting Census Survey Response Rates With Parsimonious Additive Models and Structured Interactions [12.818275315985971]
  We consider the problem of predicting survey response rates using a family of flexible and interpretable nonparametric models.<n>The study is motivated by the US Census Bureau's well-known ROAM application.
  arXiv  Detail & Related papers  (2021-08-24T17:49:55Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.