Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?

• URL: http://arxiv.org/abs/2303.04204v2
• Date: Thu, 22 Feb 2024 17:46:54 GMT
• Title: Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?
• Authors: Qingyi Wang, Shenhao Wang, Yunhan Zheng, Hongzhou Lin, Xiaohu Zhang, Jinhua Zhao, Joan Walker
• Abstract summary: This study creates a theoretical framework of deep hybrid models with a crossing structure consisting of a mixing operator and a behavioral predictor. It is applied to analyze travel mode choice using the MyDailyTravel Survey from Chicago as the numeric inputs and the satellite images as the imagery inputs. We found that deep hybrid models outperform both the traditional demand models and the recent deep learning in predicting the aggregate and disaggregate travel behavior.
• Score: 10.5335227537056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classical demand modeling analyzes travel behavior using only low-dimensional numeric data (i.e. sociodemographics and travel attributes) but not high-dimensional urban imagery. However, travel behavior depends on the factors represented by both numeric data and urban imagery, thus necessitating a synergetic framework to combine them. This study creates a theoretical framework of deep hybrid models with a crossing structure consisting of a mixing operator and a behavioral predictor, thus integrating the numeric and imagery data into a latent space. Empirically, this framework is applied to analyze travel mode choice using the MyDailyTravel Survey from Chicago as the numeric inputs and the satellite images as the imagery inputs. We found that deep hybrid models outperform both the traditional demand models and the recent deep learning in predicting the aggregate and disaggregate travel behavior with our supervision-as-mixing design. The latent space in deep hybrid models can be interpreted, because it reveals meaningful spatial and social patterns. The deep hybrid models can also generate new urban images that do not exist in reality and interpret them with economic theory, such as computing substitution patterns and social welfare changes. Overall, the deep hybrid models demonstrate the complementarity between the low-dimensional numeric and high-dimensional imagery data and between the traditional demand modeling and recent deep learning. It generalizes the latent classes and variables in classical hybrid demand models to a latent space, and leverages the computational power of deep learning for imagery while retaining the economic interpretability on the microeconomics foundation.

Related papers

• Learning Deep Hybrid Models with Sharpness-Aware Minimization [4.8941886361557625]
  We propose to focus on the flatness of loss minima in learning hybrid models, aiming to make the model as simple as possible.<n>We employ the idea of sharpness-aware minimization and adapt it to the hybrid modeling setting.<n> Numerical experiments show that the SAM-based method works well across different choices of models and datasets.
  arXiv  Detail & Related papers  (2026-02-06T16:27:19Z)
• Demographics-Informed Neural Network for Multi-Modal Spatiotemporal forecasting of Urban Growth and Travel Patterns Using Satellite Imagery [3.378738346115004]
  This study presents a novel demographics informed deep learning framework designed to forecast urban spatial transformations.<n>The proposed model employs an encoder-decoder architecture with temporal gated residual connections, integrating satellite imagery and demographic data.<n>The study also contributes a comprehensive multimodal dataset pairing satellite imagery sequences with corresponding demographic and travel behavior attributes.
  arXiv  Detail & Related papers  (2025-06-14T11:33:40Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships. Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Where Am I and What Will I See: An Auto-Regressive Model for Spatial Localization and View Prediction [60.964512894143475]
  We present Generative Spatial Transformer ( GST), a novel auto-regressive framework that jointly addresses spatial localization and view prediction. Our model simultaneously estimates the camera pose from a single image and predicts the view from a new camera pose, effectively bridging the gap between spatial awareness and visual prediction.
  arXiv  Detail & Related papers  (2024-10-24T17:58:05Z)
• Towards Model-Agnostic Dataset Condensation by Heterogeneous Models [13.170099297210372]
  We develop a novel method to produce universally applicable condensed images through cross-model interactions. By balancing the contribution of each model and maintaining their semantic meaning closely, our approach overcomes the limitations associated with model-specific condensed images.
  arXiv  Detail & Related papers  (2024-09-22T17:13:07Z)
• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective [60.64922606733441]
  We introduce a mathematical model that formalizes relational learning as hypergraph recovery to study pre-training of Foundation Models (FMs) In our framework, the world is represented as a hypergraph, with data abstracted as random samples from hyperedges. We theoretically examine the feasibility of a Pre-Trained Model (PTM) to recover this hypergraph and analyze the data efficiency in a minimax near-optimal style.
  arXiv  Detail & Related papers  (2024-06-17T06:20:39Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Transforming Norm-based To Graph-based Spatial Representation for Spatio-Temporal Epidemiological Models [6.396288020763144]
  Pemics pose significant threats to global health, mortality, economic stability, and political landscapes.<n>Models can be roughly divided into two main spatial categories: norm-based graph-based models.<n>In this study, we explore the ability to transform from norm-based to graph-based spatial representation.
  arXiv  Detail & Related papers  (2024-02-22T13:30:52Z)
• p$^3$VAE: a physics-integrated generative model. Application to the pixel-wise classification of airborne hyperspectral images [0.6849746341453253]
  We introduce p$3$VAE, a generative model that integrates a physical model. We apply p$3$VAE to the pixel-wise classification of airborne hyperspectral images.
  arXiv  Detail & Related papers  (2022-10-19T09:32:15Z)
• Statistical Deep Learning for Spatial and Spatio-Temporal Data [0.0]
  We present an overview of traditional statistical and machine learning perspectives for modeling spatial andtemporal data. We then focus on a variety of hybrid models that have recently been developed for latent process, data, and parameter specifications. These hybrid models integrate modeling ideas with deep neural network models in order to take advantage of the strengths of each modeling paradigm.
  arXiv  Detail & Related papers  (2022-06-05T16:49:10Z)
• Geometric and Topological Inference for Deep Representations of Complex Networks [13.173307471333619]
  We present a class of statistics that emphasize the topology as well as the geometry of representations. We evaluate these statistics in terms of the sensitivity and specificity that they afford when used for model selection. These new methods enable brain and computer scientists to visualize the dynamic representational transformations learned by brains and models.
  arXiv  Detail & Related papers  (2022-03-10T17:14:14Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Strong coupling between scales in a multi-scalar model of urban dynamics [0.0]
  We introduce in this paper a multi-scalar simulation model of urban growth, coupling a system of cities interaction model at the macroscopic scale with morphogenesis models for the evolution of urban form at the scale of metropolitan areas. Strong coupling between scales is achieved through an update of model parameters at each scale depending on trajectories at the other scale.
  arXiv  Detail & Related papers  (2021-01-29T18:37:44Z)
• Learning Opinion Dynamics From Social Traces [25.161493874783584]
  We propose an inference mechanism for fitting a generative, agent-like model of opinion dynamics to real-world social traces. We showcase our proposal by translating a classical agent-based model of opinion dynamics into its generative counterpart. We apply our model to real-world data from Reddit to explore the long-standing question about the impact of backfire effect.
  arXiv  Detail & Related papers  (2020-06-02T14:48:17Z)
• Hybrid modeling: Applications in real-time diagnosis [64.5040763067757]
  We outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models. We are using such models for real-time diagnosis applications.
  arXiv  Detail & Related papers  (2020-03-04T00:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.