Mechanistic Interpretability Tool for AI Weather Models

• URL: http://arxiv.org/abs/2604.20467v1
• Date: Wed, 22 Apr 2026 11:54:09 GMT
• Title: Mechanistic Interpretability Tool for AI Weather Models
• Abstract summary: We present an open-source, highly adaptable tool which incorporates concepts from mechanistic interpretability.<n>The tool organises internal latent representations from the model processor and allows for initial analyses.<n>Applying our tool to the graph neural network GraphCast, we present preliminary case studies for mid-latitude synoptic-scale waves and specific humidity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial Intelligence (AI) weather models are improving rapidly, and their forecasts are already competitive with long-established traditional Numerical Weather Prediction (NWP). To build confidence in this new methodology, it is critical that we understand how these predictions are generated. This is a huge challenge as these AI weather models remain largely black boxes. In other areas of Machine Learning (ML), mechanistic interpretability has emerged as a framework for understanding ML predictions by analysing the building blocks responsible for them. Here we present an open-source, highly adaptable tool which incorporates concepts from mechanistic interpretability. The tool organises internal latent representations from the model processor and allows for initial analyses, including cosine similarity and Principal Component Analysis (PCA), enabling the user to identify directions in latent space potentially associated with meteorological features. Applying our tool to the graph neural network GraphCast, we present preliminary case studies for mid-latitude synoptic-scale waves and specific humidity. These demonstrate the tool's ability to identify linear combinations of latent channels that appear to correspond to interpretable features.

Related papers

• Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting [63.8116386935854]
  We demonstrate that state-of-the-art probabilistic skill requires neither intricate architectural constraints nor specialized trainings.<n>We introduce a scalable framework for learning multi-scale atmospheric dynamics by combining a directly downsampled latent space with a history-conditioned local projector.<n>We find that our framework design is robust to the choice of probabilistic estimators, seamlessly supporting interpolants, diffusion models, and CRPS-based ensemble training.
  arXiv  Detail & Related papers  (2026-01-26T03:52:16Z)
• Towards mechanistic understanding in a data-driven weather model: internal activations reveal interpretable physical features [0.0]
  We adapt tools from interpretability research in Large Language Models to analyze intermediate computational layers in GraphCast.<n>We uncover distinct features on a wide range of length and time scales that correspond to tropical cyclones, atmospheric rivers, diurnal and seasonal behavior, large-scale precipitation patterns, specific geographical coding, and sea-ice extent.
  arXiv  Detail & Related papers  (2025-12-30T19:50:30Z)
• Exploring Multimodal AI Reasoning for Meteorological Forecasting from Skew-T Diagrams [4.036372578802888]
  Vision-Language Models (VLMs) have shown promise in other scientific domains, but their application to meteorological diagram interpretation remains largely unexplored.<n>We present a lightweight AI assistant that interprets Skew-T diagrams using a small language model (LM) and a small VLM fine-tuned to emulate human forecasters.
  arXiv  Detail & Related papers  (2025-08-17T01:36:31Z)
• XForecast: Evaluating Natural Language Explanations for Time Series Forecasting [72.57427992446698]
  Time series forecasting aids decision-making, especially for stakeholders who rely on accurate predictions. Traditional explainable AI (XAI) methods, which underline feature or temporal importance, often require expert knowledge. evaluating forecast NLEs is difficult due to the complex causal relationships in time series data.
  arXiv  Detail & Related papers  (2024-10-18T05:16:39Z)
• Tackling the Accuracy-Interpretability Trade-off in a Hierarchy of Machine Learning Models for the Prediction of Extreme Heatwaves [41.94295877935867]
  We perform probabilistic forecasts of extreme heatwaves over France using a hierarchy of increasingly complex Machine Learning models.<n>CNNs provide higher accuracy, but their black-box nature severely limits interpretability.<n>ScatNet achieves similar performance to CNNs while providing greater transparency.
  arXiv  Detail & Related papers  (2024-10-01T18:15:04Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which generalizes weather forecasts to finer-grained temporal scales beyond training dataset.<n>Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale.<n>We also introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• Interpretable Machine Learning for Weather and Climate Prediction: A Survey [24.028385794099435]
  We review current interpretable machine learning approaches applied to meteorological predictions. Design inherently interpretable models from scratch using architectures like tree ensembles and explainable neural networks. We discuss research challenges around achieving deeper mechanistic interpretations aligned with physical principles.
  arXiv  Detail & Related papers  (2024-03-24T14:23:35Z)
• A Practical Probabilistic Benchmark for AI Weather Models [0.7978324349017066]
  We show that two leading AI weather models, i.e. GraphCast and Pangu, are tied on the probabilistic CRPS metric. We also reveal how multiple time-step loss functions, which many data-driven weather models have employed, are counter-productive.
  arXiv  Detail & Related papers  (2024-01-27T05:53:16Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
  We develop an AI-based cyclic weather forecasting system, FengWu-4DVar. FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model. Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
  arXiv  Detail & Related papers  (2023-12-16T02:07:56Z)
• Hessian-based toolbox for reliable and interpretable machine learning in physics [58.720142291102135]
  We present a toolbox for interpretability and reliability, extrapolation of the model architecture. It provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an agnostic score for the model predictions. Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.
  arXiv  Detail & Related papers  (2021-08-04T16:32:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.