Machine learning (ML) models are typically considered black boxes. However, over the last decade, explainable AI tools have emerged to peer inside this so-called black box and extract the knowledge learned by the ML model. Interpretable AI approaches have also gained traction for scientific applications, as they are specifically designed to be understood by humans. Researchers at NSF NCAR are utilizing both explainable and interpretable ML approaches across a variety of applications to better understand our Earth System and the models we use to represent it.

XAI Proxies for Scientific Reliability

Researchers at NSF NCAR have identified measurable indicators (“proxies”) to discover physically meaningful and reliable explanations from XAI, establishing a new diagnostic for reliable scientific applications of XAI. Learn more here. 

ai-xai-1.png

Figure: Integrated Gradients heatmaps for different models applied to a representative test sample. The noise-free/data-rich transformer (top left) aligns closely with the ground truth (top right), while data-poor models (bottom row) diverge significantly. Note: colorbar ranges differ between panels. Figure from Mamalakis et al. 2025. 

Transfer Learning + XAI for Bias Detection

NSF NCAR scientists demonstrate a potential method to combine transfer learning and XAI to identify biases in traditional Earth System models that vary across different conditions. Learn more here.

ai-xai-2.png

Figure: Schematic of the method used to identify tropical state-dependent biases relevant for North Pacific subseasonal predictability, in particular, the application of XAI to identify the biases in tropical sources of predictability. Figure from Mayer et al. 2025.

Interpretable ML for Subseasonal Predictability

NSF NCAR scientists, in collaboration with a SOARS student, constructed interpretable ML models to identify sources of subseasonal predictability and offer new insights into favorable forecasting windows for Madden-Julian Oscillation teleconnections. Learn more here.

ai-xai-3.jpg

Figure: Schematic of the interpretable neural network architecture. Input into the (a) ENSO-network, (b) MJO-network. The predictions from each network are linearly combined (gray shaded box) to make the final network prediction. Figure from Mayer et al. 2024.

For more information or for partnership opportunities, please reach out to Kirsten Mayer.

Interactive AI for Winter Precipitation Type

NSF NCAR scientists developed an interactive website to visualize winter precipitation type predictions and enable users to understand prediction sensitivities by altering vertical profiles of temperature and dewpoint to see how it affects the ML model predictions.

ai-xai-4.png

Image from precipitationtype.com and algorithm in Becker et al. 2025.