Advancing Multimodal and Multidimensional AI to Discover Connections between Cloud Microphysics and Precipitation

Virginia Tech: Gabriel Isaacman-VanWertz (PI), Hoda Eldardiry, Hosein Foroutan, Craig Ramseyer, Stephanie Zick

Argonne National Laboratory: Jiwen Fan

Loyola University Maryland: Sibren Isaacman

Predicting exactly when, where, and how hard it will rain is critical for protecting national infrastructure from extreme weather and for estimating the availability of water for agriculture and energy production. However, precipitation modeling remains one of the most difficult challenges for models of Earth’s weather and environmental systems. Most models struggle to predict the timing and location of heavy storms, the size and number of raindrops, and the intensity of rainfall. This difficulty stems from the complexity of the interactions inside clouds that lead to rainfall, such as the sizes of droplets inside clouds and how they collide and interact with each other. The Atmospheric Radiation Measurement (ARM) facility operated by the U.S. Department of Energy (DOE) has gathered decades of highly detailed data on clouds, rain, and the processes that drive them, but the volume and complexity of these measurements exceeds the capability for human-driven analysis.

To convert the large observational datasets of the DOE into predictive power, this project introduces a unique approach by training an AI that is “multimodal,” that is, able to simultaneously analyze numerical data (like precise temperature and humidity readings) and visual representations of that data (like graphs and radar images). Just as a human scientist looks at a weather map or a graph to spot shapes and patterns, the AI will use computer vision to recognize large-scale patterns and at the same time use precise measurements to quantify the relationships within these patterns. In this project, data from some of ARM sites with the longest-term data records in the United States will be used to build multimodal AI models that can bridge fine-scale details with large-scale pattern recognition. Furthermore, the AI model’s discoveries will be compared against current DOE models to identify failure points in the models to facilitate future improvement in precipitation forecasting.

By coupling the rich, long-term dataset available from the DOE with this state-of-the-art AI-driven analytical approach, the team will extract new understanding of the processes inside clouds to better predict precipitation and pave the way for safer, better-prepared communities. The new methods and next-generation multimodal AI model will not only unlock new information about clouds and precipitation, but is expected to have broad applications for analysis of complex or large-scale data across science and engineering. In addition, by building a team across academic and national lab institutions that combines leaders in atmospheric science with experts in advanced AI techniques and machine learning, this research lays a foundation for future research efforts to tackle the most difficult and pressing issues in environmental and atmospheric science with the most cutting-edge analytic methods.