Public Abstract

DE-SC0025705: Linear and Non-linear Dimensionality Reduction Methods for Unstructured PDE Data Compression

Award Status: Active

Institution: The Regents of the University of Colorado d/b/a University of Colorado, Boulder, CO
UEI: SPVKK1RC2MZ3
DUNS: 007431505

Most Recent Award Date: 01/14/2025
Number of Support Periods: 1
PM: Spotz, William

Current Budget Period: 12/01/2024 - 11/30/2025
Current Project Period: 12/01/2024 - 11/30/2027
PI: Doostan, Alireza

Supplement Budget Period: N/A

Public Abstract

The utilization of massive parallelization on high performance systems, such as those operated by the DOE, has significantly enhanced simulations of complex physics problems modeled by partial differential equations (PDEs). This, in turn, has resulted in a drastic increase in the production rate of high-fidelity data that are key to creating knowledge and design of such systems. However, this growth of computational power has considerably outpaced the development of input/output (I/O) storage and data analytics resources. This disparity has created a bottleneck where the time allocated to writing/processing data compared to data generation has steadily increased. To address this challenge, scientific data reduction has emerged as a promising direction and motivated the development of increasingly more scalable, accurate, and versatile strategies. State-of-the-art data reduction algorithms, however, often achieve only modest compression factors, are constrained to data on uniform grids and Cartesian domains, primarily ignore temporal correlation, or may require multiple passes over the data, hindering their ability to handle streaming data effectively. The main goal of this project is to overcome these challenges and limitations through novel computational strategies and software. This project will focus on the development and massively parallel implementation of novel, linear and non-linear dimensionality reduction techniques for spatial and spatial-temporal data compression. A particular focus will be placed on error-controlled, in situ compression of high-dimensional PDE data over non-uniform (unstructured) grids, possibly defined over complex geometries. Such tools are necessary in practical scenarios where raw data streams are of sizes beyond current I/O bandwidth. The research tasks of this project are founded on two key hypotheses: (i) Data from many complex PDE systems can be accurately described by representations that are of much lower dimensions than their explicit dimensions, and (ii) these lower-dimensional representations can be recovered in situ and with a small overhead on the data generation process itself. Key to these constructions are the development of reduced basis and deep neural models that are specialized to data reduction and are applicable to a broad class of solvers agnostic to the discretization scheme and physical system at hand. This research will yield new algorithms and software for in situ data reduction to all disciplines currently making use of computational simulation of PDEs. For applications of interest to DOE, ranging from weather/climate, astrophysics, detonation, propulsion, and energy, across all processes governed by PDEs, the proposed research will enable reduction, accurate reconstruction, and fast query of big simulation data. The new algorithms developed in this project are synergistic with various ongoing DOE programs on AI and machine learning. Teaching modules will be developed with the aim to improve the technical competencies of University of Colorado Boulder's engineering and applied math graduate students in high performance computing, big data, machine learning, and computational data sciences and engineering in general.

The utilization of massive parallelization on high performance systems, such as those operated by the DOE, has significantly enhanced simulations of complex physics problems modeled by partial differential equations (PDEs). This, in turn, has resulted in a drastic increase in the production rate of high-fidelity data that are key to creating knowledge and design of such systems. However, this growth of computational power has considerably outpaced the development of input/output (I/O) storage and data analytics resources. This disparity has created a bottleneck where the time allocated to writing/processing data compared to data generation has steadily increased. To address this challenge, scientific data reduction has emerged as a promising direction and motivated the development of increasingly more scalable, accurate, and versatile strategies. State-of-the-art data reduction algorithms, however, often achieve only modest compression factors, are constrained to data on uniform grids and Cartesian domains, primarily ignore temporal correlation, or may require multiple passes over the data, hindering their ability to handle streaming data effectively. The main goal of this project is to overcome these challenges and limitations through novel computational strategies and software.

This project will focus on the development and massively parallel implementation of novel, linear and non-linear dimensionality reduction techniques for spatial and spatial-temporal data compression. A particular focus will be placed on error-controlled, in situ compression of high-dimensional PDE data over non-uniform (unstructured) grids, possibly defined over complex geometries. Such tools are necessary in practical scenarios where raw data streams are of sizes beyond current I/O bandwidth. The research tasks of this project are founded on two key hypotheses: (i) Data from many complex PDE systems can be accurately described by representations that are of much lower dimensions than their explicit dimensions, and (ii) these lower-dimensional representations can be recovered in situ and with a small overhead on the data generation process itself. Key to these constructions are the development of reduced basis and deep neural models that are specialized to data reduction and are applicable to a broad class of solvers agnostic to the discretization scheme and physical system at hand.

This research will yield new algorithms and software for in situ data reduction to all disciplines currently making use of computational simulation of PDEs. For applications of interest to DOE, ranging from weather/climate, astrophysics, detonation, propulsion, and energy, across all processes governed by PDEs, the proposed research will enable reduction, accurate reconstruction, and fast query of big simulation data. The new algorithms developed in this project are synergistic with various ongoing DOE programs on AI and machine learning.

Teaching modules will be developed with the aim to improve the technical competencies of University of Colorado Boulder's engineering and applied math graduate students in high performance computing, big data, machine learning, and computational data sciences and engineering in general.