Collaborative Research: OAC Core: Topology-Aware Data Compression for Scientific Analysis and Visualization

Today's large-scale simulations are producing vast amounts of data that are revolutionizing scientific thinking and practices. For instance, a fusion simulation can produce 200 petabytes of data in a single run, while a climate simulation can generate 260 terabytes of data every 16 seconds with a 1 square kilometer resolution. As the disparity between data generation rates and available I/O bandwidths continues to grow, data storage and movement are becoming significant bottlenecks for extreme-scale scientific simulations in terms of in situ and post hoc analysis and visualization. The disparity necessitates data compression, which compresses large-scale simulations data in situ, and decompresses data in situ and/or post hoc for analysis and exploration. On the other hand, a critical step in extracting insight from large-scale simulations involves the definition, extraction, and evaluation of features of interest. Topological data analysis has provided powerful tools to capture features from scientific data in turbulent combustion, astronomy, climate science, computational physics and chemistry, and ecology. While lossy compression is leveraged to address the big data challenges, most existing lossy compressors are agnostic of and thus fail to preserve topological features that are essential to scientific discoveries. This project aims to research and develop advanced lossy compression techniques and software that preserve topological features in data for in situ and post hoc analysis and visualization at extreme scales. The success of this project will promote scientific research on driving applications in cosmology, climate, and fusion by enabling efficient and effective compression for scientific data, and the impact scales to other science and engineering disciplines. Furthermore, the research products of this project will be integrated into visualization and parallel processing curricula, disseminated via research and training workshops, and used to attract underrepresented students for broadening participation in computing. This project tackles the data compression, analysis, and visualization needs in extreme-scale scientific simulations by developing a suite of topology-aware data compression algorithms for scalar field and vector field data. Such algorithms effectively reduce the size of data while preserving critical features defined by topological notions. This project will define and enforce topology-aware constraints over advanced lossy compression algorithms. Such capabilities have not been studied systematically within today’s data compression paradigm. This project will impact specific fields, including computational science, data analysis, data compression, and visualization, and the broader scientific community. The research products of this project will be delivered as publicly available software to significantly advance the research cyberinfrastructure for current and upcoming exascale systems. This project will foster novel discoveries in multiple scientific disciplines beyond cosmology, climate, and fusion by enabling efficient and effective compression on a wide range of platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当今的大规模模拟正在产生大量的数据，这些数据正在彻底改变科学思维和实践。例如，融合模拟可以在一次运行中产生200 pb的数据，而气候模拟可以每16秒以1平方公里的分辨率生成每16秒的260台数据。随着数据生成速率和可用I/O带宽之间的差异继续增长，就原位和事后分析和可视化而言，数据存储和移动正在成为极端尺度科学模拟的重要瓶颈。差异必要的数据压缩，该数据压缩了原位压缩大规模仿真数据，并在原位和/或事后解压缩数据以进行分析和探索。另一方面，从大规模模拟中提取洞察力的关键步骤涉及对感兴趣的特征的定义，提取和评估。拓扑数据分析提供了强大的工具，可以从湍流压缩，天文学，气候科学，计算物理和化学以及生态学中捕获科学数据的特征。尽管损失压缩是为了应对大数据挑战，但大多数现有的损失压缩机都是不可知论的，因此无法保留对科学发现必不可少的拓扑特征。该项目旨在研究和开发先进的损失压缩技术和软件，这些技术和软件保留了原位数据中的拓扑特征，以及在极端尺度上的事后分析和可视化。该项目的成功将通过为科学数据实现高效有效的压缩以及对其他科学和工程学科的影响量表来促进驱动宇宙学，气候和融合应用的科学研究。此外，该项目的研究产品将被整合到可视化和并行处理课程中，通过研究和培训研讨会进行传播，并用于吸引占卜不足的学生以扩大计算的参与。该项目将定义和强制对高级损耗压缩算法的拓扑感知限制。此类功能尚未在当今的数据压缩范式中系统地研究。该项目将影响特定领域，包括计算科学，数据分析，数据压缩和可视化以及更广泛的科学界。该项目的研究产品将作为公开软件提供，以大大推动当前和即将到来的Exascale系统的网络基础架构。该项目将通过在广泛的平台上实现有效的压缩，在宇宙学，气候和融合以外的多个科学学科中培养新发现。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准通过评估来评估的。