A Hands-on Tutorial in Data Generation, Processing, and Delivery for High Performance Computing and High Resolution Imaging
Dates: June 28-29 From 8:30am to 12noon
Location: KAUST Library Computer Classroom
Massive amounts of scientific data are an increasing challenge for scientists and engineers. Difficulties once limited to developing efficient output systems now involve the entire data lifespan, from creation and distribution through analysis and visualization. In this tutorial we will provide hands-on training in the use of a tool chain that improves each stage of data management while seamlessly integrating into existing infrastructures. The tool chain begins with an I/O library that scales on HPC platforms with over 700K cores (e.g. on MIRA at ANL), directly producing a multi-resolution format which enables a fast, flexible post-processing. This is complemented by a lightweight web-based server that enables data streaming for remote access as well as on-demand data conversion. The environment is completed with tools for data analysis and advanced visualization that can significantly shorten the time from simulation or experiment to scientific insight.