Efficient GPU Offloading with OpenMP for a Hyperbolic Finite Volume Solver on Dynamically Adaptive Meshes

Wille, M; Weinzierl, T; Brito Gadeschi, G; Bader, M

Abstract

We identify and show how to overcome an OpenMP bottleneck in the administration of GPU memory. It arises for a wave equation solver on dynamically adaptive block-structured Cartesian meshes, which keeps all CPU threads busy and allows all of them to offload sets of patches to the GPU. Our studies show that multithreaded, concurrent, non-deterministic access to the GPU leads to performance breakdowns, since the GPU memory bookkeeping as offered through OpenMP’s map clause, i.e., the allocation and freeing, becomes another runtime challenge besides expensive data transfer and actual computation. We, therefore, propose to retain the memory management responsibility on the host: A caching mechanism acquires memory on the accelerator for all CPU threads, keeps hold of this memory and hands it out to the offloading threads upon demand. We show that this user-managed, CPU-based memory administration helps us to overcome the GPU memory bookkeeping bottleneck and speeds up the time-to-solution of Finite Volume kernels by more than an order of magnitude.

Citation

Wille, M., Weinzierl, T., Brito Gadeschi, G., & Bader, M. (2023, December). Efficient GPU Offloading with OpenMP for a Hyperbolic Finite Volume Solver on Dynamically Adaptive Meshes. Presented at ISC High Performance 2023, Hamburg

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: ISC High Performance 2023
• Acceptance Date: Feb 25, 2023
• Online Publication Date: May 10, 2023
• Publication Date: 2023
• Deposit Date: Mar 8, 2023
• Publicly Available Date: Jun 6, 2023
• Print ISSN: 0302-9743
• Publisher: Springer Verlag
• Volume: 13948
• Pages: 65-85
• Series Title: Lecture Notes in Computer Science
• Series ISSN: 0302-9743
• Book Title: High Performance Computing. ISC High Performance 2023.
• ISBN: 9783031320408
• DOI: https://doi.org/10.1007/978-3-031-32041-5_4
• Public URL: https://durham-repository.worktribe.com/output/1134318