teaMPI---replication-based resiliency without the (performance) pain

Samfass, Philipp; Weinzierl, Tobias; Hazelwood, Benjamin; Bader, Michael; Sadayappan, Ponnuswamy; Chamberlain, Bradford L.; Juckeland, Guido; Ltaief, Hatem

Abstract

In an era where we can not afford to checkpoint frequently, replication is a generic way forward to construct numerical simulations that can continue to run even if hardware parts fail. Yet, replication often is not employed on larger scales, as naïvely mirroring a computation once effectively halves the machine size, and as keeping replicated simulations consistent with each other is not trivial. We demonstrate for the ExaHyPE engine—a task-based solver for hyperbolic equation systems—that it is possible to realise resiliency without major code changes on the user side, while we introduce a novel algorithmic idea where replication reduces the time-to-solution. The redundant CPU cycles are not burned “for nothing”. Our work employs a weakly consistent data model where replicas run independently yet inform each other through heartbeat messages whether they are still up and running. Our key performance idea is to let the tasks of the replicated simulations share some of their outcomes, while we shuffle the actual task execution order per replica. This way, replicated ranks can skip some local computations and automatically start to synchronise with each other. Our experiments with a production-level seismic wave-equation solver provide evidence that this novel concept has the potential to make replication affordable for large-scale simulations in high-performance computing.

Citation

Samfass, P., Weinzierl, T., Hazelwood, B., Bader, M., Sadayappan, P., Chamberlain, B. L., …Ltaief, H. (2020). teaMPI---replication-based resiliency without the (performance) pain. In High Performance Computing: 35th International Conference, ISC High Performance 2020, Frankfurt/Main, Germany, June 22–25, 2020 ; proceedings (455-473). https://doi.org/10.1007/978-3-030-50743-5_23

• Conference Name: ISC High Performance
• Conference Location: Frankfurt
• Acceptance Date: May 15, 2020
• Online Publication Date: Jun 15, 2020
• Publication Date: 2020
• Deposit Date: May 25, 2020
• Publicly Available Date: Jun 15, 2021
• Publisher: Springer Verlag
• Volume: 12151
• Pages: 455-473
• Series Title: Lecture Notes in Computer Science
• Book Title: High Performance Computing: 35th International Conference, ISC High Performance 2020, Frankfurt/Main, Germany, June 22–25, 2020 ; proceedings.
• ISBN: 9783030507428
• DOI: https://doi.org/10.1007/978-3-030-50743-5_23