Principles for automated and reproducible benchmarking

Koskela, T. ORCID: https://orcid.org/0000-0002-5813-6539, Christidi, I. ORCID: https://orcid.org/0000-0002-5045-7987, Giordano, M. ORCID: https://orcid.org/0000-0002-7218-2873, Dubrovska, E. ORCID: https://orcid.org/0009-0003-8066-5458, Quinn, J. ORCID: https://orcid.org/0000-0002-0268-7032, Maynard, C. ORCID: https://orcid.org/0000-0002-6253-9154, Case, D. ORCID: https://orcid.org/0009-0001-3735-5687, Olgu, K. ORCID: https://orcid.org/0000-0003-0351-2055 and Deakin, T. ORCID: https://orcid.org/0000-0002-6439-4171 (2023) Principles for automated and reproducible benchmarking. In: SC-W 2023: Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, 12-17 Nov 2023, Denver, Colorado, pp. 609-618. doi: 10.1145/3624062.3624133 (ISBN: 9798400707858)

Abstract/Summary

The diversity in processor technology used by High Performance Computing (HPC) facilities is growing, and so applications must be written in such a way that they can attain high levels of performance across a range of different CPUs, GPUs, and other accelerators. Measuring application performance across this wide range of platforms becomes crucial, but there are significant challenges to do this rigorously, in a time efficient way, whilst assuring results are scientifically meaningful, reproducible, and actionable. This paper presents a methodology for measuring and analysing the performance portability of a parallel application and shares a software framework which combines and extends adopted technologies to provide a usable benchmarking tool. We demonstrate the flexibility and effectiveness of the methodology and benchmarking framework by showcasing a variety of benchmarking case studies which utilise a stable of supercomputing resources at a national scale.

Item Type	Conference or Workshop Item (Paper)
URI	https://reading-clone.eprints-hosting.org/id/eprint/114121
Identification Number/DOI	10.1145/3624062.3624133
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science Science > School of Mathematical, Physical and Computational Sciences > Department of Meteorology
Publisher	ACM
Download/View statistics	View download statistics for this item