An MPI-IO In-Memory driver for non-volatile pooled memory of the Kove XPD

Kunkel, J. and Betke, E. (2017) An MPI-IO In-Memory driver for non-volatile pooled memory of the Kove XPD. In: Kunkel, J., Yokota, R., Taufer, M. and Shalf, J. (eds.) High Performance Computing. Lecture Notes in Computer Science, 10524 (10524). Springer, Cham, pp. 679-690. ISBN 9783319676296 doi: 10.1007/978-3-319-67630-2_48

Abstract/Summary

Many scientific applications are limited by the performance offered by parallel file systems. SSD based burst buffers provide significant better performance than HDD backed storage but at the expense of capacity. Clearly, achieving wire-speed of the interconnect and predictable low latency I/O is the holy grail of storage. In-memory storage promises to provide optimal performance exceeding SSD based solutions. Kove®’s XPD® offers pooled memory for cluster systems. This remote memory is asynchronously backed up to storage devices of the XPDs and considered to be non-volatile. Albeit the system offers various APIs to access this memory such as treating it as a block device, it does not allow to expose it as file system that offers POSIX or MPI-IO semantics. In this paper, we (1) describe the XPD-MPIIO-driver which supports the scale-out architecture of the XPDs. This MPI-agnostic driver enables high-level libraries to utilize the XPD’s memory as storage. (2) A thorough performance evaluation of the XPD is conducted. This includes scale-out testing of the infrastructure and “metadata” operations but also performance variability. We show that the driver and storage architecture is able to nearly saturate wire-speed of Infiniband (60+ GiB/s with 14FDR links) while providing low latency and little performance variability.

Item Type	Book or Report Section
URI	https://reading-clone.eprints-hosting.org/id/eprint/77670
Identification Number/DOI	10.1007/978-3-319-67630-2_48
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Publisher	Springer
Download/View statistics	View download statistics for this item