Predicting I/O performance in HPC using artificial neural networks

Schmidt, J. F. and Kunkel, J. M. (2016) Predicting I/O performance in HPC using artificial neural networks. Supercomputing Frontiers and Innovations, 3 (3). pp. 19-33. ISSN 2313-8734 doi: 10.14529/jsfi160303

Abstract/Summary

The prediction of file access times is an important part for the modeling of supercomputer's storage systems. These models can be used to develop analysis tools which support the users to integrate efficient I/O behavior. In this paper, we analyze and predict the access times of a Lustre file system from the client perspective. Therefore, we measure file access times in various test series and developed different models for predicting access times. The evaluation shows that in models utilizing artificial neural networks the average prediciton error is about 30% smaller than in linear models. A phenomenon in the distribution of file access times is of particular interest: File accesses with identical parameters show several typical access times.The typical access times usually differ by orders of magnitude and can be explained with a different processing of the file accesses in the storage system - an alternative I/O path. We investigate a method to automatically determine the alternative I/O path and quantify the significance of knowledge about the internal processing. It is shown that the prediction error is improved significantly with this approach.

Item Type	Article
URI	https://reading-clone.eprints-hosting.org/id/eprint/77675
Identification Number/DOI	10.14529/jsfi160303
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Publisher	South Urals University
Download/View statistics	View download statistics for this item