Performance Analysis of the Subgroup Method for Collective I/O

Authors: Kwangho Cha, Hyeyoung Cho, Sungho Kim

Abstract:

As many scientific applications require large data processing, the importance of parallel I/O has been increasingly recognized. Collective I/O is one of the considerable features of parallel I/O and enables application programmers to easily handle their large data volume. In this paper we measured and analyzed the performance of original collective I/O and the subgroup method, the way of using collective I/O of MPI effectively. From the experimental results, we found that the subgroup method showed good performance with small data size.

Keywords: Collective I/O, MPI, parallel file system.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1055974

References:

[1] John M. May, "Parallel I/O for High Performance Computing," Morgan Kaufmann, 2000.
[2] MPICH-A Portable Implementation of MPI, http://www-unix.mcs.anl. gov/mpi/mpich
[3] LAM/MPI Parallel Computing, http://www.lam-mpi.org
[4] Kwangho Cha, Taeyoung Hong, and Jeongwoo Hong, "The Subgroup Method for Collective I/O," Proc. The 5th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT 2004), LNCS 3320, pp. 301Ôê╝304, Dec. 2004.
[5] Avery Ching, Alok Choudhary, Wei-keng Liao, Rob Ross, and William Gropp, "Noncontiguous I/O through PVFS," Proc. IEEE International Conference on Cluster Computing, pp 405Ôê╝414, 2002.
[6] Philip H. Carns, Walter B. Ligon, III, Robert B. Ross, and Rajeev Thakur, "PVFS: A Parallel File System for Linux Clusters," Proc. 4th Annual Linux Showcase and Conference, pp 317Ôê╝327, 2000.
[7] David Kotz, "Disk-directed I/O for MIMD multiprocessors," ACM Transactions on Computer Systems, Vol. 15, No. 1, pp 41Ôê╝74, Feb. 1997.
[8] Rajesh Bordawekar, "Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences," Proc. 11th international conference on Supercomputing, pp 20Ôê╝27, 1997.
[9] Rajeev Thakur, William Gropp, and Ewing Lusk, "Data sieving and collective I/O in ROMIO," Proc. of the 7th Symposium on the Frontiers of Massively Parallel Computation, pp 182Ôê╝189, 1999.
[10] William Gropp, Ewing Lusk, and Rajeev Thakur, "Using MPI-2: Advanced Features of the Message Passing Interface," The MIT Press, 1999.
[11] Hakan Taki and Gil Utard, "MPI-IO on a parallel file system for cluster of workstations," Proc. 1st IEEE Computer Society International Workshop on Cluster Computing, pp 150Ôê╝157, 1999.
[12] ROMIO: A High-Performance, Portable MPI-IO Implementation, http://www-unix.mcs.anl.gov/romio
[13] Jean-Pierre Prost, Richard Treumann, Robert Blackmore, Carol Hartan, Richard Hedges, Bin Jia, Alice Koniges, and Alison White, "Towards a High-Performance Implementation of MPI-IO on Top of GPFS," Proc. The 6th International Euro-Par Conference, LNCS 1900, pp 1253Ôê╝1262, Sep. 2000.
[14] Jean-Pierre Prost, Richard Treumann, Richard Hedges, Bin Jia, and Alice Koniges, "MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS," Proc. 2001 ACM/IEEE conference on Supercomputing( CDROM), pp 17Ôê╝17, Nov. 2001.