Scalable Deployment and Configuration of High-Performance Virtual Clusters

Authors: Kyrre M Begnum, Matthew Disney

Abstract:

Virtualization and high performance computing have been discussed from a performance perspective in recent publications. We present and discuss a flexible and efficient approach to the management of virtual clusters. A virtual machine management tool is extended to function as a fabric for cluster deployment and management. We show how features such as saving the state of a running cluster can be used to avoid disruption. We also compare our approach to the traditional methods of cluster deployment and present benchmarks which illustrate the efficiency of our approach.

Keywords: Cluster management, clusters, high-performance, virtual machines, Xen

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1083735

References:

[1] Mark F. Mergen, Volkmar Uhlig, Orran Krieger, and Jimi Xenidis. Virtualization for high-performance computing. SIGOPS Oper. Syst. Rev., 40(2):8-11, 2006.
[2] Ivan Krsul, Arijit Ganguly, Jian Zhang, Jose A. B. Fortes, and Renato J. Figueiredo. VMPlants: Providing and Managing Virtual Machine Execution Environments for Grid Computing. In SC -04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing, page 7, Washington, DC, USA, 2004. IEEE Computer Society.
[3] Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. Xen and the art of virtualization. In SOSP -03: Proceedings of the nineteenth ACM symposium on Operating systems principles, pages 164-177, New York, NY, USA, 2003. ACM Press.
[4] K. Begnum and J. Sechrest. The MLN homepage. http://mln.sourceforge.net. Last accessed August 28, 2006.
[5] Jeff Dike. A user-mode port of the Linux kernel. In Proceedings of the 2000 Linux Showcase and Conference, October 2000.
[6] Kyrre Begnum. Manage large virtual networks. In To appear in: Proceedings of the 20th Large installation system administration conference. USENIX, 2006.
[7] Ubuntu Linux. http://www.ubuntu.com/. Last accessed August 27, 2006.
[8] TORQUE Resource Manager 2.0. http://www.clusterresources.com/pages/products/torqueresource- manager.php. Last accessed August 27, 2006.
[9] Greg Burns, Raja Daoud, and James Vaigl. LAM: An Open Cluster Environment for MPI. In Proceedings of Supercomputing Symposium, pages 379-386, 1994.
[10] Jeffrey M. Squyres and Andrew Lumsdaine. A Component Architecture for LAM/MPI. In Proceedings, 10th European PVM/MPI Users- Group Meeting, number 2840 in Lecture Notes in Computer Science, pages 379-387, Venice, Italy, September / October 2003. Springer-Verlag.
[11] R. Clint Whaley and Jack J. Dongarra. Automatically tuned linear algebra software. In Supercomputing -98: Proceedings of the 1998 ACM/IEEE conference on Supercomputing (CDROM), pages 1-27, Washington, DC, USA, 1998. IEEE Computer Society.
[12] A. Petitet, R. C. Whaley, J. Dongarra, and A. Cleary. HPL - A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers. http://www.netlib.org/benchmark/hpl/. Last accessed August 27, 2006.
[13] Mark Burgess. Cfengine: a site configuration engine. In Computing systems, Vol8, No. 3. USENIX, 1995.
[14] Jack Dongarra. The LINPACK Benchmark: An Explanation. In Proceedings of the 1st International Conference on Supercomputing, pages 456-474, London, UK, 1988. Springer-Verlag.
[15] Top500 Supercomputing Sites. http://www.top500.org/. Last accessed August 27, 2006.
[16] FAI - Fully Automatic Installation. http://www.informatik.unikoeln. de/fai/. Last accessed August 28, 2006.
[17] OSCAR (Open Source Cluster Application Resources). http://oscar.openclustergroup.org. Last accessed August 28, 2006.
[18] Rocks Clusters. http://www.rocksclusters.org/. Last accessed August 28, 2006.
[19] XenSource - XenEnterprise. http://www.xensource.com/products/ xen_enterprise/. Last accessed August 28, 2006.
[20] VMWare - An EMC Company. http://www.vmware.com. Last accessed August 28, 2006.