High Performance Computing Using Out-of- Core Sparse Direct Solvers

Authors: Mandhapati P. Raju, Siddhartha Khaitan

Abstract:

In-core memory requirement is a bottleneck in solving large three dimensional Navier-Stokes finite element problem formulations using sparse direct solvers. Out-of-core solution strategy is a viable alternative to reduce the in-core memory requirements while solving large scale problems. This study evaluates the performance of various out-of-core sequential solvers based on multifrontal or supernodal techniques in the context of finite element formulations for three dimensional problems on a Windows platform. Here three different solvers, HSL_MA78, MUMPS and PARDISO are compared. The performance of these solvers is evaluated on a 64-bit machine with 16GB RAM for finite element formulation of flow through a rectangular channel. It is observed that using out-of-core PARDISO solver, relatively large problems can be solved. The implementation of Newton and modified Newton's iteration is also discussed.

Keywords: Out-of-core, PARDISO, MUMPS, Newton.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1060080

References:

[1] T. A. Davis and I.S. Duff, "A combined unifrontal/multifrontal method for unsymmetric sparse matrices," ACM Trans. Math. Soft., vol. 25, no. 1, 1997, pp. 1-19.
[2] O. Schenk, K. Gartner, and W. Fichtner, "Efficient Sparse LU Factorization with Left-right Looking Strategy on Shared Memory Multiprocessors," BIT, vol. 40, no. 1, 2000, pp. 158-176.
[3] B. M. Irons, "A frontal solution scheme for finite element analysis," Numer. Meth. Engg., vol. 2, 1970, pp. 5-32.
[4] M. P. Raju, and J. S. T-ien, "Development of Direct Multifrontal Solvers for Combustion Problems," Numerical Heat Transfer-Part B, vol. 53, 2008, pp. 1-17.
[5] M. P. Raju, and J. S. T-ien, "Modelling of Candle Wick Burning with a Self-trimmed Wick," Comb. Theory Modell., vol. 12, no. 2, 2008, pp. 367-388.
[6] M. P. Raju, and J. S. T-ien, "Two-phase flow inside an externally heated axisymmetric porous wick," vol. 11, no. 8, 2008, pp. 701-718.
[7] P. K. Gupta, and K. V. Pagalthivarthi, "Application of Multifrontal and GMRES Solvers for Multisize Particulate Flow in Rotating Channels," Prog. Comput Fluid Dynam., vol. 7, 2007, pp. 323-336.
[8] S. Khaitan, J. McCalley, Q. Chen, "Multifrontal solver for online power system time-domain simulation," IEEE Transactions on Power Systems, vol. 23, no. 4, 2008, pp. 1727-1737.
[9] S. Khaitan, C. Fu, J. D. McCalley, "Fast parallelized algorithms for online extended-term dynamic cascading analysis," PSCE, 2009, pp. 1- 7.
[10] J. McCalley, S. Khaitan, "Risk of Cascading outages", Final Report, PSrec Report, S-26, August 2007. Available at http://www.pserc.org/docsa/Executive_Summary_Dobson_McCalley_C ascading_Outage_ S-2626_PSERC_ Final_Report.pdf
[11] P. R. Amestoy, and I. S. Duff, "Vectorization of a multiprocessor multifrontal code," International Journal of Supercomputer Applications, vol. 3, 1989, pp. 41-59.
[12] P. R. Amestoy, I. S. Duff, J. Koster and J. Y. L-Excellent, "A fully asynchronous multifrontal solver using distributed dynamic scheduling," SIAM Journal on Matrix Analysis and Applications, vol. 23, no. 1, 2001, pp. 15-41.
[13] P. R. Amestoy, I. S. Duff, and J. Y. L-Excellent, "Multifrontal parallel distributed symmetric and unsymmetric solvers," Comput. Methods Appl. Mech. Eng., vol. 184, 2000, pp. 501-520.
[14] O. Schenk, "Scalable Parallel Sparse LU Factorization Methods on Shared Memory Multiprocessors," Ph.D. dissertation, ETH Zurich, 2000.
[15] O. Schenk, and K. Gartner, "Sparse Factorization with Two-Level Scheduling in PARDISO," in Proc. 10th SIAM conf. Parallel Processing for Scientific Computing, Portsmouth, Virginia, March 12-14, 2001.
[16] O. Schenk, and K. Gartner, "Two-level scheduling in PARDISO: Improved Scalability on Shared Memory Multiprocessing Systems," Parallel Computing, vol. 28, 2002, pp. 187-197.
[17] O. Schenk, and K. Gartner, "Solving Unsymmetric Sparse Systems of Linear Equations with PARDISO," Journal Future Generation Computer Systems, vol. 20, no. 3, 2004, pp. 475-487.
[18] Intel MKL Reference Manual, Intel® Math Kernel Library (MKL), 2007. Available: http://www.intel.com/software/products/mkl/
[19] J. A. Scott, Numerical Analysis Group Progress Report, RAL-TR-2008- 001, Rutherford Appleton Laboratory, 2008.
[20] P. R. Amestoy, T. A. Davis, and I. S. Duff, "An approximate minimum degree ordering algorithm," SIAM Journal on Matrix Analysis and Applications, vol. 17, 1996, pp. 886-905.
[21] P. R. Amestoy, "Recent progress in parallel multifrontal solvers for unsymmetric sparse matrices," in Proc. 15th World Congress on Scientific Computation, Modelling and Applied Mathematics, IMACS, Berlin, 1997.
[22] J. Schulze, "Towards a tighter coupling of bottom-up and top-down sparse matrix ordering methods," BIT, vol. 41, no. 4, 2001, pp. 800-841.
[23] G. Karypis, and V. Kumar, "METIS - A Software Package for Partitioning Unstructured Graphs, Partitioning Meshes, and Computing Fill-Reducing Orderings of Sparse Matrices - Version 4.0," University of Minnesota, September 1998.