A scalability Study of Columbia using the NAS Parallel Benchmarks
Saini Subhash, Chang Johnny, Hood Robert T., Jin Haoqiang H.
NASA Advanced Supercomputing Division
NASA Ames Research Center
Moffett Field, California 94035-1000, USA
e-mail: {ssaini/jchang/rhood/hjin}@mail.arc.nasa.gov
Received:
Rec. 12 July 2006
DOI: 10.12921/cmst.2006.SI.01.33-45
OAI: oai:lib.psnc.pl:601
Abstract:
The Columbia system at the NASA Advanced Supercomputing (NAS) facility is a cluster of 20 SGI Altix nodes, each with 512 Itanium 2 processors and 1 terabyte (TB) of shared-access memory. Four of the nodes are organized as a 2048-processor capabilitycomputing platform connected by two low-latency interconnects – NUMALink4 (NL4) and InfiniBand (IB). To evaluate the scalability of Columbia with respect to both increased processor counts and increased problem sizes, we used seven of the NAS Parallel Benchmarks and all three of the NAS multi-zone benchmarks. For NPB we ran three Classes B, C, and D of benchmarks. To measure the impact of some architectural features, we compared Columbia results with results obtained on a Cray Opteron Cluster consisting of 64 nodes, each with 2 AMD Opteron processors and 2 gigabytes (GB) of memory, connected with Myrinet 2000. In these experiments, we measured performance degradation due to contention for the memory buses on the SGI Altix BX2 nodes. We also observed the effectiveness of SGI’s NL4 interconnect over Myrinet. Finally, we saw that computations spanning multiple BX2 nodes connected with NL4 performed well. Some computations did almost as well when the IB interconnects was used.
Key words:
benchmarking, Computer architectures, performance evaluation
References:
[1] Top500, http://www.top500.org
[2] S. Saini, Hot Chips and Hot Interconnects for High End Computing Systems, M4, IEEE/ACM SC 2004, Pittsburgh (2004).
[3] S. Saini, Performance Comparison of Columbia 2048 and IBM Blue Gene/L, SGIUG 2005 Technical Conference and Tutorials, June 13-16, 2005 Munich (2005).
[4] NAS Parallel Benchmarks, http://www.nas.nasa.gov/ Resources/Software/npb.html (2006).
[5] S. Saini, R. Ciotti, T. N. Gunney, T. E. Spelce, A. Koniges, D. Dossa, P. Adamidis, R. Rabenseifner, S. R. Tiyyagura, M. Mueller, and Rod Fatoohi, Performance Evaluation of
Supercomputers using HPCC and IMB Benchmarks IPDPS 2006, PMEO, April 25-29, Rhodes, Greece (2006).
[6] S. Saini, R. Fatoohi, and R. Ciotti, Interconnect Performance Evaluation of SGI Altix 3700 BX2 Cray X1, Cray Opteron Cluster, and Dell PowerEdge, IPDPS 2006, PMEO, April 25-29, Rhodes, Greece (2006).
[7] S. Saini, R. Ciotti, T. N. Gunney, T. E. Spelce, A. Koniges, D. Dossa, P. Adamidis, R. Rabenseifner, S. R. Tiyyagura, M. Mueller, and Rod Fatoohi, Performance Comparison of
Cray X1 and Cray Opteron Cluster with Other Leading Platforms Using HPCC and IMB Benchmarks, CUG 2006, May 8-11, 2006 Lugano, Switzerland, (2006).
[8] S. Saini, P. Adamidis, R. Fatoohi,, J. Chang, and R. Ciotti, Performance Analysis of Cray X1 and Cray Opteron Cluster, CUG 2006, May 8-11, 2006 Lugano, Switzerland (2006).
[9] D. Lenoski, J. Laudon, K. Gharachorloo, A. Gupta and J. Hennessy, International Conference on Computer Architecture archive Proceedings of the 17th annual international symposium on Computer Architecture, Seattle, Washington, USA, 148-159 (1990).
[10] InfiniBand Trade Association, InfiniBand Architecture Specifications, Release 1.0 October 24, 2000, http://www.infinibandta.org/home/
[11] Advanced Micro Devices, http://www.amd.com/us-en/
[12] HyperTransport Consortium, http://www.hypertransport.org/
[13] Myricom, http://www.myri.com/
[14] H. Jin and R. Van de Wijngaart, Performance Characteristics of the Multi-zone NAS Parallel Benchmarks, Proceedings of International Parallel and Distributed Processing, Santa Fe, New Mexico, USA, (2004).
[15] The Blue Gene/L Team, IEEE/ACM Proceedings of SC 2002, Baltimore, Maryland, USA (2002).
[16] HPC Challenge Benchmark, http://icl.cs.utk.edu/hpcc/, (2006).
[17] R. Biswas, M. J. Djomehri, R. Hood, H. Jin, C. Kiris, and S. Saini, An Application-Based Performance Characterization of the Columbia Supercluster, IEEE/ACM SC 2005: 26 (2006).
[18] S. Saini, D. Talcott, H. Yeung, G. Myers, and R. Ciotti, A Scalability Study of SGI Clustered XFS Using HDF5 Based AMR Application, SGIUG 2006 Technical Conference and Tutorials, June 6-9, 2006 – Las Vegas, USA (2006).
The Columbia system at the NASA Advanced Supercomputing (NAS) facility is a cluster of 20 SGI Altix nodes, each with 512 Itanium 2 processors and 1 terabyte (TB) of shared-access memory. Four of the nodes are organized as a 2048-processor capabilitycomputing platform connected by two low-latency interconnects – NUMALink4 (NL4) and InfiniBand (IB). To evaluate the scalability of Columbia with respect to both increased processor counts and increased problem sizes, we used seven of the NAS Parallel Benchmarks and all three of the NAS multi-zone benchmarks. For NPB we ran three Classes B, C, and D of benchmarks. To measure the impact of some architectural features, we compared Columbia results with results obtained on a Cray Opteron Cluster consisting of 64 nodes, each with 2 AMD Opteron processors and 2 gigabytes (GB) of memory, connected with Myrinet 2000. In these experiments, we measured performance degradation due to contention for the memory buses on the SGI Altix BX2 nodes. We also observed the effectiveness of SGI’s NL4 interconnect over Myrinet. Finally, we saw that computations spanning multiple BX2 nodes connected with NL4 performed well. Some computations did almost as well when the IB interconnects was used.
Key words:
benchmarking, Computer architectures, performance evaluation
References:
[1] Top500, http://www.top500.org
[2] S. Saini, Hot Chips and Hot Interconnects for High End Computing Systems, M4, IEEE/ACM SC 2004, Pittsburgh (2004).
[3] S. Saini, Performance Comparison of Columbia 2048 and IBM Blue Gene/L, SGIUG 2005 Technical Conference and Tutorials, June 13-16, 2005 Munich (2005).
[4] NAS Parallel Benchmarks, http://www.nas.nasa.gov/ Resources/Software/npb.html (2006).
[5] S. Saini, R. Ciotti, T. N. Gunney, T. E. Spelce, A. Koniges, D. Dossa, P. Adamidis, R. Rabenseifner, S. R. Tiyyagura, M. Mueller, and Rod Fatoohi, Performance Evaluation of
Supercomputers using HPCC and IMB Benchmarks IPDPS 2006, PMEO, April 25-29, Rhodes, Greece (2006).
[6] S. Saini, R. Fatoohi, and R. Ciotti, Interconnect Performance Evaluation of SGI Altix 3700 BX2 Cray X1, Cray Opteron Cluster, and Dell PowerEdge, IPDPS 2006, PMEO, April 25-29, Rhodes, Greece (2006).
[7] S. Saini, R. Ciotti, T. N. Gunney, T. E. Spelce, A. Koniges, D. Dossa, P. Adamidis, R. Rabenseifner, S. R. Tiyyagura, M. Mueller, and Rod Fatoohi, Performance Comparison of
Cray X1 and Cray Opteron Cluster with Other Leading Platforms Using HPCC and IMB Benchmarks, CUG 2006, May 8-11, 2006 Lugano, Switzerland, (2006).
[8] S. Saini, P. Adamidis, R. Fatoohi,, J. Chang, and R. Ciotti, Performance Analysis of Cray X1 and Cray Opteron Cluster, CUG 2006, May 8-11, 2006 Lugano, Switzerland (2006).
[9] D. Lenoski, J. Laudon, K. Gharachorloo, A. Gupta and J. Hennessy, International Conference on Computer Architecture archive Proceedings of the 17th annual international symposium on Computer Architecture, Seattle, Washington, USA, 148-159 (1990).
[10] InfiniBand Trade Association, InfiniBand Architecture Specifications, Release 1.0 October 24, 2000, http://www.infinibandta.org/home/
[11] Advanced Micro Devices, http://www.amd.com/us-en/
[12] HyperTransport Consortium, http://www.hypertransport.org/
[13] Myricom, http://www.myri.com/
[14] H. Jin and R. Van de Wijngaart, Performance Characteristics of the Multi-zone NAS Parallel Benchmarks, Proceedings of International Parallel and Distributed Processing, Santa Fe, New Mexico, USA, (2004).
[15] The Blue Gene/L Team, IEEE/ACM Proceedings of SC 2002, Baltimore, Maryland, USA (2002).
[16] HPC Challenge Benchmark, http://icl.cs.utk.edu/hpcc/, (2006).
[17] R. Biswas, M. J. Djomehri, R. Hood, H. Jin, C. Kiris, and S. Saini, An Application-Based Performance Characterization of the Columbia Supercluster, IEEE/ACM SC 2005: 26 (2006).
[18] S. Saini, D. Talcott, H. Yeung, G. Myers, and R. Ciotti, A Scalability Study of SGI Clustered XFS Using HDF5 Based AMR Application, SGIUG 2006 Technical Conference and Tutorials, June 6-9, 2006 – Las Vegas, USA (2006).
