| MPI2007 license: | 4 | Test date: | Jun-2011 |
|---|---|---|---|
| Test sponsor: | SGI | Hardware Availability: | Mar-2011 |
| Tested by: | SGI | Software Availability: | Aug-2011 |
Results Table
| Benchmark | Base | Peak | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | |
| Results appear in the order in which they were run. Bold underlined text indicates a median measurement. | ||||||||||||||
| 104.milc | 768 | 27.5 | 56.9 | 25.8 | 60.6 | 25.7 | 60.9 | 768 | 27.5 | 56.9 | 25.8 | 60.6 | 25.7 | 60.9 |
| 107.leslie3d | 768 | 95.2 | 54.8 | 94.1 | 55.4 | 94.8 | 55.1 | 768 | 95.2 | 54.8 | 94.1 | 55.4 | 94.8 | 55.1 |
| 113.GemsFDTD | 768 | 747 | 8.45 | 749 | 8.42 | 749 | 8.42 | 192 | 409 | 15.4 | 407 | 15.5 | 407 | 15.5 |
| 115.fds4 | 768 | 25.4 | 76.7 | 25.4 | 76.7 | 25.9 | 75.3 | 696 | 27.6 | 70.8 | 25.1 | 77.7 | 25.2 | 77.4 |
| 121.pop2 | 768 | 192 | 21.5 | 190 | 21.7 | 195 | 21.1 | 768 | 192 | 21.5 | 190 | 21.7 | 195 | 21.1 |
| 122.tachyon | 768 | 45.1 | 62.0 | 45.1 | 62.1 | 44.9 | 62.2 | 768 | 45.1 | 62.0 | 45.1 | 62.1 | 44.9 | 62.2 |
| 126.lammps | 768 | 256 | 11.4 | 274 | 10.7 | 273 | 10.7 | 192 | 214 | 13.6 | 214 | 13.6 | 214 | 13.6 |
| 127.wrf2 | 768 | 102 | 76.5 | 101 | 77.3 | 101 | 77.3 | 768 | 102 | 76.5 | 101 | 77.3 | 101 | 77.3 |
| 128.GAPgeofem | 768 | 29.0 | 71.1 | 29.1 | 71.0 | 29.2 | 70.7 | 768 | 29.0 | 71.1 | 29.1 | 71.0 | 29.2 | 70.7 |
| 129.tera_tf | 768 | 61.1 | 45.3 | 61.1 | 45.3 | 61.8 | 44.8 | 768 | 61.1 | 45.3 | 61.1 | 45.3 | 61.8 | 44.8 |
| 130.socorro | 768 | 127 | 30.0 | 126 | 30.2 | 128 | 29.9 | 768 | 127 | 30.0 | 126 | 30.2 | 128 | 29.9 |
| 132.zeusmp2 | 768 | 58.5 | 53.0 | 58.5 | 53.0 | 58.7 | 52.9 | 528 | 57.0 | 54.4 | 56.9 | 54.5 | 56.8 | 54.6 |
| 137.lu | 768 | 58.0 | 63.4 | 58.0 | 63.4 | 57.9 | 63.5 | 504 | 55.5 | 66.2 | 55.3 | 66.4 | 55.4 | 66.4 |
| Hardware Summary | |
|---|---|
| Type of System: | Homogeneous |
| Compute Node: | SGI Altix ICE 8400EX Compute Node |
| Interconnect: | InfiniBand (MPI and I/O) |
| File Server Node: | SGI InfiniteStorage 4000 |
| Total Compute Nodes: | 32 |
| Total Chips: | 64 |
| Total Cores: | 768 |
| Total Threads: | 768 |
| Total Memory: | 2 TB |
| Base Ranks Run: | 768 |
| Minimum Peak Ranks: | 192 |
| Maximum Peak Ranks: | 768 |
| Software Summary | |
|---|---|
| C Compiler: | Intel C Compiler for Linux Version 11.1, Build 20100806 |
| C++ Compiler: | Intel C++ Compiler for Linux Version 11.1, Build 20100806 |
| Fortran Compiler: | Intel Fortran Compiler for Linux Version 11.1, Build 20100806 |
| Base Pointers: | 64-bit |
| Peak Pointers: | 64-bit |
| MPI Library: | SGI MPT 2.04 Patch 10789 |
| Other MPI Info: | OFED 1.4.2 |
| Pre-processors: | None |
| Other Software: | None |
Node Description: SGI Altix ICE 8400EX Compute Node
| Hardware | |
|---|---|
| Number of nodes: | 32 |
| Uses of the node: | compute |
| Vendor: | SGI |
| Model: | SGI Altix ICE 8400EX (AMD Opteron 6180 SE, 2.5GHz) |
| CPU Name: | AMD Opteron 6180 SE |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 24 |
| Cores per chip: | 12 |
| Threads per core: | 1 |
| CPU Characteristics: | 12 Cores/chip, 2.5 GHz |
| CPU MHz: | 2500 |
| Primary Cache: | 64 KB I + 64 KB D on chip per core |
| Secondary Cache: | 512 KB I+D on chip per core |
| L3 Cache: | 12 MB I+D on chip per chip, 6 MB shared / 6 cores |
| Other Cache: | None |
| Memory: | 64 GB (16 x 4 GB, 2Rx4 PC3-10600R-9, ECC) |
| Disk Subsystem: | None |
| Other Hardware: | None |
| Adapter: | Mellanox MT26428 ConnectX IB QDR (PCIe x8 Gen2 5 GT/s) |
| Number of Adapters: | 1 |
| Slot Type: | PCIe x8 Gen2 |
| Data Rate: | InfiniBand 4x QDR |
| Ports Used: | 2 |
| Interconnect Type: | InfiniBand |
| Software | |
|---|---|
| Adapter: | Mellanox MT26428 ConnectX IB QDR (PCIe x8 Gen2 5 GT/s) |
| Adapter Driver: | OFED-1.4.2 |
| Adapter Firmware: | 2.7.0 |
| Operating System: | SUSE Linux Enterprise Server 11 SP1 (x86_64) Kernel 2.6.32.27-0.2-default |
| Local File System: | NFSv3 |
| Shared File System: | NFSv3 IPoIB |
| System State: | Run Level 3 (Multi-User) |
| Other Software: | SGI Performance Suite 1.0, Build 702r19.sles11-1010072114 SGI Tempo Compute Node 2.2, Build 702r19.sles11-1010072114 |
Node Description: SGI InfiniteStorage 4000
| Hardware | |
|---|---|
| Number of nodes: | 1 |
| Uses of the node: | fileserver |
| Vendor: | SGI |
| Model: | SGI Altix 450 (Intel Itanium 2, 1.6GHz) |
| CPU Name: | Intel Itanium 2 9030 |
| CPU(s) orderable: | 2-38 chips |
| Chips enabled: | 2 |
| Cores enabled: | 4 |
| Cores per chip: | 2 |
| Threads per core: | 1 |
| CPU Characteristics: | 1.6GHz/8MB, 533MHz FSB |
| CPU MHz: | 1600 |
| Primary Cache: | 16 KB I + 16 KB D on chip per core |
| Secondary Cache: | 1 MB I + 256 KB D on chip per core |
| L3 Cache: | 4 MB I+D on chip per core |
| Other Cache: | None |
| Memory: | 24 GB (12 x 2 GB, 2Rx4 PC2-3200-3, ECC) |
| Disk Subsystem: | 16 TB RAID 5 32 x 500 GB SATA (Seagate Barracuda 7.2K) |
| Other Hardware: | None |
| Adapter: | Mellanox MT25208 InfiniHost III Ex (PCIe x8 Gen1 2.5 GT/s) |
| Number of Adapters: | 2 |
| Slot Type: | PCIe x8 Gen1 |
| Data Rate: | InfiniBand 4x DDR |
| Ports Used: | 2 |
| Interconnect Type: | InfiniBand |
| Software | |
|---|---|
| Adapter: | Mellanox MT25208 InfiniHost III Ex (PCIe x8 Gen1 2.5 GT/s) |
| Adapter Driver: | OFED-1.4.2 |
| Adapter Firmware: | 5.3.0 |
| Operating System: | SUSE Linux Enterprise Server 11 SP1 (ia64) Kernel 2.6.32.12-0.7-default |
| Local File System: | xfs |
| Shared File System: | -- |
| System State: | Run Level 3 (Multi-User) |
| Other Software: | SGI ProPack 7SP1 for Linux, Build 701r2.sles11-1005242307 |
Interconnect Description: InfiniBand (MPI and I/O)
| Hardware | |
|---|---|
| Vendor: | Mellanox Technologies and SGI |
| Model: | None |
| Switch Model: | SGI QDR_1.5_HYPR_2454 with Mellanox Device 48438 (Infiniscale IV) |
| Number of Switches: | 4 |
| Number of Ports: | 36 |
| Data Rate: | InfiniBand 4x QDR |
| Firmware: | 5040005 |
| Topology: | Enhanced Hypercube |
| Primary Use: | MPI and I/O traffic |
Submit Notes
The config file option 'submit' was used.
General Notes
Software environment: export MPI_REQUEST_MAX=65536 export MPI_TYPE_MAX=32768 export MPI_BUFS_THRESHOLD=1 ulimit -s unlimited BIOS settings: AMI BIOS version 1.0a Job Placement: Each MPI job is assigned to a topologically compact set of nodes, i.e. the minimal needed number of switches was used for each job: 1 switch for up to 192 ranks, 2 switches for 384 ranks, 4 switches for 768 ranks, 8 switches for 1536 ranks and 16 switches for 3072 ranks. Peak run: In the peak run, some benchmarks used different number of ranks from base. It is the only difference between base and peak.
Compiler Invocation
C benchmarks:
| icc |
C++ benchmarks:
| 126.lammps: | icpc |
Fortran benchmarks:
| ifort |
Benchmarks using both Fortran and C:
| icc ifort |
Portability Flags
| 121.pop2: | -DSPEC_MPI_CASE_FLAG |
| 127.wrf2: | -DSPEC_MPI_CASE_FLAG -DSPEC_MPI_LINUX |
Base Optimization Flags
C benchmarks:
| -O3 -xSSE2 -ipo -no-prec-div |
C++ benchmarks:
| 126.lammps: | -O3 -xSSE2 -ipo -no-prec-div -ansi-alias |
Fortran benchmarks:
| -O3 -xSSE2 -ipo -no-prec-div |
Benchmarks using both Fortran and C:
| -O3 -xSSE2 -ipo -no-prec-div |
Peak Optimization Flags
C benchmarks:
| 104.milc: | basepeak = yes |
| 122.tachyon: | basepeak = yes |
C++ benchmarks:
| 126.lammps: | -O3 -xSSE2 -ipo -no-prec-div -ansi-alias |
Fortran benchmarks:
| 107.leslie3d: | basepeak = yes |
| 113.GemsFDTD: | -O3 -xSSE2 -ipo -no-prec-div |
| 129.tera_tf: | basepeak = yes |
| 137.lu: | Same as 113.GemsFDTD |
Benchmarks using both Fortran and C:
| 115.fds4: | -O3 -xSSE2 -ipo -no-prec-div |
| 121.pop2: | basepeak = yes |
| 127.wrf2: | basepeak = yes |
| 128.GAPgeofem: | basepeak = yes |
| 130.socorro: | basepeak = yes |
| 132.zeusmp2: | Same as 115.fds4 |