| MPI2007 license: | 14 | Test date: | Mar-2016 |
|---|---|---|---|
| Test sponsor: | SGI | Hardware Availability: | Mar-2016 |
| Tested by: | SGI | Software Availability: | May-2016 |
Results Table
| Benchmark | Base | Peak | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | |
| Results appear in the order in which they were run. Bold underlined text indicates a median measurement. | ||||||||||||||
| 104.milc | 176 | 64.1 | 24.4 | 63.2 | 24.8 | 63.8 | 24.5 | |||||||
| 107.leslie3d | 176 | 179 | 29.2 | 178 | 29.3 | 177 | 29.5 | |||||||
| 113.GemsFDTD | 176 | 189 | 33.3 | 196 | 32.2 | 197 | 32.0 | |||||||
| 115.fds4 | 176 | 54.8 | 35.6 | 54.5 | 35.8 | 55.1 | 35.4 | |||||||
| 121.pop2 | 176 | 187 | 22.1 | 188 | 22.0 | 188 | 22.0 | |||||||
| 122.tachyon | 176 | 97.5 | 28.7 | 99.4 | 28.1 | 121 | 23.1 | |||||||
| 126.lammps | 176 | 127 | 23.0 | 126 | 23.2 | 126 | 23.1 | |||||||
| 127.wrf2 | 176 | 132 | 59.0 | 132 | 59.3 | 131 | 59.4 | |||||||
| 128.GAPgeofem | 176 | 46.7 | 44.3 | 47.4 | 43.6 | 47.2 | 43.8 | |||||||
| 129.tera_tf | 176 | 84.1 | 32.9 | 84.3 | 32.8 | 84.1 | 32.9 | |||||||
| 130.socorro | 176 | 74.7 | 51.1 | 74.3 | 51.4 | 73.2 | 52.2 | |||||||
| 132.zeusmp2 | 176 | 83.0 | 37.4 | 83.2 | 37.3 | 83.6 | 37.1 | |||||||
| 137.lu | 176 | 75.1 | 48.9 | 73.5 | 50.0 | 73.3 | 50.2 | |||||||
| Hardware Summary | |
|---|---|
| Type of System: | Homogeneous |
| Compute Node: | SGI Rackable C2112-4GP3 Compute Node |
| Interconnects: | InfiniBand MPI InfiniBand I/O |
| File Server Node: | SGI MIS Server |
| Total Compute Nodes: | 4 |
| Total Chips: | 8 |
| Total Cores: | 176 |
| Total Threads: | 176 |
| Total Memory: | 512 GB |
| Base Ranks Run: | 176 |
| Minimum Peak Ranks: | -- |
| Maximum Peak Ranks: | -- |
| Software Summary | |
|---|---|
| C Compiler: | Intel C++ Composer XE 2013 for Linux, Version 14.0.3.174 Build 20140422 |
| C++ Compiler: | Intel C++ Composer XE 2013 for Linux, Version 14.0.3.174 Build 20140422 |
| Fortran Compiler: | Intel Fortran Composer XE 2013 for Linux, Version 14.0.3.174 Build 20140422 |
| Base Pointers: | 64-bit |
| Peak Pointers: | Not Applicable |
| MPI Library: | SGI MPT 2.14 |
| Other MPI Info: | MLNX_OFED_LINUX-3.1-1.0.3 |
| Pre-processors: | None |
| Other Software: | None |
Node Description: SGI Rackable C2112-4GP3 Compute Node
| Hardware | |
|---|---|
| Number of nodes: | 4 |
| Uses of the node: | compute |
| Vendor: | SGI |
| Model: | SGI Rackable C2112-4GP3 (Intel Xeon E5-2699 v4, 2.20 GHz) |
| CPU Name: | Intel Xeon E5-2699 v4 |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 44 |
| Cores per chip: | 22 |
| Threads per core: | 1 |
| CPU Characteristics: | 22 Core, 2.20 GHz, 9.6 GT/s QPI Intel Turbo Boost Technology up to 3.60 GHz Hyper-Threading Technology disabled |
| CPU MHz: | 2220 |
| Primary Cache: | 32 KB I + 32 KB D on chip per core |
| Secondary Cache: | 256 KB I+D on chip per core |
| L3 Cache: | 55 MB I+D on chip per chip |
| Other Cache: | None |
| Memory: | 128 GB (8 x 16 GB 2Rx4 PC4-2400T-R) |
| Disk Subsystem: | None |
| Other Hardware: | None |
| Adapter: | Mellanox MT27620 with ConnectX-4 (PCIe x16 Gen3 8 GT/s) |
| Number of Adapters: | 1 |
| Slot Type: | PCIe x16 Gen3 |
| Data Rate: | InfiniBand 4x EDR |
| Ports Used: | 1 |
| Interconnect Type: | InfiniBand |
| Adapter: | Mellanox MT27500 with ConnectX-3 (PCIe x8 Gen3 8 GT/s) |
| Number of Adapters: | 1 |
| Slot Type: | PCIe x8 Gen3 |
| Data Rate: | InfiniBand 4x FDR |
| Ports Used: | 1 |
| Interconnect Type: | InfiniBand |
| Software | |
|---|---|
| Adapter: | Mellanox MT27620 with ConnectX-4 (PCIe x16 Gen3 8 GT/s) |
| Adapter Driver: | OFED-3.1.1-0.3 |
| Adapter Firmware: | 12.12.1240 |
| Adapter: | Mellanox MT27500 with ConnectX-3 (PCIe x8 Gen3 8 GT/s) |
| Adapter Driver: | OFED-3.1.1-0.0 |
| Adapter Firmware: | 2.35.5100 |
| Operating System: | SUSE Linux Enterprise Server 12 (x86_64), Kernel 3.12.44-52.10-default |
| Local File System: | ext3 |
| Shared File System: | NFSv3 IPoIB |
| System State: | Multi-user, run level 3 |
| Other Software: | SGI Tempo Service Node 3.2.0, Build 713r26.sles12-1510192000 |
Node Description: SGI MIS Server
| Hardware | |
|---|---|
| Number of nodes: | 1 |
| Uses of the node: | fileserver |
| Vendor: | SGI |
| Model: | SGI MIS Server (Intel Xeon X2670, 2.60 GHz) |
| CPU Name: | Intel Xeon E5-2670 |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 16 |
| Cores per chip: | 8 |
| Threads per core: | 2 |
| CPU Characteristics: | Intel Turbo Boost Technology up to 3.30 GHz Hyper-Threading Technology enabled |
| CPU MHz: | 2601 |
| Primary Cache: | 32 KB I + 32 KB D on chip per core |
| Secondary Cache: | 256 KB I+D on chip per core |
| L3 Cache: | 20 MB I+D on chip per chip |
| Other Cache: | None |
| Memory: | 128 GB (8 * 16 GB 2Rx4 PC3-10600R-9, ECC) |
| Disk Subsystem: | 45 TB RAID 6 12 x 1 TB SATA (Seagate Constellation, 7200RPM) |
| Other Hardware: | None |
| Adapter: | Mellanox MT27500 with ConnectX-3 ASIC |
| Number of Adapters: | 2 |
| Slot Type: | PCIe x8 Gen3 |
| Data Rate: | InfiniBand 4x FDR |
| Ports Used: | 2 |
| Interconnect Type: | InfiniBand |
| Software | |
|---|---|
| Adapter: | Mellanox MT27500 with ConnectX-3 ASIC |
| Adapter Driver: | MLNX_OFED_LINUX-3.1-1.0.3 |
| Adapter Firmware: | 2.35.5100 |
| Operating System: | SUSE Linux Enterprise Server 11 SP3 (x86_64), Kernel 3.0.101-0.46-default |
| Local File System: | xfs |
| Shared File System: | -- |
| System State: | Multi-user, run level 5 |
| Other Software: | SGI Foundation Software 2.10 Build 710r16.sles11sp3-1404092103 |
Interconnect Description: InfiniBand MPI
| Hardware | |
|---|---|
| Vendor: | Mellanox Technologies |
| Model: | None |
| Switch Model: | Mellanox SB7790 |
| Number of Switches: | 6 |
| Number of Ports: | 36 |
| Data Rate: | InfiniBand 4x EDR |
| Firmware: | 11.1.102 |
| Topology: | Fat Tree |
| Primary Use: | MPI traffic |
Interconnect Description: InfiniBand I/O
| Hardware | |
|---|---|
| Vendor: | Mellanox Technologies |
| Model: | None |
| Switch Model: | Mellanox MSX6036F-1SFS |
| Number of Switches: | 2 |
| Number of Ports: | 36 |
| Data Rate: | InfiniBand 4x FDR |
| Firmware: | 9.3.5080 |
| Switch Model: | Mellanox MSX6025 |
| Number of Switches: | 4 |
| Number of Ports: | 36 |
| Data Rate: | InfiniBand 4x FDR |
| Firmware: | 9.3.6000 |
| Topology: | Fat Tree |
| Primary Use: | I/O traffic |
Submit Notes
The config file option 'submit' was used.
General Notes
130.socorro (base): "nullify_ptrs" src.alt was used.
129.tera_tf (base): "add_rank_support" src.alt was used.
Software environment:
export MPI_REQUEST_MAX=65536
export MPI_TYPE_MAX=32768
export MPI_IB_DEVS=1
export MPI_CONNECTIONS_THRESHOLD=0
export MPI_IB_UPGRADE_SENDS=50
export MPI_IB_IMM_UPGRADE=false
export MPI_IB_HYPER_LAZY=false
ulimit -s unlimited
BIOS settings:
AMI BIOS version T20151001184140
Hyper-Threading Technology disabled
Transparent HugePages enabled
Intel Turbo Boost Technology enabled (default)
Intel Turbo Boost Technology activated with
modprobe acpi_cpufreq
cpupower frequency-set -u 2601MHz -d 2601MHz -g performance
Job Placement:
Each MPI job was assigned to a topologically compact set
of nodes, i.e. the minimal needed number of leaf switches
was used for each job: 1 switch for up to 32 sockets, and
2 switches for up to 64 sockets.
Additional notes regarding interconnect:
The Infiniband network consists of two independent planes,
with half the switches in the system allocated to each plane.
I/O traffic is restricted to one plane, while MPI traffic is
restricted to the other plane.
Base Compiler Invocation
C benchmarks:
| icc |
C++ benchmarks:
| 126.lammps: | icpc |
Fortran benchmarks:
| ifort |
Benchmarks using both Fortran and C:
| icc ifort |
Base Portability Flags
| 121.pop2: | -DSPEC_MPI_CASE_FLAG |
| 127.wrf2: | -DSPEC_MPI_CASE_FLAG -DSPEC_MPI_LINUX |
| 130.socorro: | -assume nostd_intent_in |