| MPI2007 license: | 0018 | Test date: | May-2007 |
|---|---|---|---|
| Test sponsor: | QLogic Corporation | Hardware Availability: | Nov-2006 |
| Tested by: | QLogic Performance Engineering | Software Availability: | Jul-2007 |
Results Table
| Benchmark | Base | Peak | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | |
| Results appear in the order in which they were run. Bold underlined text indicates a median measurement. | ||||||||||||||
| 104.milc | 128 | 154 | 10.2 | 155 | 10.1 | 155 | 10.1 | |||||||
| 107.leslie3d | 128 | 617 | 8.46 | 616 | 8.48 | 571 | 9.13 | |||||||
| 113.GemsFDTD | 128 | 473 | 13.3 | 501 | 12.6 | 498 | 12.7 | |||||||
| 115.fds4 | 128 | 208 | 9.38 | 209 | 9.33 | 220 | 8.86 | |||||||
| 121.pop2 | 128 | 413 | 9.99 | 449 | 9.19 | 416 | 9.93 | |||||||
| 122.tachyon | 128 | 480 | 5.83 | 480 | 5.83 | 468 | 5.98 | |||||||
| 126.lammps | 128 | 251 | 11.6 | 252 | 11.6 | 253 | 11.5 | |||||||
| 127.wrf2 | 128 | 466 | 16.7 | 468 | 16.7 | 466 | 16.7 | |||||||
| 128.GAPgeofem | 128 | 155 | 13.3 | 155 | 13.3 | 155 | 13.3 | |||||||
| 129.tera_tf | 128 | 300 | 9.21 | 300 | 9.23 | 301 | 9.21 | |||||||
| 130.socorro | 128 | 299 | 12.8 | 293 | 13.0 | 299 | 12.8 | |||||||
| 132.zeusmp2 | 128 | 276 | 11.2 | 276 | 11.3 | 276 | 11.3 | |||||||
| 137.lu | 128 | 304 | 12.1 | 306 | 12.0 | 309 | 11.9 | |||||||
| Software Summary | |
|---|---|
| C Compiler: | QLogic PathScale C Compiler 3.0 |
| C++ Compiler: | QLogic PathScale C++ Compiler 3.0 |
| Fortran Compiler: | QLogic PathScale Fortran Compiler 3.0 |
| Base Pointers: | 64-bit |
| Peak Pointers: | 64-bit |
| MPI Library: | QLogic InfiniPath MPI 2.1 |
| Other MPI Info: | None |
| Pre-processors: | No |
| Other Software: | None |
Node Description: Rackable, IWILL, AMD
| Hardware | |
|---|---|
| Number of nodes: | 32 |
| Uses of the node: | compute, head |
| Vendor: | Rackable Systems, IWILL, AMD |
| Model: | Rackable Systems C1000 chassis, IWILL DK8-HTX motherboard |
| CPU Name: | AMD Opteron 290 |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 4 |
| Cores per chip: | 2 |
| Threads per core: | 1 |
| CPU Characteristics: | -- |
| CPU MHz: | 2800 |
| Primary Cache: | 64 KB I + 64 KB D on chip per core |
| Secondary Cache: | 1 MB I+D on chip per core |
| L3 Cache: | None |
| Other Cache: | None |
| Memory: | 8 GB (8 x 1 GB DDR400) |
| Disk Subsystem: | 250 GB, SATA |
| Other Hardware: | Nodes custom-built by Rackable Systems. The Rackable C1000 chassis is half-depth with 450W, 48 VDC Power Supply. Integrated Gigabit Ethernet for admin/filesystem. |
| Adapter: | Intel 82541PI Gigabit Ethernet controller |
| Number of Adapters: | 1 |
| Slot Type: | integrated on motherboard |
| Data Rate: | 1 Gbps Ethernet |
| Ports Used: | 1 |
| Interconnect Type: | Ethernet |
| Adapter: | QLogic InfiniPath QHT7140 |
| Number of Adapters: | 1 |
| Slot Type: | HTX |
| Data Rate: | InfiniBand 4x SDR |
| Ports Used: | 1 |
| Interconnect Type: | InfiniBand |
| Software | |
|---|---|
| Adapter: | Intel 82541PI Gigabit Ethernet controller |
| Adapter Driver: | Part of Linux kernel modules |
| Adapter Firmware: | None |
| Adapter: | QLogic InfiniPath QHT7140 |
| Adapter Driver: | InfiniPath 2.1 |
| Adapter Firmware: | None |
| Operating System: | ClusterCorp Rocks 4.2.1 (Based on RedHat Enterprise Linux 4.0 Update 4) |
| Local File System: | Linux ext3 |
| Shared File System: | NFS |
| System State: | Multi-User |
| Other Software: | Sun Grid Engine 6.0 |
Node Description: Headnode NFS filesystem
| Hardware | |
|---|---|
| Number of nodes: | 1 |
| Uses of the node: | file server, other |
| Vendor: | Tyan |
| Model: | Thunder K8QSD Pro (S4882) motherboard |
| CPU Name: | AMD Opteron 885 |
| CPU(s) orderable: | 1-4 chips |
| Chips enabled: | 4 |
| Cores enabled: | 8 |
| Cores per chip: | 2 |
| Threads per core: | 1 |
| CPU Characteristics: | -- |
| CPU MHz: | 2600 |
| Primary Cache: | 64 KB I + 64 KB D on chip per core |
| Secondary Cache: | 1 MB I+D on chip per core |
| L3 Cache: | None |
| Other Cache: | None |
| Memory: | 16 GB (16 x 1 GB DDR400 dimms) |
| Disk Subsystem: | 250 GB, SATA, 7200 RPM |
| Other Hardware: | None |
| Adapter: | Broadcom BCM5704C |
| Number of Adapters: | 2 |
| Slot Type: | integrated on motherboard |
| Data Rate: | 1 Gbps Ethernet |
| Ports Used: | 2 |
| Interconnect Type: | Ethernet |
| Software | |
|---|---|
| Adapter: | Broadcom BCM5704C |
| Adapter Driver: | Part of Linux kernel modules |
| Adapter Firmware: | None |
| Operating System: | ClusterCorp Rocks 4.2.1 (Based on RedHat Enterprise Linux 4.0 Update 4) |
| Local File System: | Linux ext3 |
| Shared File System: | NFS |
| System State: | Multi-User |
| Other Software: | Sun Grid Engine 6.0 |
General Notes
"other" purposes of this node: login, compile, job submission and queuing. This node assembled with a 2U chassis and 700 watt ATX 12V Power Supply.
Interconnect Description: QLogic InfiniBand HCAs and switches
| Hardware | |
|---|---|
| Vendor: | QLogic |
| Model: | InfiniPath and Silverstorm |
| Switch Model: | QLogic SilverStorm 9120 Fabric Director |
| Number of Switches: | 1 |
| Number of Ports: | 144 |
| Data Rate: | InfiniBand 4x SDR and InfiniBand 4x DDR |
| Firmware: | 3.4.0.5.2 |
| Topology: | Single switch (star) |
| Primary Use: | MPI traffic |
General Notes
The data rate between InifniPath HCAs and SilverStorm switches is SDR. However, DDR is used for inter-switch links.
Interconnect Description: Broadcom NICs, Force10 switches
| Hardware | |
|---|---|
| Vendor: | Force10 |
| Model: | E300 |
| Switch Model: | Force10 E300 Gig-E switch |
| Number of Switches: | 1 |
| Number of Ports: | 288 |
| Data Rate: | 1 Gbps Ethernet |
| Firmware: | N/A |
| Topology: | Single switch (star) |
| Primary Use: | file system traffic |
Base Compiler Invocation
C benchmarks:
| /usr/bin/mpicc -cc=pathcc |
C++ benchmarks:
| 126.lammps: | /usr/bin/mpicxx -CC=pathCC |
Fortran benchmarks:
| 107.leslie3d: | /usr/bin/mpif90 -f90=pathf90 |
| 113.GemsFDTD: | /usr/bin/mpif90 -f90=pathf90 |
| 115.fds4: | /usr/bin/mpif90 -f90=pathf90 |
| 129.tera_tf: | /usr/bin/mpif90 -f90=pathf90 |
| 132.zeusmp2: | /usr/bin/mpif90 -f90=pathf90 |
| 137.lu: | /usr/bin/mpif90 -f90=pathf90 |
Benchmarks using both Fortran and C (except as noted below):
| /usr/bin/mpicc -cc=pathcc /usr/bin/mpif90 -f90=pathf90 |
Base Portability Flags
| 104.milc: | -DSPEC_MPI_LP64 |
| 121.pop2: | -DSPEC_MPI_DOUBLE_UNDERSCORE -DSPEC_MPI_LP64 |
| 122.tachyon: | -DSPEC_MPI_LP64 |
| 127.wrf2: | -DF2CSTYLE -DSPEC_MPI_DOUBLE_UNDERSCORE -DSPEC_MPI_LINUX -DSPEC_MPI_LP64 |
| 128.GAPgeofem: | -DSPEC_MPI_LP64 |
| 130.socorro: | -fno-second-underscore -DSPEC_MPI_LP64 |
Base Optimization Flags
C benchmarks:
| -march=opteron -Ofast -OPT:malloc_alg=1 |
C++ benchmarks:
| 126.lammps: | -march=opteron -O3 -OPT:Ofast -CG:local_fwd_sched=on |
Fortran benchmarks:
| 107.leslie3d: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 113.GemsFDTD: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 115.fds4: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 129.tera_tf: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 132.zeusmp2: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 137.lu: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
Benchmarks using both Fortran and C:
| 121.pop2: | -march=opteron -Ofast -OPT:malloc_alg=1 -O3 -OPT:Ofast -LANG:copyinout=off |
| 127.wrf2: | Same as 121.pop2 |
| 128.GAPgeofem: | Same as 121.pop2 |
| 130.socorro: | Same as 121.pop2 |
Base Other Flags
C benchmarks:
| -IPA:max_jobs=4 |
C++ benchmarks:
| 126.lammps: | -IPA:max_jobs=4 |
Fortran benchmarks:
| 107.leslie3d: | -IPA:max_jobs=4 |
| 113.GemsFDTD: | -IPA:max_jobs=4 |
| 115.fds4: | -IPA:max_jobs=4 |
| 129.tera_tf: | -IPA:max_jobs=4 |
| 132.zeusmp2: | -IPA:max_jobs=4 |
| 137.lu: | -IPA:max_jobs=4 |
Benchmarks using both Fortran and C (except as noted below):
| -IPA:max_jobs=4 |