| MPI2007 license: | 0018 | Test date: | May-2007 |
|---|---|---|---|
| Test sponsor: | QLogic Corporation | Hardware Availability: | Nov-2006 |
| Tested by: | QLogic Performance Engineering | Software Availability: | Jul-2007 |
Results Table
| Benchmark | Base | Peak | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | |
| Results appear in the order in which they were run. Bold underlined text indicates a median measurement. | ||||||||||||||
| 104.milc | 64 | 299 | 5.24 | 299 | 5.23 | 300 | 5.21 | |||||||
| 107.leslie3d | 64 | 1049 | 4.98 | 1253 | 4.17 | 1239 | 4.21 | |||||||
| 113.GemsFDTD | 64 | 736 | 8.57 | 729 | 8.66 | 736 | 8.57 | |||||||
| 115.fds4 | 64 | 397 | 4.91 | 397 | 4.91 | 396 | 4.93 | |||||||
| 121.pop2 | 64 | 616 | 6.71 | 647 | 6.38 | 622 | 6.64 | |||||||
| 122.tachyon | 64 | 582 | 4.81 | 570 | 4.91 | 586 | 4.77 | |||||||
| 126.lammps | 64 | 500 | 5.84 | 500 | 5.83 | 500 | 5.83 | |||||||
| 127.wrf2 | 64 | 884 | 8.82 | 882 | 8.84 | 884 | 8.82 | |||||||
| 128.GAPgeofem | 64 | 275 | 7.52 | 275 | 7.52 | 274 | 7.54 | |||||||
| 129.tera_tf | 64 | 516 | 5.37 | 515 | 5.38 | 517 | 5.36 | |||||||
| 130.socorro | 64 | 491 | 7.77 | 488 | 7.83 | 489 | 7.80 | |||||||
| 132.zeusmp2 | 64 | 513 | 6.05 | 510 | 6.08 | 512 | 6.07 | |||||||
| 137.lu | 64 | 711 | 5.17 | 711 | 5.17 | 708 | 5.19 | |||||||
| Software Summary | |
|---|---|
| C Compiler: | QLogic PathScale C Compiler 3.0 |
| C++ Compiler: | QLogic PathScale C++ Compiler 3.0 |
| Fortran Compiler: | QLogic PathScale Fortran Compiler 3.0 |
| Base Pointers: | 64-bit |
| Peak Pointers: | 64-bit |
| MPI Library: | QLogic InfiniPath MPI 2.1 |
| Other MPI Info: | None |
| Pre-processors: | No |
| Other Software: | None |
Node Description: Rackable, IWILL, AMD
| Hardware | |
|---|---|
| Number of nodes: | 16 |
| Uses of the node: | compute, head |
| Vendor: | Rackable Systems, IWILL, AMD |
| Model: | Rackable Systems C1000 chassis, IWILL DK8-HTX motherboard |
| CPU Name: | AMD Opteron 290 |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 4 |
| Cores per chip: | 2 |
| Threads per core: | 1 |
| CPU Characteristics: | -- |
| CPU MHz: | 2800 |
| Primary Cache: | 64 KB I + 64 KB D on chip per core |
| Secondary Cache: | 1 MB I+D on chip per core |
| L3 Cache: | None |
| Other Cache: | None |
| Memory: | 8 GB (8 x 1 GB DDR400) |
| Disk Subsystem: | 250 GB, SATA |
| Other Hardware: | Nodes custom-built by Rackable Systems. The Rackable C1000 chassis is half-depth with 450W, 48 VDC Power Supply. Integrated Gigabit Ethernet for admin/filesystem. |
| Adapter: | Intel 82541PI Gigabit Ethernet controller |
| Number of Adapters: | 1 |
| Slot Type: | integrated on motherboard |
| Data Rate: | 1 Gbps Ethernet |
| Ports Used: | 1 |
| Interconnect Type: | Ethernet |
| Adapter: | QLogic InfiniPath QHT7140 |
| Number of Adapters: | 1 |
| Slot Type: | HTX |
| Data Rate: | InfiniBand 4x SDR |
| Ports Used: | 1 |
| Interconnect Type: | InfiniBand |
| Software | |
|---|---|
| Adapter: | Intel 82541PI Gigabit Ethernet controller |
| Adapter Driver: | Part of Linux kernel modules |
| Adapter Firmware: | None |
| Adapter: | QLogic InfiniPath QHT7140 |
| Adapter Driver: | InfiniPath 2.1 |
| Adapter Firmware: | None |
| Operating System: | ClusterCorp Rocks 4.2.1 (Based on RedHat Enterprise Linux 4.0 Update 4) |
| Local File System: | Linux ext3 |
| Shared File System: | NFS |
| System State: | Multi-User |
| Other Software: | Sun Grid Engine 6.0 |
Node Description: Headnode NFS filesystem
| Hardware | |
|---|---|
| Number of nodes: | 1 |
| Uses of the node: | file server, other |
| Vendor: | Tyan |
| Model: | Thunder K8QSD Pro (S4882) motherboard |
| CPU Name: | AMD Opteron 885 |
| CPU(s) orderable: | 1-4 chips |
| Chips enabled: | 4 |
| Cores enabled: | 8 |
| Cores per chip: | 2 |
| Threads per core: | 1 |
| CPU Characteristics: | -- |
| CPU MHz: | 2600 |
| Primary Cache: | 64 KB I + 64 KB D on chip per core |
| Secondary Cache: | 1 MB I+D on chip per core |
| L3 Cache: | None |
| Other Cache: | None |
| Memory: | 16 GB (16 x 1 GB DDR400 dimms) |
| Disk Subsystem: | 250 GB, SATA, 7200 RPM |
| Other Hardware: | None |
| Adapter: | Broadcom BCM5704C |
| Number of Adapters: | 2 |
| Slot Type: | integrated on motherboard |
| Data Rate: | 1 Gbps Ethernet |
| Ports Used: | 2 |
| Interconnect Type: | Ethernet |
| Software | |
|---|---|
| Adapter: | Broadcom BCM5704C |
| Adapter Driver: | Part of Linux kernel modules |
| Adapter Firmware: | None |
| Operating System: | ClusterCorp Rocks 4.2.1 (Based on RedHat Enterprise Linux 4.0 Update 4) |
| Local File System: | Linux ext3 |
| Shared File System: | NFS |
| System State: | Multi-User |
| Other Software: | Sun Grid Engine 6.0 |
General Notes
"other" purposes of this node: login, compile, job submission and queuing. This node assembled with a 2U chassis and 700 watt ATX 12V Power Supply.
Interconnect Description: QLogic InfiniBand HCAs and switches
| Hardware | |
|---|---|
| Vendor: | QLogic |
| Model: | InfiniPath and Silverstorm |
| Switch Model: | QLogic SilverStorm 9120 Fabric Director |
| Number of Switches: | 1 |
| Number of Ports: | 144 |
| Data Rate: | InfiniBand 4x SDR and InfiniBand 4x DDR |
| Firmware: | 3.4.0.5.2 |
| Topology: | Single switch (star) |
| Primary Use: | MPI traffic |
General Notes
The data rate between InifniPath HCAs and SilverStorm switches is SDR. However, DDR is used for inter-switch links.
Interconnect Description: Broadcom NICs, Force10 switches
| Hardware | |
|---|---|
| Vendor: | Force10 |
| Model: | E300 |
| Switch Model: | Force10 E300 Gig-E switch |
| Number of Switches: | 1 |
| Number of Ports: | 288 |
| Data Rate: | 1 Gbps Ethernet |
| Firmware: | N/A |
| Topology: | Single switch (star) |
| Primary Use: | file system traffic |
Base Compiler Invocation
C benchmarks:
| /usr/bin/mpicc -cc=pathcc |
C++ benchmarks:
| 126.lammps: | /usr/bin/mpicxx -CC=pathCC |
Fortran benchmarks:
| 107.leslie3d: | /usr/bin/mpif90 -f90=pathf90 |
| 113.GemsFDTD: | /usr/bin/mpif90 -f90=pathf90 |
| 115.fds4: | /usr/bin/mpif90 -f90=pathf90 |
| 129.tera_tf: | /usr/bin/mpif90 -f90=pathf90 |
| 132.zeusmp2: | /usr/bin/mpif90 -f90=pathf90 |
| 137.lu: | /usr/bin/mpif90 -f90=pathf90 |
Benchmarks using both Fortran and C (except as noted below):
| /usr/bin/mpicc -cc=pathcc /usr/bin/mpif90 -f90=pathf90 |
Base Portability Flags
| 104.milc: | -DSPEC_MPI_LP64 |
| 121.pop2: | -DSPEC_MPI_DOUBLE_UNDERSCORE -DSPEC_MPI_LP64 |
| 122.tachyon: | -DSPEC_MPI_LP64 |
| 127.wrf2: | -DF2CSTYLE -DSPEC_MPI_DOUBLE_UNDERSCORE -DSPEC_MPI_LINUX -DSPEC_MPI_LP64 |
| 128.GAPgeofem: | -DSPEC_MPI_LP64 |
| 130.socorro: | -fno-second-underscore -DSPEC_MPI_LP64 |
Base Optimization Flags
C benchmarks:
| -march=opteron -Ofast -OPT:malloc_alg=1 |
C++ benchmarks:
| 126.lammps: | -march=opteron -O3 -OPT:Ofast -CG:local_fwd_sched=on |
Fortran benchmarks:
| 107.leslie3d: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 113.GemsFDTD: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 115.fds4: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 129.tera_tf: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 132.zeusmp2: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
| 137.lu: | -march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off |
Benchmarks using both Fortran and C:
| 121.pop2: | -march=opteron -Ofast -OPT:malloc_alg=1 -O3 -OPT:Ofast -LANG:copyinout=off |
| 127.wrf2: | Same as 121.pop2 |
| 128.GAPgeofem: | Same as 121.pop2 |
| 130.socorro: | Same as 121.pop2 |
Base Other Flags
C benchmarks:
| -IPA:max_jobs=4 |
C++ benchmarks:
| 126.lammps: | -IPA:max_jobs=4 |
Fortran benchmarks:
| 107.leslie3d: | -IPA:max_jobs=4 |
| 113.GemsFDTD: | -IPA:max_jobs=4 |
| 115.fds4: | -IPA:max_jobs=4 |
| 129.tera_tf: | -IPA:max_jobs=4 |
| 132.zeusmp2: | -IPA:max_jobs=4 |
| 137.lu: | -IPA:max_jobs=4 |
Benchmarks using both Fortran and C (except as noted below):
| -IPA:max_jobs=4 |