| MPI2007 license: | 28 | Test date: | Mar-2021 |
|---|---|---|---|
| Test sponsor: | Lenovo Global Technology | Hardware Availability: | Mar-2021 |
| Tested by: | Lenovo Global Technology | Software Availability: | Mar-2021 |
Results Table
| Benchmark | Base | Peak | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | Ranks | Seconds | Ratio | Seconds | Ratio | Seconds | Ratio | |
| Results appear in the order in which they were run. Bold underlined text indicates a median measurement. | ||||||||||||||
| 104.milc | 512 | 22.3 | 70.0 | 15.4 | 101 | 14.9 | 105 | 512 | 22.3 | 70.0 | 15.4 | 101 | 14.9 | 105 |
| 107.leslie3d | 512 | 69.2 | 75.4 | 68.7 | 76.0 | 68.8 | 75.9 | 512 | 69.2 | 75.4 | 68.7 | 76.0 | 68.8 | 75.9 |
| 113.GemsFDTD | 512 | 243 | 26.0 | 242 | 26.1 | 243 | 26.0 | 128 | 220 | 28.7 | 220 | 28.6 | 221 | 28.5 |
| 115.fds4 | 512 | 23.7 | 82.2 | 22.2 | 87.9 | 23.2 | 84.2 | 512 | 23.7 | 82.2 | 22.2 | 87.9 | 23.2 | 84.2 |
| 121.pop2 | 512 | 111 | 37.2 | 111 | 37.1 | 114 | 36.3 | 512 | 111 | 37.2 | 111 | 37.1 | 114 | 36.3 |
| 122.tachyon | 512 | 23.7 | 118 | 23.5 | 119 | 24.9 | 112 | 512 | 23.7 | 118 | 23.5 | 119 | 24.9 | 112 |
| 126.lammps | 512 | 93.8 | 31.1 | 96.2 | 30.3 | 95.0 | 30.7 | 512 | 93.8 | 31.1 | 96.2 | 30.3 | 95.0 | 30.7 |
| 127.wrf2 | 512 | 43.5 | 179 | 42.8 | 182 | 44.3 | 176 | 512 | 43.5 | 179 | 42.8 | 182 | 44.3 | 176 |
| 128.GAPgeofem | 512 | 17.0 | 121 | 17.1 | 121 | 18.4 | 112 | 512 | 17.0 | 121 | 17.1 | 121 | 18.4 | 112 |
| 129.tera_tf | 512 | 39.9 | 69.4 | 39.5 | 70.0 | 38.7 | 71.5 | 512 | 39.9 | 69.4 | 39.5 | 70.0 | 38.7 | 71.5 |
| 130.socorro | 512 | 78.8 | 48.5 | 79.0 | 48.3 | 79.7 | 47.9 | 512 | 78.8 | 48.5 | 79.0 | 48.3 | 79.7 | 47.9 |
| 132.zeusmp2 | 512 | 37.6 | 82.5 | 38.9 | 79.8 | 37.6 | 82.5 | 512 | 37.6 | 82.5 | 38.9 | 79.8 | 37.6 | 82.5 |
| 137.lu | 512 | 26.4 | 139 | 25.8 | 143 | 27.6 | 133 | 512 | 26.4 | 139 | 25.8 | 143 | 27.6 | 133 |
| Hardware Summary | |
|---|---|
| Type of System: | Homogeneous |
| Compute Node: | ThinkSystem SR665 |
| Interconnect: | Mellanox ConnectX-6 HDR |
| File Server Node: | NFS |
| Total Compute Nodes: | 4 |
| Total Chips: | 8 |
| Total Cores: | 512 |
| Total Threads: | 512 |
| Total Memory: | 4 TB |
| Base Ranks Run: | 512 |
| Minimum Peak Ranks: | 128 |
| Maximum Peak Ranks: | 512 |
| Software Summary | |
|---|---|
| C Compiler: | AMD Optimizing C Compiler for Linux Version 2.3.0 Build 2020_11_10 |
| C++ Compiler: | AMD Optimizing C++ Compiler for Linux Version 2.3.0 Build 2020_11_10 |
| Fortran Compiler: | AMD Optimizing Fortran Compiler for Linux Version 2.3.0 Build 2020_11_10 |
| Base Pointers: | 64-bit |
| Peak Pointers: | Not Applicable |
| MPI Library: | Open MPI Library Version 4.1.0 |
| Other MPI Info: | None |
| Pre-processors: | No |
| Other Software: | None |
Node Description: ThinkSystem SR665
| Hardware | |
|---|---|
| Number of nodes: | 4 |
| Uses of the node: | compute |
| Vendor: | Lenovo Global Technology |
| Model: | SR665 |
| CPU Name: | AMD EPYC 7763 |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 128 |
| Cores per chip: | 64 |
| Threads per core: | 1 |
| CPU Characteristics: | None |
| CPU MHz: | 2450 |
| Primary Cache: | 32 KB I + 32 KB D on chip per core |
| Secondary Cache: | 512 KB I+D on chip per core |
| L3 Cache: | 256 MB I+D on chip per chip 32 MB shared / 8 cores |
| Other Cache: | None |
| Memory: | 1 TB (16 x 64 GB 2Rx4 PC4-3200AA-R) |
| Disk Subsystem: | 1 x 480 GB SATA 2.5" SSD |
| Other Hardware: | None |
| Adapter: | Mellanox ConnectX-6 HDR Infiniband |
| Number of Adapters: | 1 |
| Slot Type: | PCI-Express 4.0 x16 |
| Data Rate: | 200 Gbs/s |
| Ports Used: | 1 |
| Interconnect Type: | Mellanox ConnectX-6 HDR Infiniband Adapter |
| Software | |
|---|---|
| Adapter: | Mellanox ConnectX-6 HDR Infiniband |
| Adapter Driver: | 5.2-1.0.4 |
| Adapter Firmware: | 20.25.2006 |
| Operating System: | Red Hat Enterprise Linux Server release 8.3 4.18.0-240.el8.x86_64 |
| Local File System: | xfs |
| Shared File System: | None |
| System State: | Multi-user, run level 3 |
| Other Software: | None |
Node Description: NFS
| Hardware | |
|---|---|
| Number of nodes: | 1 |
| Uses of the node: | Fileserver |
| Vendor: | Lenovo Global Technology |
| Model: | ThinkSystem SR665 |
| CPU Name: | AMD EPYC 7763 CPU |
| CPU(s) orderable: | 1-2 chips |
| Chips enabled: | 2 |
| Cores enabled: | 128 |
| Cores per chip: | 64 |
| Threads per core: | 1 |
| CPU Characteristics: | None |
| CPU MHz: | 2450 |
| Primary Cache: | 32 KB I + 32 KB D on chip per core |
| Secondary Cache: | 512 KB I+D on chip per core |
| L3 Cache: | 256 MB I+D on chip per chip 32 MB shared / 8 cores |
| Other Cache: | None |
| Memory: | 1 TB (16 x 64 GB 2Rx4 PC4-3200AA-R) |
| Disk Subsystem: | 1 x 480 GB SATA 2.5" SSD |
| Other Hardware: | None |
| Adapter: | Mellanox ConnectX-6 HDR Infiniband |
| Number of Adapters: | 1 |
| Slot Type: | PCI-Express 4.0 x16 |
| Data Rate: | 200 Gb/s |
| Ports Used: | 1 |
| Interconnect Type: | Mellanox ConnectX-6 HDR Infiniband |
| Software | |
|---|---|
| Adapter: | Mellanox ConnectX-6 HDR Infiniband |
| Adapter Driver: | 5.2-1.0.4 |
| Adapter Firmware: | 20.25.2006 |
| Operating System: | Red Hat Enterprise Linux Server release 8.3 |
| Local File System: | None |
| Shared File System: | NFS |
| System State: | Multi-User, run level 3 |
| Other Software: | None |
Interconnect Description: Mellanox ConnectX-6 HDR
| Hardware | |
|---|---|
| Vendor: | Mellanox |
| Model: | Infiniband HDR 200Gb/s Switch |
| Switch Model: | QM8700 Series |
| Number of Switches: | 1 |
| Number of Ports: | 40 |
| Data Rate: | 200 Gb/s |
| Firmware: | 3.9.0606 |
| Topology: | Mesh |
| Primary Use: | MPI Traffic |
Submit Notes
The config file option 'submit' was used.
General Notes
MPI startup command: mpiexec command was used to start MPI jobs. RAM configuration: Compute nodes have 1 x 64 GB RDIMM on each memory channel. Add "idle=poll" into grub BIOS settings: Operating Mode : Maximum Performance Mode Hyper-Threading Technology (SMT): Enabled NPS4 Yes: The test sponsor attests, as of date of publication, that CVE-2017-5754 (Meltdown) is mitigated in the system as tested and documented. Yes: The test sponsor attests, as of date of publication, that CVE-2017-5753 (Spectre variant 1) is mitigated in the system as tested and documented. Yes: The test sponsor attests, as of date of publication, that CVE-2017-5715 (Spectre variant 2) is mitigated in the system as tested and documented.
Compiler Invocation
C benchmarks:
| mpicc |
C++ benchmarks:
| 126.lammps: | mpicxx |
Fortran benchmarks:
| mpifort |
Benchmarks using both Fortran and C:
| mpicc mpifort |
Portability Flags
| 121.pop2: | -DSPEC_MPI_CASE_FLAG |
| 126.lammps: | -DMPICH_IGNORE_CXX_SEEK |
| 127.wrf2: | -DSPEC_MPI_CASE_FLAG -DSPEC_MPI_LINUX -Wno-return-type |
Base Optimization Flags
C benchmarks:
| -Ofast -flto -ffast-math -march=znver2 -mno-avx2 -L/home/amd-libm/lib -lamdlibm |
C++ benchmarks:
| 126.lammps: | -Ofast -flto -ffast-math -march=znver2 -mno-avx2 -L/home/amd-libm/lib -lamdlibm |
Fortran benchmarks:
| -Ofast -flto -ffast-math -march=znver2 -mno-avx2 -funroll-loops -L/home/amd-libm/lib -lamdlibm |
Benchmarks using both Fortran and C:
| -Ofast -flto -ffast-math -march=znver2 -mno-avx2 -funroll-loops -L/home/amd-libm/lib -lamdlibm |
Peak Optimization Flags
C benchmarks:
| 104.milc: | basepeak = yes |
| 122.tachyon: | basepeak = yes |
C++ benchmarks:
| 126.lammps: | basepeak = yes |
Fortran benchmarks:
| 107.leslie3d: | basepeak = yes |
| 113.GemsFDTD: | -Ofast -flto -ffast-math -march=znver2 -mno-avx2 -funroll-loops -L/home/amd-libm/lib -lamdlibm |
| 129.tera_tf: | basepeak = yes |
| 137.lu: | basepeak = yes |
Benchmarks using both Fortran and C:
| 115.fds4: | basepeak = yes |
| 121.pop2: | basepeak = yes |
| 127.wrf2: | basepeak = yes |
| 128.GAPgeofem: | basepeak = yes |
| 130.socorro: | basepeak = yes |
| 132.zeusmp2: | basepeak = yes |