SPEChpc(TM) 2021 Tiny Result Lenovo Global Technology ThinkSystem SR655 V3 (AMD EPYC 9654P, Nvidia H100-PCIE-80G) hpc2021 License: 28 Test date: Jan-2023 Test sponsor: Lenovo Global Technology Hardware availability: Feb-2023 Tested by: Lenovo Global Technology Software availability: Feb-2023 Base Base Thrds Base Base Peak Peak Thrds Peak Peak Benchmarks Model Ranks pr Rnk Run Time Ratio Model Ranks pr Rnk Run Time Ratio -------------- ------ ------ ------ --------- --------- ------ ------ ------ --------- --------- 505.lbm_t ACC 2 1 33.4 67.4 S ACC 2 1 31.3 71.8 S 505.lbm_t ACC 2 1 33.9 66.3 S ACC 2 1 31.4 71.7 S 505.lbm_t ACC 2 1 33.9 66.4 * ACC 2 1 31.4 71.7 * 513.soma_t ACC 2 1 57.2 64.7 * ACC 2 1 57.2 64.7 * 513.soma_t ACC 2 1 57.2 64.7 S ACC 2 1 57.2 64.7 S 513.soma_t ACC 2 1 57.0 64.9 S ACC 2 1 57.0 64.9 S 518.tealeaf_t ACC 2 1 62.0 26.6 S ACC 2 1 61.6 26.8 S 518.tealeaf_t ACC 2 1 62.0 26.6 * ACC 2 1 61.6 26.8 * 518.tealeaf_t ACC 2 1 62.0 26.6 S ACC 2 1 61.7 26.8 S 519.clvleaf_t ACC 2 1 46.8 35.3 * ACC 2 1 46.8 35.3 * 519.clvleaf_t ACC 2 1 46.9 35.2 S ACC 2 1 46.9 35.2 S 519.clvleaf_t ACC 2 1 46.8 35.3 S ACC 2 1 46.8 35.3 S 521.miniswp_t ACC 2 1 68.0 23.5 S ACC 2 1 68.0 23.5 S 521.miniswp_t ACC 2 1 68.1 23.5 S ACC 2 1 68.1 23.5 S 521.miniswp_t ACC 2 1 68.1 23.5 * ACC 2 1 68.1 23.5 * 528.pot3d_t ACC 2 1 71.7 29.6 S ACC 2 1 71.7 29.6 S 528.pot3d_t ACC 2 1 71.7 29.6 S ACC 2 1 71.7 29.6 S 528.pot3d_t ACC 2 1 71.7 29.6 * ACC 2 1 71.7 29.6 * 532.sph_exa_t ACC 2 1 173 11.3 * ACC 2 1 169 11.6 S 532.sph_exa_t ACC 2 1 173 11.2 S ACC 2 1 167 11.6 * 532.sph_exa_t ACC 2 1 172 11.3 S ACC 2 1 167 11.7 S 534.hpgmgfv_t ACC 2 1 75.1 15.6 S ACC 2 1 74.5 15.8 S 534.hpgmgfv_t ACC 2 1 75.2 15.6 S ACC 2 1 74.5 15.8 * 534.hpgmgfv_t ACC 2 1 75.1 15.6 * ACC 2 1 74.5 15.8 S 535.weather_t ACC 2 1 43.6 74.0 S ACC 2 1 43.0 75.0 S 535.weather_t ACC 2 1 43.5 74.1 * ACC 2 1 43.0 75.0 * 535.weather_t ACC 2 1 43.5 74.1 S ACC 2 1 43.0 75.0 S ============================================================================================================ 505.lbm_t ACC 2 1 33.9 66.4 * ACC 2 1 31.4 71.7 * 513.soma_t ACC 2 1 57.2 64.7 * ACC 2 1 57.2 64.7 * 518.tealeaf_t ACC 2 1 62.0 26.6 * ACC 2 1 61.6 26.8 * 519.clvleaf_t ACC 2 1 46.8 35.3 * ACC 2 1 46.8 35.3 * 521.miniswp_t ACC 2 1 68.1 23.5 * ACC 2 1 68.1 23.5 * 528.pot3d_t ACC 2 1 71.7 29.6 * ACC 2 1 71.7 29.6 * 532.sph_exa_t ACC 2 1 173 11.3 * ACC 2 1 167 11.6 * 534.hpgmgfv_t ACC 2 1 75.1 15.6 * ACC 2 1 74.5 15.8 * 535.weather_t ACC 2 1 43.5 74.1 * ACC 2 1 43.0 75.0 * SPEChpc 2021_tny_base 32.1 SPEChpc 2021_tny_peak 32.6 BENCHMARK DETAILS ----------------- Type of System: Homogeneous Cluster Compute Nodes Used: 1 Total Chips: 1 Total Cores: 96 Total Threads: 96 Total Memory: 384 GB Max. Peak Threads: 1 Compiler: Nvidia HPC SDK 22.11 MPI Library: Open MPI 4.0.5 Other MPI Info: None Base Parallel Model: ACC Base Ranks Run: 2 Base Threads Run: 1 Peak Parallel Models: ACC Minimum Peak Ranks: 2 Maximum Peak Ranks: 2 Max. Peak Threads: 1 Min. Peak Threads: 1 Node Description: ThinkSystem SR655 V3 ====================================== HARDWARE -------- Number of nodes: 1 Uses of the node: compute Vendor: Lenovo Global Technology Model: ThinkSystem SR655 V3 CPU Name: AMD EPYC 9654P CPU(s) orderable: 1 chips Chips enabled: 1 Cores enabled: 96 Cores per chip: 96 Threads per core: 1 CPU Characteristics: Intel Turbo Boost Technology up to 3.7 GHz CPU MHz: 2400 Primary Cache: 32 KB I + 32 KB D on chip per core Secondary Cache: 1 MB I+D on chip per core L3 Cache: 384 MB I+D on chip per chip Other Cache: None Memory: 384 GB (12 x 16 GB 2Rx4 PC5-4800B-R) Disk Subsystem: 1x ThinkSystem 2.5" 5300 480GB SSD Other Hardware: None Accel Count: 8 Accel Model: Tesla H100 PCIe 80GB Accel Vendor: Nvidia Corporation Accel Type: GPU Accel Connection: PCIe Gen5 x16 Accel ECC enabled: Yes Accel Description: Nvidia Tesla H100 PCIe 80GB Adapter: Mellanox ConnectX-6 HDR Number of Adapters: 1 Slot Type: PCI-Express 5.0 x16 Data Rate: 200 Gb/s Ports Used: 1 Interconnect Type: Nvidia Mellanox ConnectX-6 HDR SOFTWARE -------- Accelerator Driver: 525.60.13 Adapter: Mellanox ConnectX-6 HDR Adapter Driver: 5.2-1.0.4 Adapter Firmware: 20.28.1002 Operating System: Red Hat Enterprise Linux Server release 9, Kernel 5.14.0-70.22.1.el9_0.x86_64 Local File System: xfs Shared File System: XFS System State: Multi-user, run level 3 Other Software: None Submit Notes ------------ Indiviual Ranks were bound to the CPU cores on the same NUMA node as the GPU using 'numactl' within the following "bind2.pl" perl script: ---- Start bind2.pl ------ my %bind; $bind{0} = "1-3"; $bind{1} = "144-146"; $bind{2} = "8-10"; $bind{3} = "11-14"; $bind{4} = "41-43"; $bind{5} = "44-47"; $bind{6} = "61-63"; $bind{7} = "64-67"; my $rank = $ENV{OMPI_COMM_WORLD_LOCAL_RANK}; my $cmd = "taskset -c $bind{$rank} "; while (my $arg = shift) { $cmd .= "$arg "; } my $rc = system($cmd); exit($rc); ---- End bind.pl ------ The config file option 'submit' was used. submit = mpirun --allow-run-as-root -x UCX_MEMTYPE_CACHE=n -host localhost:2 -np $ranks perl $[top]/bind2.pl $command General Notes ------------- Environment variables set by runhpc before the start of the run: UCX_MEMTYPE_CACHE = "n" UCX_TLS = "self,shm,cuda_copy" Compiler Version Notes ---------------------- ============================================================================== CC 505.lbm_t(base, peak) 513.soma_t(base, peak) 518.tealeaf_t(base, peak) 521.miniswp_t(base, peak) 534.hpgmgfv_t(base, peak) ------------------------------------------------------------------------------ nvc 22.11-0 64-bit target on x86-64 Linux -tp zen3 NVIDIA Compilers and Tools Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== CXXC 532.sph_exa_t(base, peak) ------------------------------------------------------------------------------ nvc++ 22.11-0 64-bit target on x86-64 Linux -tp zen3 NVIDIA Compilers and Tools Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== FC 519.clvleaf_t(base, peak) 528.pot3d_t(base, peak) 535.weather_t(base, peak) ------------------------------------------------------------------------------ nvfortran 22.11-0 64-bit target on x86-64 Linux -tp zen3 NVIDIA Compilers and Tools Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ Base Compiler Invocation ------------------------ C benchmarks: mpicc C++ benchmarks: mpicxx Fortran benchmarks: mpif90 Base Portability Flags ---------------------- 505.lbm_t: -DSPEC_OPENACC_NO_SELF 532.sph_exa_t: --c++17 Base Optimization Flags ----------------------- C benchmarks: -fast -acc=gpu -Mfprelaxed -Mnouniform -Mstack_arrays -DSPEC_ACCEL_AWARE_MPI C++ benchmarks: -fast -acc=gpu -Mfprelaxed -Mnouniform -Mstack_arrays -DSPEC_ACCEL_AWARE_MPI Fortran benchmarks: -DSPEC_ACCEL_AWARE_MPI -fast -acc=gpu -Mfprelaxed -Mnouniform -Mstack_arrays Base Other Flags ---------------- C benchmarks (except as noted below): -Ispecmpitime -w 521.miniswp_t: -Ispecmpitime/ -w 534.hpgmgfv_t: -Ispecmpitime -w C++ benchmarks: -Ispecmpitime -w Fortran benchmarks (except as noted below): -w 519.clvleaf_t: -Ispecmpitime -w Peak Compiler Invocation ------------------------ C benchmarks: mpicc C++ benchmarks: mpicxx Fortran benchmarks: mpif90 Peak Portability Flags ---------------------- 505.lbm_t: -DSPEC_OPENACC_NO_SELF Peak Optimization Flags ----------------------- C benchmarks: 505.lbm_t: -fast -acc=gpu -O3 -Mfprelaxed -Mnouniform -DSPEC_ACCEL_AWARE_MPI 513.soma_t: basepeak = yes 518.tealeaf_t: -fast -acc=gpu -Msafeptr -DSPEC_ACCEL_AWARE_MPI 521.miniswp_t: basepeak = yes 534.hpgmgfv_t: -fast -acc=gpu -static-nvidia -DSPEC_ACCEL_AWARE_MPI C++ benchmarks: -fast -acc=gpu -O3 -Mfprelaxed -Mnouniform -Mstack_arrays -static-nvidia -DSPEC_ACCEL_AWARE_MPI Fortran benchmarks: 519.clvleaf_t: basepeak = yes 528.pot3d_t: basepeak = yes 535.weather_t: -DSPEC_ACCEL_AWARE_MPI -fast -acc=gpu -O3 -Mfprelaxed -Mnouniform -Mstack_arrays -static-nvidia Peak Other Flags ---------------- C benchmarks (except as noted below): -Ispecmpitime -w 521.miniswp_t: -Ispecmpitime/ -w 534.hpgmgfv_t: -Ispecmpitime -w C++ benchmarks: -Ispecmpitime -w Fortran benchmarks (except as noted below): -w 519.clvleaf_t: -Ispecmpitime -w The flags file that was used to format this result can be browsed at http://www.spec.org/hpc2021/flags/nv2021_flags_v1.0.3.html You can also download the XML flags source by saving the following link: http://www.spec.org/hpc2021/flags/nv2021_flags_v1.0.3.xml SPEChpc is a trademark of the Standard Performance Evaluation Corporation. All other brand and product names appearing in this result are trademarks or registered trademarks of their respective holders. ------------------------------------------------------------------------------------------------------------- For questions about this result, please contact the tester. For other inquiries, please contact info@spec.org. Copyright 2021-2023 Standard Performance Evaluation Corporation Tested with SPEChpc2021 v1.1.7 on 2022-12-26 01:58:23-0500. Report generated on 2023-02-22 12:25:53 by hpc2021 ASCII formatter v1.0.3. Originally published on 2023-02-22.