CPU2017 Flag Description
New H3C Technologies Co., Ltd. H3C UniServer R3950 G6 (AMD EPYC 9754)

Compilers: AMD Optimizing C/C++ Compiler Suite

Implicitly Included Flags

This section contains descriptions of flags that were included implicitly by other flags, but which do not have a permanent home at SPEC.

• • -O2
  • 

  • Moderate level of optimization which enables most optimizations. This is the default when no "-O" option is specified, or if no value is specified (i.e. "-O").

    If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

  • Includes:
    • -O1
• • -O1
  • 

  • Somewhere between -O0 and -O2.

    If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

Commands and Options Used to Submit Benchmark Runs

Using numactl to bind processes and memory to cores

For multi-copy runs or single copy runs on systems with multiple sockets, it is advantageous to bind a process to a particular core. Otherwise, the OS may arbitrarily move your process from one core to another. This can affect performance. To help, SPEC allows the use of a "submit" command where users can specify a utility to use to bind processes. We have found the utility 'numactl' to be the best choice.

numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a command and inherited by all of its children. The numactl flag "--physcpubind" specifies which core(s) to bind the process. "-l" instructs numactl to keep a process's memory on the local node while "-m" specifies which node(s) to place a process's memory. For full details on using numactl, please refer to your Linux documentation, 'man numactl'

Note that some older versions of numactl incorrectly interpret application arguments as its own. For example, with the command "numactl --physcpubind=0 -l a.out -m a", numactl will interpret a.out's "-m" option as its own "-m" option. To work around this problem, we put the command to be run in a shell script and then run the shell script using numactl. For example: "echo 'a.out -m a' > run.sh ; numactl --physcpubind=0 bash run.sh"

Shell, Environment, and Other Software Settings

numactl --interleave=all runcpu

numactl --interleave=all runcpu executes the SPEC CPU command runcpu so that memory is consumed across NUMA nodes rather than consumed from a single node. This helps prevent local node out-of-memory conditions which can occur when runcpu is executed without interleaving. For full details on using numactl, please refer to your Linux documentation, 'man numactl'

Transparent Huge Pages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. It is designed to hide much of the complexity in using huge pages from system administrators and developers. Huge pages increase the memory page size from 4 kilobytes to 2 megabytes. This provides significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents huge pages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.

THP usage is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/enabled. Possible values:

• never: entirely disable THP usage.
• madvise: enable THP usage only inside regions marked MADV_HUGEPAGE using madvise(3).
• always: enable THP usage system-wide. This is the default.

The SPEC CPU benchmark codes themselves never explicitly request huge pages, as the mechanism to do that is OS-specific and can change over time. Libraries such as amdalloc which are used by the benchmarks may explicitly request huge pages, and use of such libraries can make the "madvise" setting relevant and useful.

When no huge pages are immediately available and one is requested, how the system handles the request for THP creation is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/defrag. Possible values:

• never: if no THP are available to satisfy a request, do not attempt to make any.
• defer: an allocation requesting THP when none are available gets normal pages while requesting THP creation in the background.
• defer+madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3); for all other regions it's like "defer".
• madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3). This is the default.
• always: an allocation requesting THP when none are available will stall until some are made.

An application that "always" requests THP often can benefit from waiting for an allocation until those huge pages can be assembled.
For more information see the Linux transparent hugepage documentation.

ulimit -s <n>

Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.

ulimit -l <n>

Sets the maximum size of memory that may be locked into physical memory.

powersave -f (on SuSE)

Makes the powersave daemon set the CPUs to the highest supported frequency.

/etc/init.d/cpuspeed stop (on Red Hat)

Disables the cpu frequency scaling program in order to set the CPUs to the highest supported frequency.

LD_LIBRARY_PATH

An environment variable that indicates the location in the filesystem of bundled libraries to use when running the benchmark binaries.

LIBOMP_NUM_HIDDEN_HELPER_THREADS

target nowait is supported via hidden helper task, which is a task not bound to any parallel region. A hidden helper team with a number of threads is created when the first hidden helper task is encountered.

The number of threads can be configured via the environment variable LIBOMP_NUM_HIDDEN_HELPER_THREADS. The default is 8. If LIBOMP_NUM_HIDDEN_HELPER_THREADS is 0, the hidden helper task is disabled and support falls back to a regular OpenMP task. The hidden helper task can also be disabled by setting the environment variable LIBOMP_USE_HIDDEN_HELPER_TASK=OFF.

sysctl -w vm.dirty_ratio=8

Limits dirty cache to 8% of memory.

sysctl -w vm.swappiness=1

Limits swap usage to minimum necessary.

sysctl -w vm.zone_reclaim_mode=1

Frees local node memory first to avoid remote memory usage.

kernel/numa_balancing

This OS setting controls automatic NUMA balancing on memory mapping and process placement. NUMA balancing incurs overhead for no benefit on workloads that are already bound to NUMA nodes.

Possible settings:

• 0: disables this feature
• 1: enables the feature (this is the default)

For more information see the numa_balancing entry in the Linux sysctl documentation.

kernel/randomize_va_space (ASLR)

This setting can be used to select the type of process address space randomization. Defaults differ based on whether the architecture supports ASLR, whether the kernel was built with the CONFIG_COMPAT_BRK option or not, or the kernel boot options used.

Possible settings:

• 0 - Turn the process address space randomization off. This is the default for architectures that do not support this feature anyway, and kernels that are booted with the "norandmaps" parameter.
• 1 - Randomize addresses of mmap base, stack, and VDSO pages. This is the default if the CONFIG_COMPAT_BRK option is enabled at kernel build time.
• 2 - Additionally enable heap randomization. This is the default if CONFIG_COMPAT_BRK is disabled.

Disabling ASLR can make process execution more deterministic and runtimes more consistent. For more information see the randomize_va_space entry in the Linux sysctl documentation.

vm/drop_caches

The two commands are equivalent: echo 3> /proc/sys/vm/drop_caches and sysctl -w vm.drop_caches=3 Both must be run as root. The commands are used to free up the filesystem page cache, dentries, and inodes.

Possible settings:

• 1 - Clear pagecache
• 2 - Clear dentries and inodes
• 3 - Clear pagecache, dentries, and inodes

MALLOC_CONF

The amdalloc library is a variant of jemalloc library. The amdalloc library has tunable parameters, many of which may be changed at run-time via several mechanisms, one of which is the MALLOC_CONF environment variable. Other methods, as well as the order in which they're referenced, are detailed in the jemalloc documentation's TUNING section.

The options that can be tuned at run-time are everything in the jemalloc documentation's MALLCTL NAMESPACE section that begins with "opt.".

The options that may be encountered in SPEC CPU 2017 results are detailed here:

• retain:true - Causes unused virtual memory to be retained for later reuse rather than discarding it. This is the default for 64-bit Linux.
• thp:never - Attempts to never utilize huge pages by using MADV_NOHUGEPAGE on all mappings. This option has no effect except when THP is set to "madvise".

PGHPF_ZMEM

An environment variable used to initialize the allocated memory. Setting PGHPF_ZMEM to "Yes" has the effect of initializing all allocated memory to zero.

GOMP_CPU_AFFINITY

This environment variable is used to set the thread affinity for threads spawned by OpenMP.

OMP_DYNAMIC

This environment variable is defined as part of the OpenMP standard. Setting it to "false" prevents the OpenMP runtime from dynamically adjusting the number of threads to use for parallel execution.

For more information, see chapter 4 ("Environment Variables") in the OpenMP 4.5 Specification.

OMP_SCHEDULE

This environment variable is defined as part of the OpenMP standard. Setting it to "static" causes loop iterations to be assigned to threads in round-robin fashion in the order of the thread number.

For more information, see chapter 4 ("Environment Variables") in the OpenMP 4.5 Specification.

OMP_STACKSIZE

This environment variable is defined as part of the OpenMP standard and controls the size of the stack for threads created by OpenMP.

For more information, see chapter 4 ("Environment Variables") in the OpenMP 4.5 Specification.

OMP_THREAD_LIMIT

This environment variable is defined as part of the OpenMP standard and limits the maximum number of OpenMP threads that can be created.

For more information, see chapter 4 ("Environment Variables") in the OpenMP 4.5 Specification.

Operating System Tuning Parameters

ulimit -s [n | unlimited]:

Set the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.

numactl --interleave=all "runspec command":

Launching a process with numactl --interleave=all sets the memory interleave policy so that memory will be allocated using round robin on nodes. When memory cannot be allocated on the current interleave target fall back to other nodes.

swappiness:

This control is used to define how aggressively the kernel swaps out anonymous memory relative to pagecache and other caches. Increasing the value increases the amount of swapping. The default value is 60. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary. This can be set through a command like "echo 1 > /proc/sys/vm/swappiness".

Zone Reclaim:

Zone reclaim allows the reclaiming of pages from a zone if the number of free pages falls below a watermark even if other zones still have enough pages available. Reclaiming a page can be more beneficial than taking the performance penalties that are associated with allocating a page on a remote zone, especially for NUMA machines.

vm.dirty_ratio:

This setting is the absolute maximum amount of system memory that can be filled with dirty pages before everything must get committed to disk.

cpupower frequency-set:

cpupower utility is a collection of tools for power efficiency of processor. frequency-set sub-command controls settings for processor frequency. "-g [governor]" specifies a policy to select processor frequency. The performance governor statically sets frequency of the processor cores specified by "-c" option to the highest possible for maximum performance.

kernel.randomize_va_space (ASLR)

This setting can be used to control the memory address randomization mechanism (address space layout randomization) under Linux.
Possible settings:

• 0: Turn process address space randomization off.
• 1: Randomize addresses of mmap base, stack, and VDSO pages.
• 2: Additionally randomize the heap.

Transparent Hugepages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. It is designed to hide much of the complexity in using huge pages from system administrators and developers. Huge pages increase the memory page size from 4 kilobytes to 2 megabytes. This provides significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.
THP usage is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/enabled. Possible values:

• never: entirely disable THP usage.
• madvise: enable THP usage only inside regions marked MADV_HUGEPAGE using madvise(3).
• always: enable THP usage system-wide. This is the default.

THP creation is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/defrag. Possible values:

• never: if no THP are available to satisfy a request, do not attempt to make any.
• defer: an allocation requesting THP when none are available get normal pages while requesting THP creation in the background.
• defer+madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3); for all other regions it's like "defer".
• madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3). This is the default.
• always: an allocation requesting THP when none are available will stall until some are made.

An application that "always" requests THP often can benefit from waiting for an allocation until those huge pages can be assembled.
For more information see the Linux transparent hugepage documentation.

Firmware / BIOS / Microcode Settings

SMT Control: (Default="Enabled"):

This is CPU Symmetric Multi-Threading (SMT) function. With SMT Control enabled,each physical processor core operates as two logical processor cores. Values for this BIOS setting can be:

• Disabled: Disable symmertic multi-threading.
• Enabled: Enable symmertic multi-threading.

SVM Mode:(Default="Enabled"):

This is CPU virtualization function. With SVM enabled you'll be able to install a virtual machine on your system. Values for this BIOS setting can be:

• Disabled: Disable CPU Virtualization.
• Enabled: Enable CPU Virtualization.

ACPI SRAT L3 Cache As NUMA Domain: (Default="Auto"):

Each L3 Cache will be exposed as a NUMA node when enabling ACPI SRAT L3 Cache as a NUMA node. On a dual processor system, with up to 8 L3 Caches per processor, this setting will expose 16 NUMA domains. Values for this BIOS setting can be:

• Auto: Use default value of CPI SRAT L3 Cache As NUMA Domain.
• Enabled: Each CCX in the system will be declared as a separate NUMA domain.
• Disabled: Memory Addressing \ NUMA nodes per socket will be declared.

L1/L2 Stream HW Prefetcher: (Default="Auto"):

Most workloads will benefit from the L1 and L2 Stream Hardware prefetchers gathering data and keeping the core pipeline busy. There are however some workloads that are very random in nature and will actually obtain better overall performance by disabling one or both of the prefetchers. Values for this BIOS setting can be:

• Auto: Use default value of L1/L2 Stream HW Prefetcher.
• Enabled: Enable L1/L2 Stream HW Prefetcher.
• Disabled: Disable L1/L2 Stream HW Prefetcher.

IOMMU: (Default="Auto"):

The Input-Output Memory Management Unit(IOMMU) provides several benefits and is required when using x2APIC. Enabling the IOMMU allows devices (such as the EPYC integrated SATA controller) to present separate IRQs for each attached device instead of one IRQ for the subsystem. The IOMMU also allows operating systems to provide additional protection for DMA capable I/O devices. Values for this BIOS option can be:

• Auto: Use default value of IOMMU.
• Enabled: Enable I/O memory management unit.
• Disabled: Disable I/O memory management unit.

Determinism Slider: (Default="Auto"):

This option allows for AGESA determinism to control performance. Values for this BIOS setting can be:

• Auto: Use AGESA default value for deterministic performance control.
• Performance: Provides predictable performance across all processors of the same type.
• Power: Maximizes performance within the power limits defined by cTDP and PPT.

Global C-states Control: (Default="Disabled"):

Controls IO based C-state generation and DF C-states.Values for this BIOS setting can be:

• Auto: Use default value of Global C-states Control.
• Enabled: Enable CPUs to operate in C-state power saving mode.
• Disabled: Disable CPUs to operate in C-state power saving mode.

cTDP Control: (Default="Auto"):

This option allows for user to set customized value of TDP.Values for this BIOS setting can be:

• Auto: Use the fused cTDP.
• Manual: User can set customized cTDP.

cTDP:

TDP is an acronym for "Thermal Design Power." TDP is the recommended target for power used when designing the cooling capacity for a server. EPYC processors are able to control this target power consumption within certain limits. This capability is referred to as "configurable TDP" or "cTDP." cTDP can be used to reduce power consumption for greater efficiency, or in some cases, increase power consumption above the default value to provide additional performance. cTDP is controlled using a BIOS option.

NUMA nodes per socket: (Default="Auto"):

Specifies the number of desired NUMA nodes per socket. Values for this BIOS setting can be:

• Auto: Use default value of NUMA nodes per socket.
• NPS4: sets four NUMA nodes per socket, one per Quadrant.
• NPS2: sets two NUMA nodes per socket.
• NPS1: sets one NUMA nodes per socket.
• NPS0: will attempt to interleave the two sockets together.

APBDIS: (Default="Auto"):

Application Power Management (APM) allows the processor to provide maximum performance while remaining within the specified power delivery and removal envelope. APM dynamically monitors processor activity and generates an approximation of power consumption. If power consumption exceeds a defined power limit, a P-state limit is applied by APM hardware to reduce power consumption. APM ensures that average power consumption over a thermally significant time period remains at or below the defined power limit. Set APBDIS=1 will disable Data Fabric APM and the SOC P-state will be fixed. Values for this BIOS setting can be:

• Auto: Use default value of APBDIS.
• 0: Dynamically switch Infinity Fabric P-state based on link usage.
• 1: Enable fixed Infinity Fabric P-state control.

Fix Soc P-States: (Default="Auto"):

To minimize variance or trade-off memory latency versus bandwidth, algorithm performance boost (APBDIS) can be set and specific hard-fused Data Fabric (SoC) P-states forced for optimized workloads sensitive to latency or throughput.This item is available only when APBDIS is set to 1. Values for this BIOS setting can be:

• Auto: Use default value of Fix Soc P-States.
• P0: Highest-performing Infinity Fabric P-state,which will force the Infinity Fabric and memory controllers into full-power mode.
• P1: Next-highest-performing Infinity Fabric P-state.
• P2: Next highest-performing Infinity Fabric P-state.
• P3: Minimum Infinity Fabric power P-state.

Package Power Limit Control: (Default="Auto"):

This is a per Processor Power Limit value applicable for all populated processors in the system. This can be set to limit the processor power to a certain value. Values for this BIOS option can be.

• Auto: Uses the default processor value.
• Manual: Lets the user set a power limit and exposes Package Power Limit value field that a user can enter a number into. If a number is entered higher than what the processor and/or system supports, the BIOS will limit the power to the max possible value.

Package Power Limit:

Set customize processor Package Power Limit (PPT) value to be used on all populated processors in the system.

Flag description origin markings:

	Indicates that the flag description came from the user flags file.
	Indicates that the flag description came from the suite-wide flags file.
	Indicates that the flag description came from a per-benchmark flags file.

For questions about the meanings of these flags, please contact the tester.
For other inquiries, please contact info@spec.org
Copyright 2017-2024 Standard Performance Evaluation Corporation
Tested with SPEC CPU2017 v1.1.9.
Report generated on 2024-06-24 10:43:02 by SPEC CPU2017 flags formatter v5178.