CPU2006 Flag Description
Dell Inc. PowerEdge R7425 (AMD EPYC 7601, 2.20 GHz)

Compilers: x86 Open64 Compiler Suite



Base Compiler Invocation

C benchmarks

C++ benchmarks

Fortran benchmarks

Benchmarks using both Fortran and C


Base Portability Flags

410.bwaves

416.gamess

433.milc

434.zeusmp

435.gromacs

436.cactusADM

437.leslie3d

444.namd

447.dealII

450.soplex

453.povray

454.calculix

459.GemsFDTD

465.tonto

470.lbm

481.wrf

482.sphinx3


Base Optimization Flags

C benchmarks

C++ benchmarks

Fortran benchmarks

Benchmarks using both Fortran and C


Implicitly Included Flags

This section contains descriptions of flags that were included implicitly by other flags, but which do not have a permanent home at SPEC.


Commands and Options Used to Submit Benchmark Runs

Using numactl to bind processes and memory to cores

For multi-copy runs or single copy runs on systems with multiple sockets, it is advantageous to bind a process to a particular core. Otherwise, the OS may arbitrarily move your process from one core to another. This can effect performance. To help, SPEC allows the use of a "submit" command where users can specify a utility to use to bind processes. We have found the utility 'numactl' to be the best choice.

numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a command and inherited by all of its children. The numactl flag "--physcpubind" specifies which core(s) to bind the process. "-l" instructs numactl to keep a process memory on the local node while "-m" specifies which node(s) to place a process memory. For full details on using numactl, please refer to your Linux documentation, 'man numactl'

Note that some versions of numactl, particularly the version found on SLES 10, we have found that the utility incorrectly interprets application arguments as it's own. For example, with the command "numactl --physcpubind=0 -l a.out -m a", numactl will interpret a.out's "-m" option as it's own "-m" option. To work around this problem, a user can put the command to be run in a shell script and then run the shell script using numactl. For example: "echo 'a.out -m a' > run.sh ; numactl --physcpubind=0 bash run.sh"

numactl is also used to invoke runspec so that mememory usage is spread evenly among NUMA nodes. This is accomplished as follows: runspec_command="numactl --interleave=all runspec"


Shell, Environment, and Other Software Settings

Linux Huge Page settings

In order to take full advantage of using x86 Open64's huge page runtime library, your system must be configured to use huge pages. It is safe to run binaries compiled with "-HP" on systems not configured to use huge pages, however, you will not benefit from the performance improvements huge pages offer. To configure your system for huge pages perform the following steps:

Note that further information about huge pages may be found in your Linux documentation file: /usr/src/linux/Documentation/vm/hugetlbpage.txt

HUGETLB_LIMIT

For the x86 Open64 compiler, the maximum number of huge pages an application is allowed to use can be set at run time via the environment variable HUGETLB_LIMIT. If not set, then the process may use all available huge pages when compiled with "-HP (or -HUGEPAGE)" or a maximum of n pages where the value of n is set via the compile time flag "-HP:limit=n".

Transparent Huge Pages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed to hides much of the complexity in using huge pages from system administrators and developers, as normal huge pages must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in order to be used effectively.

Set transparent_hugepage boot parameter

In the file /boot/grub/menu.lst, add the boot parameter "transparent_hugepage=never" to the OS you plan to select during boot, to instruct it to disable Transparent Huge Pages (THP). A reboot is required for this setting to take effect.

Set Ubuntu power governor to performance

To produce the best performance on Ubuntu, the system power governor must be set to performance as follows:

ulimit -s <n>

Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.

ulimit -l <n>

Sets the maximum size of memory that may be locked into physical memory.

dirty_ratio

Sets the percentage limit of system memory that can hold dirty cache data until it is written out via pdflush.

swappiness

The swappiness value can range from 1 to 100. A value of 100 will cause the kernel to swap out inactive processes frequently in favor of file system performance, resulting in large disk cache sizes. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary.

zone_reclaim_mode

When zone_reclaim_mode is set to 0, the kernel will allocate memory from a remote node, rather than try to reclaim memory from the local node. A value of 1 will cause the page allocator to reclaim local page caches that are not currently used before allocating remote node memory.

sync, drop_caches

Used in conjunction, the two commands, sync and drop_caches, free disk cache memory for other uses. sync writes dirty pages to disk, while drop_caches reclaims clean disk cache pages.

OMP_NUM_THREADS

Sets the maximum number of OpenMP parallel threads auto-parallelized (-apo) applications may use.

O64_OMP_AFFINITY_MAP

Specifies the thread-CPU relationship when the operating system's affinity mechanism is used to assign OpenMP threads to CPUs.

O64_OMP_SPIN_USER_LOCK

Specifies whether or not to use the user-level spin mechanism for OpenMP locks. If the variable is set to TRUE then user-level spin mechanisms are used. If the variable is set to FALSE then pthread mutexes are used. The default if the variable is not set is the same as FALSE.

powersave -f (on SuSE)

Makes the powersave daemon set the CPUs to the highest supported frequency.

/etc/init.d/cpuspeed stop (on Red Hat)

Disables the cpu frequency scaling program in order to set the CPUs to the highest supported frequency.

LD_LIBRARY_PATH

An environment variable set to include the x86 Open64 and SmartHeap libraries used during compilation of the binaries. This environment variable setting is not needed when building the binaries on the system under test.

kernel/randomize_va_space

This option can be used to select the type of process address space randomization that is used in the system, for architectures that support this feature. 0 - Turn the process address space randomization off. This is the default for architectures that do not support this feature anyways, and kernels that are booted with the "norandmaps" parameter. 1 - Make the addresses of mmap base, stack and VDSO page randomized. This, among other things, implies that shared libraries will be loaded to random addresses. Also for PIE-linked binaries, the location of code start is randomized. This is the default if the CONFIG_COMPAT_BRK option is enabled. 2 - Additionally enable heap randomization. This is the default if CONFIG_COMPAT_BRK is disabled.

O64_OMP_SPIN_COUNT

Specify the number of times the spin loops will spin at user-level before falling back to operating system schedule/reschedule mechanisms. The default value is 20000.


Firmware / BIOS / Microcode Settings

C States:

C States allow the processor to enter lower power states when idle. When set to Enabled (OS controlled) or when set to Autonomous (if Hardware controlled is supported), the processor can operate in all available Power States to save power, but my increase memory latency and frequency jitter.

C1E:

When set to Enabled, the processor is allowed to switch to minimum performance state when idle.

CPU Interconnect Bus Link Power Management:

When enabled, CPU interconnect bus link power management can reduce overall system power a bit while slightly reducing system performance.

CPU Performance:

Maximum Performance is typically selected for performance-centric workloads where it is acceptable to consume additional power to achieve the highest possible performance for the computing environment. This mode drives processor frequency to the maximum across all cores (although idled cores can still be frequency reduced by C-state enforcement through BIOS or OS mechanisms if enabled). This mode also offers the lowest latency of the CPU Power Management Mode options, so is always preferred.

Energy Efficient Policy:

The CPU uses the setting to manipulate the internal behavior of the processor and determines whether to target higher performance or better power savings.

Energy Efficient Turbo:

Enables or disables the Energy Efficient Turbo.
Energy Efficient Turbo (EET) is a mode of operation where a processor's core frequency is adjusted within the turbo range based on workload.

Logical Processor:

Each processor core supports up to two logical processors. When set to Enabled, the BIOS reports all logical processors. When set to Disabled, the BIOS only reports one logical processor per core. Generally, higher processor count results in increased performance for most multi-threaded workloads and the recommendation is to keep this enabled. However, there are some floating point/scientific workloads, including HPC workloads, where disabling this feature may result in higher performance.

Memory Patrol Scrub:

Patrol Scrubbing searches the memory for errors and repairs correctable errors to prevent the accumulation of memory errors. When set to Disabled, no patrol scrubbing will occur. When set to Standard Mode, the entire memory array will be scrubbed once in a 24 hour period. When set to Extended Mode, the entire memory array will be scrubbed more frequently to further increase system reliability.

PCI ASPM L1 Link Power Management:

When enabled, PCIe Advanced State Power Management (ASPM) can reduce overall system power a bit while slightly reducing system performance.

NOTE: Some devices may not perform properly (they may hang or cause the system to hang) when ASPM is enable, for this reason L1 will only be enabled for validated qualified cards.

System Profile:

When set to Custom, you can change setting of each option. Under Custom mode when C state is enabled, Monitor/Mwait should also be enabled.

Sub NUMA Cluster:

When ENABLED, Sub NUMA Clustering (SNC) is a feature for breaking up the LLC into disjoint clusters based on address range, with each cluster bound to a subset of the memory controllers in the system. It improves average latency to the LLC.

Uncore Frequency:

Selects the Processor Uncore Frequency.
Dynamic mode allows processor to optimize power resources across the cores and uncore during runtime. The optimization of the uncore frequency to either save power or optimize performance is influenced by the setting of the Energy Efficiency Policy.

Virtualization technology:

When set to Enabled, the BIOS will enable processor Virtualization features and provide the virtualization support to the Operating System (OS) through the DMAR table. In general, only virtualized environments such as VMware(r) ESX (tm), Microsoft Hyper-V(r) , Red Hat(r) KVM, and other virtualized operating systems will take advantage of these features. Disabling this feature is not known to significantly alter the performance or power characteristics of the system, so leaving this option Enabled is advised for most cases.

nohz_full:

This kernel option sets adaptive tick mode (NOHZ_FULL) to specified processors. Since the number of interrupts is reduced to ones per second, latency-sensitive applications can take advantage of it.

Memory Interleaving:

When Enabled, memory interleaving is supported if a symmetric memory configuration is installed. When set to Disabled, the system supports Non-Uniform Memory Access (NUMA) (asymmetric) memory configurations.


Flag description origin markings:

[user] Indicates that the flag description came from the user flags file.
[suite] Indicates that the flag description came from the suite-wide flags file.
[benchmark] Indicates that the flag description came from a per-benchmark flags file.

The flags files that were used to format this result can be browsed at
http://www.spec.org/cpu2006/flags/x86-openflags-rate-revA-I.html,
http://www.spec.org/cpu2006/flags/Dell-Platform-Flags-PowerEdge14G-revD.20171221.html.

You can also download the XML flags sources by saving the following links:
http://www.spec.org/cpu2006/flags/x86-openflags-rate-revA-I.xml,
http://www.spec.org/cpu2006/flags/Dell-Platform-Flags-PowerEdge14G-revD.20171221.xml.


For questions about the meanings of these flags, please contact the tester.
For other inquiries, please contact webmaster@spec.org
Copyright 2006-2018 Standard Performance Evaluation Corporation
Tested with SPEC CPU2006 v1.2.
Report generated on Tue Feb 20 18:12:10 2018 by SPEC CPU2006 flags formatter v6906.