SPEC CPU2017 Platform Settings for Intel-based systems

Operating System Tuning Parameters

ulimit -s <n>

Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.

cpupower:

The OS 'cpupower' utility is used to change CPU power governors settings. Available settings are:

• performance: Run the CPU at the maximum frequency.
• powersave(default): Run the CPU at the minimum frequency.

tuned-adm:

The 'tuned' provides a number of predefined profiles for typical use cases. The 'tuned-adm' command is used to change settings of the tuned daemon. The tuned-adm command can query current settings, list available profiles, recommend a tuning profile for the system, change profiles directly, or turn off tuning. Available profiles are:

• accelerator-performance: Throughput-performance based tuning with disabled higher latency STOP states.
• balanced: General non-specialized tuned profile.
• desktop: Optimize for the desktop use-case.
• hpc-compute: Optimize for HPC compute workloads.
• intel-sst: Configure for Intel Speed Select Base Frequency.
• latency-performance: Optimize for deterministic performance at the cost of increased power consumption.
• network-latency: Optimize for deterministic performance at the cost of increased power consumption, focused on low latency network performance.
• network-throughput: Optimize for streaming network throughput, generally only necessary on older CPUs or 40G+ networks.
• optimize-serial-console: Optimize for serial console use.
• powersave: Optimize for low power consumption.
• throughput-performance(default): Broadly applicable tuning that provides excellent performance across a variety of common server workloads.
• virtual-guest: Optimize for running inside a virtual guest.
• virtual-host: Optimize for running KVM guests.

transparent_hugepages:

Processes are allowed to use hugepages by writing "always" to the system file/sys/kernel/mm/transparent_hugepage. Possible options are:

• always: means that an application requesting THP will stall on allocation failure and directly reclaim pages and compact memory in an effort to allocate a THP immediately. This may be desirable for virtual machines that benefit heavily from THP use and are willing to delay the VM start to utilise them.
• madvise: will enter direct reclaim like always but only for regions that are have used madvise(MADV_HUGEPAGE). This is the default behaviour.
• never: should be self-explanatory.

numactl --interleave=all "runspec command"

Launching a process with numactl --interleave=all sets the memory interleave policy so that memory will be allocated using round robin on nodes. When memory cannot be allocated on the current interleave target fall back to other nodes.

Firmware / BIOS / Microcode Settings

Patrol Scrub

This BIOS option allows to configure the periodicity of memory scrubbing.

• Setting this BIOS option to Disable, no systematic periodic scrubbing is done
• Setting this BIOS option to an integer from 1 to 24 ensures all the memory is scanned in that number of hours

SNC

Sub NUMA Clustering (SNC) is a feature for breaking up the LLC into disjoint clusters based on address range, with each cluster bound to a subset of the memory controllers in the system. It improves average latency to the LLC and is an important pre-requisite to enabling KTI prefetch and is a replacement for the Cluster-on-Die feature found in previous processors.

• Enable SNC4 (4-clusters): supports 2-way clustering. Utilizes LLC capacity efficiently and reduces latency due to core/IMC proximity. This may provide performance improvement on NUMA-aware operating systems.
• Enable SNC2 (2-clusters): supports 4-way clustering. Utilizes LLC capacity more efficiently and reduces latency due to core/IMC proximity. This may provide performance improvement on NUMA-aware operating systems.
• Disabled: supports 1-cluster and 2-way IMC interleave, the LLC is treated as one cluster.

Power Performance Tuning

Define who is responsible for managing Energy / Performance Balance

• OS Controls EPB: Energy/ performance balance is managed by the operating system
• BIOS Controls EPB: Energy/ performance balance is managed by the BIOS
• PECI Controls EPB: Energy/ performance balance is managed by Intel's Platform Environment Control Interface

Energy Perf Bias CFG mode

This BIOS option allows for processor performance and power optmization. Available settings are:

• Performance0: High performance with less need for power saving.
• Balanced Performance0 (Default Setting): Provides optimal performance efficiency.
• Balanced Power0: Provides optimal power efficiency.
• Power0: High power saving with less need for performance.

DCU Streamer Prefetcher

DCU (Level 1 Data Cache) streamer prefetcher is an L1 data cache prefetcher. Lightly threaded applications and some benchmarks can benefit from having the DCU streamer prefetcher enabled.

Default setting is Enable.

Last Level Cache (LLC)Prefetch

This option configures the processor last level cache (LLC) prefetch feature as a result of the non-inclusive cache architecture. The LLC prefetcher exists on top of other prefetchers that can prefetch data into the core data cache unit (DCU) and mid-level cache (MLC). In some cases, setting this option to disabled can improve performance. Typically, setting this option to enable provides better performance.

Values for this BIOS option can be:

• Disabled: Disables the LLC prefetcher. The other core prefetchers are unaffected.
• Enabled: Gives the core prefetcher the ability to prefetch data directly to the LLC.

LLC dead line alloc:

In the processor non-inclusive cache scheme, MLC evictions are filled into the LLC. When lines are evicted from the MLC, the core can flag them as “dead” (i.e., not likely to be read again). The LLC has the option to drop dead lines and not fill them in the LLC. This can help save space in the LLC and prevent the LLC from evicting useful data. Available settings are:

• Disabled: Disabling this option can save space in the LLC by never moving MLC dead lines into the LLC.
• Enabled(default): Opportunistically move MLC dead lines into the LLC, if space is available.

Package C State

This option allows the user to select the highest sleep state authororized on the system.

• C0/C1 state: the CPUs are always fully turned on
• C2: Core and bus clocks are off. The processor maintains all software-visible state, but can take longer to wake up.
• C6: authorizes very deep sleep, making the CPU take a long time to be operational after each sleep

Link disable

Four UPI links are present between the two sockets of the BullSequana SH20. Each UPI link is power consuming and is not necessary for a well localized benchmark as specCPU

Disabling three out of four links allows power saving at UPI level and increased frequency of the core part of the CPU package

The BIOS setting "Link Disable" is present for all 4 UPI ports of the 2 CPU packages.

This setting can be found in Advanced settings / Socket configuration / Uncore configuration / Uncore Per Socket Configuration / CPU Nb / CPU Nb UPI Port 0-3

Values for this BIOS option can be:

• No (default): The UPI port is enabled
• Yes: The UPI port is disabled

To keep only one link, all settings are set to Yes except CPU 0 UPI Port 0 and CPU 1 UPI Port 1

Enable dIout tuning

Performance tuning option:

• Enabled: system runs with optimized performances
• Disabled: these optmizations will not be activated

FansFullSpeed

By default, the fan speed is regulated to keep the processor temperature just below the maximum To lower the processor temperature, fans are set to the maximum speed using a redfish command on the BMC.

curl -k -u "bmcLogin":"passwd" -H "Content-Type: application/json" -d '{"Oem":{"Eviden_com":{"FansFullSpeed": true}}}' -X PATCH https://"bmc ip address"/redfish/v1/Chassis/Module0/Thermal