Platform Settings for GIGABYTE SRF server systems

Operating System Tuning Parameters

OS Tuning

setterm:

Used to disable screen blanking on text console:

• "setterm -powersave off -blank 0"

ulimit:

Used to set user limits of system-wide resources. Provides control over resources available to the shell and processes started by it. Some common ulimit commands may include:

• ulimit -s [n | unlimited]: Set the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.
• ulimit -l (number): Set the maximum size that can be locked into memory.

Disabling Linux services:

Certain Linux services may be disabled to minimize tasks that may consume CPU cycles.

irqbalance:

Disabled through "service irqbalance stop". Depending on the workload involved, the irqbalance service reassigns various IRQ's to system CPUs. Though this service might help in some situations, disabling it can also help environments which need to minimize or eliminate latency to more quickly respond to events.

Performance Governors (Linux):

In-kernel CPU frequency governors are pre-configured power schemes for the CPU. The CPUfreq governors use P-states to change frequencies and lower power consumption. The dynamic governors can switch between CPU frequencies, based on CPU utilization to allow for power savings while not sacrificing performance.

Other options beside a generic performance governor can be set, such as the perf-bias:

--perf-bias, -b

On supported Intel processors, this option sets a register which allows the cpupower utility (or other software/firmware) to set a policy that controls the relative importance of performance versus energy savings to the processor. The range of valid numbers is 0-15, where 0 is maximum performance and 15 is maximum energy efficiency.

The processor uses this information in model-specific ways when it must select trade-offs between performance and energy efficiency. This policy hint does not supersede Processor Performance states (P-states) or CPU Idle power states (C-states), but allows software to have influence where it would otherwise be unable to express a preference.

On many Linux systems one can set the perf-bias for all CPUs through the cpupower utility with one of the following commands:

• "cpupower frequency-set -g performance"
• "cpupower -c all set -b 0"
• "cpupower -c all set --perf-bias 0"
• "cpupower set -b 0"

Tuning Kernel parameters:

The following Linux Kernel parameters were tuned to better optimize performance of some areas of the system:

• dirty_background_ratio: Set through "echo 40 > /proc/sys/vm/dirty_background_ratio". This setting can help Linux disk caching and performance by setting the percentage of system memory that can be filled with dirty pages.
• dirty_ratio: Set through "echo 40 > /proc/sys/vm/dirty_ratio". This setting is the absolute maximum amount of system memory that can be filled with dirty pages before everything must get committed to disk.
• swappiness: The swappiness value can range from 1 to 100. A value of 100 will cause the kernel to swap out inactive processes frequently in favor of file system performance, resulting in large disk cache sizes. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary. This can be set through a command like "echo 1 > /proc/sys/vm/swappiness"
• ksm/sleep_millisecs: Set through "echo 200 > /sys/kernel/mm/ksm/sleep_millisecs". This setting controls how many milliseconds the ksmd (KSM daeomn) should sleep before the next scan.
• khugepaged/scan_sleep_millisecs: Set through "echo 50000 > /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs". This setting controls how many milliseconds to wait in khugepaged is there is a hugepage allocation failure to throttle the next allocation attempt.
• numa_balancing: Disabled through "echo 0 > /proc/sys/kernel/numa_balancing". This feature will automatically migrate data on demand so memory nodes are aligned to the local CPU that is accessing data. Depending on the workload involved, enabling this can boost the performance if the workload performs well on NUMA hardware. If the workload is statically set to balance between nodes, then this service may not provide a benefit.
• Zone Reclaim Mode: Zone reclaim allows the reclaiming of pages from a zone if the number of free pages falls below a watermark even if other zones still have enough pages available. Reclaiming a page can be more beneficial than taking the performance penalties that are associated with allocating a page on a remote zone, especially for NUMA machines. To tell the kernel to free local node memory rather than grabbing free memory from remote nodes, use a command like "echo 1 > /proc/sys/vm/zone_reclaim_mode"
• max_map_count-n: The maximum number of memory map areas a process may have. Memory map areas are used as a side-effect of calling malloc, directly by mmap and mprotect, and also when loading shared libraries.

Transparent Huge Pages (THP):

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed to hide much of the complexity in using huge pages from system administrators and developers, as normal huge pages must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in order to be used effectively. Transparent Hugepages increase the memory page size from 4 kilobytes to 2 megabytes. Transparent Hugepages provide significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.

Linux Huge Page settings:

If you need finer control and manually set the Huge Pages you can follow the below steps:

• Create a mount point for the huge pages: "mkdir /mnt/hugepages"
• The huge page file system needs to be mounted when the systems reboots. Add the following to a system boot configuration file before any services are started: "mount -t hugetlbfs nodev /mnt/hugepages"
• Set vm/nr_hugepages=N in your /etc/sysctl.conf file where N is the maximum number of pages the system may allocate.
• Reboot to have the changes take effect.

Note that further information about huge pages may be found in your Linux documentation file: /usr/src/linux/Documentation/vm/hugetlbpage.txt

Firmware / BIOS / Microcode Settings

Firmware Settings

Power Policy Quick Settings: (Default = Standard)

This BIOS option controls the performance or power save setting, user can use this item to change it. Best performance can maximize the performance of the server. Energy efficiency can maximize the power efficiency of the server.

Values for this BIOS setting can be:

• Standard: BIOS default setting.
• Best Performance: Maximize the performance of the server.
• Energy Efficient: Maximize the power efficiency of the server.

SR-IOV Support (Default = Enabled):

If system has SR-IOV capable PCIe Devices, this option Enables or Disables Single Root IO Virtualization Support. Values for this BIOS option can be:

• Enabled: Enables Single Root IO Virtualization Support.
• Disabled: Disables Single Root IO Virtualization Support.

Stale AtoS (Default = Auto):

Stale AtoS (sometimes called A2S) is the transition of a directory line state. The in-memory directory has three states: I, A, and S. Invalid (I) state means the data is clean and does not exist in any other socket’s cache. snoopAll (A) state means the data may exist in another socket in exclusive or modified state. Shared (S) state means the data is clean and may be shared across one or more socket’s caches. When doing a read to memory, if the directory line is in the A state we must snoop all the other sockets because another socket may have the line in modified state. If this is the case, the snoop will return the modified data. However, it may be the case that a line is read in A state and all the snoops come back a miss. This can happen if another socket read the line earlier and then silently dropped it from its cache without modifying it. Values for this BIOS option can be:

• Enable: In the situation where a line in A state returns only snoop misses, the line will transition to S state. That way, subsequent reads to the line will encounter it in S state and not have to snoop, saving latency and snoop bandwidth
• Disable: Disabling this option allows the feature to process memory directory states as described previously.
• Auto: Auto decides based on Si Compatibility.

LLC dead line alloc (Default = Enable):

In the Intel® Xeon® Scalable family non-inclusive cache scheme, MLC evictions are filled into the LLC. When lines are evicted from the MLC, the core can flag them as “dead” (in other words, not likely to be read again). The LLC has the option to drop dead lines and not fill them in the LLC. This can help save space in the LLC and prevent the LLC from evicting useful data. Values for this BIOS option can be:

• Enable: Opportunistically move MLC dead lines into the LLC, if space is available.
• Disable: Disabling this option can save space in the LLC by never moving MLC dead lines into the LLC.
• Auto: BIOS will choose the correct setting.

Hardware P-States (Default = Native Mode):

Processors provide a flexible interface between hardware and the platform, enhancing performance management and improving energy efficiency. Hardware P-states (HWP) refers to the set of algorithms used to dynamically determine core frequency. When HWP is 'Disable' or 'OOB', Dynamic SST-PP will be disabled. SST-BF can only be enabled when HWP mode is 'Native Mode with No Legacy Support'. Values for this BIOS option can be:

• Disable: Hardware chooses a P-state based on OS Request(Legacy P-States).
• Native Mode: Hardware chooses a P-state based on OS guidance.
• Out of Band Mode: Hardware autonomously chooses a P-state (no OS guidance).
• Native Mode with No Legacy Support: This mode is the same as native mode, except that the processor will disregard inputs from the legacy performance control interface.

Hyper-Performance (Default = Disabled):

Hyper-Performance support depends on the CPU. Enabling Hyper-Performance may require a better cooling solution (such as a heatsink or D2C liquid cooling) to prevent the CPU from overheating and throttling. This BIOS option providing a knob for tuning processor Voltage Regulator chipset(power supply), allowing the user to adjust the performance level. Values for this BIOS option can be:

• Disabled: Disable Hyper-Performance, BIOS default setting.
• Level 1: Provides better performance than the BIOS default setting.
• Level 2: Provides better performance than Level 1.
• Level 3: Provides better performance than Level 2.
• Maximum: The highest level of Hyper-Performance. Maximize the performance of the server.

Enforce DDR Memory Frequency POR (Default = Enforce POR):

Enforces Plan Of Record restrictions for DDR frequency voltage programming, [Disable] option is over spec and depend on memory quality. Values for this BIOS option can be:

• Enforce POR: Memory speeds will be capped at Intel guidelines.
• Enforce Stretch Goals: Enforce DDR memory frequency stretch goal.
• Disable: allows user selection of additional supported memory speeds.

First created August 21, 2024.