Performance Summary:

Power Summary:

Power Details:

Performance Details:

Validation Errors:

•   No Validation Errors Found

Power Measurement Configuration:

Physical Configuration:

Virtual Configuration:

Notes:

Physical System Notes

• BIOS settings used:
• Power Profile set to Maximum Performance
• Thermal Configuration set to Increased Cooling
• CPU Core Hardware Prefetcher set to Disabled
• Primary cache architecture for the processor is 512 KB I cache on chip per chip
• with 64 KB shared per 2 cores, and 16 KB D cache on chip per core

Storage Notes

• Each port of the SUT's HP StorageWorks 82Q 8Gb Fibre Channel HBA was connected
• one 8Gb Fibre Channel link to 1 x StorageWorks 8/24 SAN switch. 4 x
• StorageWorks P2000 storage enclosures were connected via 2 x 8Gb Fibre Channel
• links to the StorageWorks SAN switch. All P2000's contained 24 x 146 GB SFF
• 15 K SAS drives and were partitioned into 3 x vdisks. Each vdisk was configured
• as RAID10 with 8 drives and contained one tile's VMs.

Virtualization Software Notes

• Settings made to drivers via esxcfg-module in ESX:
• for qla2xxx driver: ql2xexecution_throttle=255, ql2xmaxqdepth=128, ql2xintrdelaytimer=1
• for ixgbe driver: VMDQ=1,1 InterruptThrottleRate=2000,2000
• All VMs' for a given tiles were assigned to the virtual switch attached to one of the 10GbE ports. One
• 10 GbE port was assigned to 8 tiles and one to 9 tiles. The second virtual NIC for the appserver, dbserver,
• infraserverand the webserver were attached to an internal virtual switch unique to their tile. These internal
• switches were set to use jumbo frames by using the command: esxcfg-vswitch -m 9000 vSwitch[x].
• All VMs started from ESX using command: vmware-cmd {path to vmx file} start
• Advanced ESX Software settings used:
• /adv/BufferCache/SoftMaxDirty = 75
• /adv/Disk/SchedNumReqOutstanding = 128
• /adv/Irq/BestVcpuRouting = 0
• /adv/Net/MaxNetifRxQueueLen = 1000
• /adv/Net/MaxPortRxQueueLen = 500
• /adv/Net/VmxnetTxCopySize = 1500
• /adv/Net/VmxnetThroughputWeight = 128
• /adv/Misc/TimerMinHardPeriod = 2000
• /adv/Numa/PageMigRateMax = 10
• /adv/Numa/MigPreventTermThresh = 20
• /adv/Cpu/CoschedCrossCall = 0

Web Server VM Notes

• Boot parameters set: divider=10, selinux=0, audit=0.
• SELINUX=disabled was set in /etc/selinux/config.
• Both Virtual NICs used VMXNET 3 driver.
• Internal NIC set to use jumbo frames by adding 'MTU=9000' to /etc/sysconfig/network-scripts/ifconfig-eth1
• VM setting CPU shares was set to 'high' via vConsole
• VM setting Memory Reservation was set to '8704' via vConsole
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000
• net.core.wmem_default = 1000000
• Add line in httpd.conf to 'Include conf/extra/httpd-default.conf'
• Add line in httpd.conf to 'Include conf/extra/httpd-mpm.conf'
• Change values in extra/httpd-default.conf:
• MaxKeepAliveRequest 0
• Change values in extra/httpd-mpm.conf:
• StartServers 20
• MinSpareServers 20
• MaxSpareServers 20
• ServerLimit 1536
• MaxClients 1536
• MaxRequestsPerChild 4000

Application Server VM Notes

• Boot parameters set: divider=10, selinux=0, audit=0.
• SELINUX=disabled was set in /etc/selinux/config.
• Both virtual NICs used VMXNET 3 driver
• Internal NIC set to use jumbo frames by adding 'MTU=9000' to /etc/sysconfig/network-scripts/ifconfig-eth1
• VM setting CPU shares was set to '3600' via vConsole
• VM setting Memory Reservation was set to '2560' via vConsole
• The following lines were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000
• Huge pages were mounted by using the following lines in /etc/rc.local:
• echo 768 > /proc/sys/vm/nr_hugepages
• echo 500 > /proc/sys/vm/hugetlb_shm_group
• mkdir /dev/hugepages
• mount -t hugetlbfs hugetlbfs /dev/hugepages
• The following lines were added to /etc/security/limits.conf:
• soft memlock 2048000
• hard memlock 2048000
• The following lines were added to /opt/glassfish/domains/spec2004-1/domain.xml:
• To jdbc-connection-pool line parameter 'max-pool-size = 64' was set
• To request-processing line parameter 'initial-thread-count = 90' was set
• To ejb-container line, parameter 'max-pool-size=64' was set
• To mdb-container line, parameter 'max-pool-size = 64' was set
• jvm-option '-XX:+AggressiveHeap' was set
• jvm-option '-Xmn600m' was set
• jvm-option '-Xmx1024m' was set
• jvm-option '-Xms1024m' was set
• jvm-option '-XX:PermSize=192m' was set
• jvm-option '-Xss128k' was set
• jvm-option '-XX:+DisableExplicitGC' was set
• jvm-option '-XX:+UseParallelOldGC' was set
• jvm-option '-XX:+UseLargePages' was set
• jvm-option '-XX:LargePageSizeInBytes=2m' was set
• jvm-option '-XX:-UseBiasedLocking' was set
• jvm-option '-XX:CompileThreshold=1000' was set
• jvm-option '-XX:+AggressiveOpts' was set
• jvm-option '-XX:+UseCompressedOops' was set
• jvm-option '-XX:+UseFastAccessorMethods' was set
• jvm-option '-XX:+UseStringCache' was set
• jvm-option '-XX:+OptimizeStringConcat' was set

Mail Server VM Notes

• Boot parameters set: divider=10, selinux=0, audit=0.
• SELINUX=disabled was set in /etc/selinux/config.
• Virtual NIC used VMXNET 3 driver
• VM setting CPU shares was set to '450' via vConsole
• VM setting Memory Reservation was set to '512' via vConsole
• The following lines were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000
• dovecot was upgraded to 2.0.4-1_119.el5.x84_64. The dbox format was used.
• Contents of dovecot.conf file:
• base_dir = /var/run/dovecot/
• disable_plaintext_auth = no
• listen = *, ::
• log_path = /var/log/dovecot
• mail_debug = no
• mail_location = dbox:/mailstore/dovecot/maildir/%u
• mdbox_rotate_size = 2048
• passdb {
• args = /etc/passwd.dovecot
• driver = passwd-file
• }
• protocols = imap
• service auth {
• user = root
• }
• service imap-login {
• inet_listener imap {
• port = 143
• }
• process_min_avail = 10
• }
• ssl = no
• userdb {
• driver = passwd
• }
• protocol lda {
• postmaster_address = postmaster@example.com
• mail_fsync = optimized
• }

Database Server VM Notes

• Boot parameters set: divider=10, selinux=0, audit=0.
• SELINUX=disabled was set in /etc/selinux/config.
• Both virtual NICs used VMXNET 3 driver
• Internal NIC set to use jumbo frames by adding 'MTU=9000' to /etc/sysconfig/network-scripts/ifconfig-eth1
• VM setting CPU shares was set to '2400' via vConsole
• VM setting Memory Reservation was set to '1536' via vConsole
• The following lines were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000

Infraserver VM Notes

• Boot parameters set: divider=10, selinux=0, audit=0.
• SELINUX=disabled was set in /etc/selinux/config.
• Both Virtual NICs used VMXNET 3 driver
• Internal NIC set use jumbo frames by adding 'MTU=9000' to /etc/sysconfig/network-scripts/ifconfig-eth1
• VM setting Memory Reservation was set to '512' via vConsole
• The following lines were added to sysctl.conf:
• fs.file-max = 500000
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000

Idle Server VM Notes

• Boot parameters set: divider=10, selinux=0, audit=0.
• SELINUX=disabled was set in /etc/selinux/config.
• Virtual NIC used VMXNET 3 driver
• VM setting CPU shares was set to '600' via vConsole
• VM setting Memory Reservation was set to '512' via vConsole
• VM setting Memory shares was set to 'low' via vConsole
• VM setting Storage shares was set to 'low' via vConsole
• The following lines were added to sysctl.conf:
• fs.file-max = 500000
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000

Client Driver Notes

• Each physical client runs 4 virtual clients. Each virtual client uses 1 x vCPU, 3 GB memory, and 1 x 1Gb NIC port.
• The first virtual client on the first pysical client is dedicated as the prime controller
• One virtual client was used for each tile.

Power Configuration Notes

• PTD[0] was connected to SUT only.
• PTD[1] was connected to 6 x P2000 enclosures and 1 x StorageWorks 8/8 Fibre switches.

Other Notes

• A single ProCurve 2910al 48-port switch was used to connect clients to SUT. The 10GbE ports were attached to
• a dual port 10 GbE SFP+ expansion module (J9008A)