SPEC SFS®2014_swbuild Result
Copyright © 2016-2019 Standard Performance Evaluation Corporation
|
SPEC SFS®2014_swbuild ResultCopyright © 2016-2019 Standard Performance Evaluation Corporation |
| NetApp, Inc. | SPEC SFS2014_swbuild = 6200 Builds |
|---|---|
| NetApp 12-node AFF A800 with FlexGroup | Overall Response Time = 0.83 msec |
|
|
| NetApp 12-node AFF A800 with FlexGroup | |
|---|---|
| Tested by | NetApp, Inc. | Hardware Available | May 2018 | Software Available | January 2019 | Date Tested | November 2018 | License Number | 33 | Licensee Locations | Sunnyvale, CA USA |
Combined with the industry's first support of NVMe inside and out and NetApp
ONTAP data management software, AFF A800 all-flash systems accelerate, manage,
and protect your business-critical data with the industry's highest
performance, superior flexibility, and best-in-class data management and cloud
integration. By combining low-latency NVMe solid-state drives (SSDs) and the
first NVMe over Fibre Channel (NVMe/FC) connectivity, the AFF A800 delivers
ultra-low latency and massive throughput, and scales up to 24 nodes in a
cluster.
The FlexGroup feature of ONTAP 9 enables massive scaling in a
single namespace to over 20PB with over 400 billion files while evenly
spreading the performance across the cluster. This makes the AFF A800 a great
system for engineering and design applications as well as DevOps. It is
particularly well-suited for chip development and software builds that are
typically high file-count environments with high meta-data traffic.
| Item No | Qty | Type | Vendor | Model/Name | Description |
|---|---|---|---|---|---|
| 1 | 6 | Storage System | NetApp | AFF A800 Flash System (HA Pair, Active-Active Dual Controller) | A single NetApp AFF A800 system is a single chassis with 2 controllers and 48 drive slots. Each set of 2 controllers comprises a High-Availability (HA) Pair. The words "controller" and "node" are used interchangeably in this document. Each AFF A800 HA Pair includes 1280GB of ECC memory, 128GB of NVRAM, 8 PCIe expansion slots and a set of included I/O ports: * 4x 40/100 GbE ports, in Slots numbered 1 in the controllers, configured as 100 GbE, used for cluster interconnect and HA connections; * 4x 40/100 GbE ports, in the mezzanine location in the controllers, configured as 100 GbE, used for cluster interconnect and HA connections; Included Premium Bundle which includes All Protocols, SnapRestore, SnapMirror, SnapVault, FlexClone, SnapManager Suite, Single Mailbox Recovery (SMBR), SnapCenter Foundation. Only the NFS protocol license is active in the test, also available in the BASE bundle. |
| 2 | 12 | Network Interface Card | NetApp | 2-Port 40/100 GbE QSFP28 X1146A | 1 card in Slot 3 of each controller; 2 cards per HA pair; each has 2 ports; used for data connections |
| 3 | 288 | Solid-State Drive | NetApp | 3.84TB NVMe SSD X4002A | NVMe Solid-State Drives (NVMe SSDs) installed in chassis, 48 per HA pair |
| 4 | 72 | Network Interface Card | Intel | XXV710-DA2 | 2-port 25 GbE NIC, one installed per client. Only one 25GbE port per client was active during the test. |
| 5 | 1 | Switch | Cisco | Cisco Nexus 9000 C9516 | Used for Ethernet data connections between clients and storage systems. Large switch in use due to testing having been done in a large shared-infrastructure lab. Only the ports used for this test are listed in this report. See the 'Transport Configuration - Physical' section for connectivity details. |
| 6 | 12 | Linecard | Cisco | N9K-X97160YC-EX | Cisco Nexus 9500 48p 1/10/25G SFP+ plus 4p 100G QSFP cloud-scale line card. Clients connected to the 10/25 ports on these cards. |
| 7 | 4 | Linecard | Cisco | N9K-X9732C-EX | Cisco Nexus 9500 32x100G Ethernet Module. The AFF A800 connected to these cards. |
| 8 | 24 | Fibre Channel Interface Card | Emulex | Quad Port 32 Gb FC X1135A | Located in Slots 2 and 5 of each controller; there were 2 such cards in each controller; they were not used for this test. They were in place because this is a large shared-infrastructure lab environment; no I/O was directed through these cards during this test. |
| 9 | 1 | Switch | Cisco | Cisco Nexus 3000 C3232C | Used for 100 GbE cluster interconnections |
| 10 | 72 | Client | SuperMicro | Custom build from Superserver 5018R-W base | Clients are a custom build ordered from Supermicro. The base build was a Superserver 5018R-W. The custom build includes a X10SRW-F motherboard, a CSE-815TQ-600WB chassis, a RSC-R1UW-2E16-O-P Dual PCI-Express 3.0 x16 Riser, a RSC-R1UW-E8R-O-P Single PCI-Express 3.0 x8 Riser Card, Intel Xeon E5-1630 v4 Quad-core Haswell Processors, and 4 DDR 2400 16GB ECC/REG DIMMs. 1 used as Prime Client; 71 used to generate the workload. |
| 11 | 10944 | Software Enablement/License | NetApp | OS-ONTAP1-CAP1-PREM-2P | Capacity-based License, Per 0.1TB |
| Item No | Component | Type | Name and Version | Description |
|---|---|---|---|---|
| 1 | Linux | OS | Centos Linux 6.10 (Kernel 4.18.12) | Operating System (OS) for the 72 clients |
| 2 | ONTAP | Storage OS | 9.5 | Storage Operating System |
| 3 | Data Switch | Operating System | 7.0(3)I6(1) | Cisco switch NX-OS (system software) |
| 4 | Cluster Switch | Operating System | 7.0(3)I6(1) | Cisco switch NX-OS (system software) |
| Storage | Parameter Name | Value | Description |
|---|---|---|
| MTU | 9000 | Jumbo Frames configured for Cluster Interconnect ports |
| MTU | 9000 | Jumbo Frames configured for data ports |
NetApp AFF A800 storage controller 100 GbE ports used for cluster interconnects and HA connections (8 per HA pair) were set up with MTU of 9000. Data network was set up with MTU of 9000.
| Clients | Parameter Name | Value | Description |
|---|---|---|
| rsize,wsize | 65536 | NFS mount options for data block size |
| protocol | tcp | NFS mount option for protocol |
| nfsvers | 3 | NFS mount option for NFS version |
| somaxconn | 65536 | Max tcp backlog an application can request |
Tuned the necessary client parameters as shown above, for communication between clients and storage controllers over Ethernet, to optimize data transfer and minimize overhead.
None
| Item No | Description | Data Protection | Stable Storage | Qty |
|---|---|---|---|---|
| 1 | 3.84 TB NVMe SSDs used for data and storage operating system; used to build three RAID-DP RAID groups per storage controller node in the cluster | RAID-DP | Yes | 288 |
| 2 | 960GB NVMe M.2 device, 2 per HA pair; used as boot media | none | Yes | 12 |
| Number of Filesystems | 1 | Total Capacity | 671.5 TiB | Filesystem Type | NetApp FlexGroup |
|---|
The single FlexGroup consumed all data volumes from all of the aggregates across all of the nodes. In order to support the high number of files required by the benchmark, immediately after creation the default inode density of the FlexGroup was increased. At the same time the FlexGroup's snapshot reserve percentage was also set to zero. These actions were accomplished with a single 'volume modify' command with option '-files 12000000000' and option '-percent-snapshot-space 0'.
The storage configuration consisted of 6 AFF A800 HA pairs (12 controller nodes
total). The two controllers in each HA pair were connected in a SFO (storage
failover) configuration. Together, all 12 controllers (configured as 6 HA
Pairs) comprise the tested AFF A800 HA cluster. Stated in the reverse, the
tested AFF A800 HA cluster consists of 6 HA Pairs, each of which consists of 2
controllers (also referred to as nodes).
Each storage controller was
connected to its own and partner's NVMe drives in a multi-path HA
configuration.
All NVMe SSDs were in active use during the test. In
addition to the factory configured RAID Group housing its root aggregate, each
storage controller was configured with two 21+2 RAID-DP RAID Groups. There were
2 data aggregates on each node, each of which consumed one of the node's two
21+2 RAID-DP RAID Groups. 8x volumes, holding benchmark data, were created
within each aggregate. "Root aggregates" hold ONTAP operating system related
files. Note that spare (unused) drive partitions are not included in the
"storage and filesystems" table because they held no data during the benchmark
execution.
A storage virtual machine or "SVM" was created on the
cluster, spanning all storage controller nodes. Within the SVM, a single
FlexGroup volume was created using the two data aggregates on each controller.
A FlexGroup volume is a scale-out NAS single-namespace container that provides
high performance along with automatic load distribution and scalability.
| Item No | Transport Type | Number of Ports Used | Notes |
|---|---|---|---|
| 1 | 25 GbE and 100 GbE | 96 | For the client-to-storage network, the AFF A800 Cluster used a total of 24x 100 GbE connections from storage to the switch, communicating via NFSv3 over TCP/IP to 72 clients, via one 25 GbE connection to the switch for each client. MTU=9000 was used for data switch ports. The benchmark was conducted in a large shared-infrastructure lab; only the ports shown and documented were used on the Cisco Nexus C9516 switch for this benchmark test. |
| 2 | 100 GbE | 24 | The Cluster Interconnect network is connected via 100 GbE to a Cisco C3232 switch, with 4 connections to each HA pair. |
Each NetApp AFF A800 HA Pair used 4x 100 GbE ports for data transport connectivity to clients (through the Cisco C9516 switch), Item 1 above. Each of the clients driving workload used one 25GbE port for data transport. All ports on the Item 1 network utilized MTU=9000. The Cluster Interconnect network, Item 2 above, utilized MTU=9000. All interfaces associated with dataflow are visible to all other interfaces associated with dataflow.
| Item No | Switch Name | Switch Type | Total Port Count | Used Port Count | Notes |
|---|---|---|---|---|---|
| 1 | Cisco Nexus C9516 | 25GbE and 100 GbE Switch | 96 | 96 | 72 client-side 25 GbE data connections; 24 storage-side 100 GbE data connections. Only the ports on the Cisco Nexus C9516 used for the solution under test are included in the total port count. |
| 2 | Cisco Nexus C3232 | 100 GbE Switch | 32 | 24 | For Cluster Interconnect |
| Item No | Qty | Type | Location | Description | Processing Function |
|---|---|---|---|---|---|
| 1 | 24 | CPU | Storage Controller | 2.10 GHz Intel Xeon Platinum 8160 | NFS, TCP/IP, RAID and Storage Controller functions |
| 2 | 72 | CPU | Client | 3.70 GHz Intel Xeon E5-1630 | NFS Client, Linux OS |
Each of the 12 NetApp AFF A800 Storage Controllers contains 2 Intel Xeon 8160 processors with 24 cores each; 2.10 GHz, hyperthreading disabled. Each client contains 1 Intel Xeon E5-1630 processor with 4 cores at 3.70 GHz, hyperthreading enabled.
| Description | Size in GiB | Number of Instances | Nonvolatile | Total GiB |
|---|---|---|---|---|
| Main Memory for each NetApp AFF A800 HA Pair | 1280 | 6 | V | 7680 |
| NVDIMM (NVRAM) Memory for each NetApp AFF A800 HA pair | 128 | 6 | NV | 768 |
| Memory for each client; 71 of these drove the workload | 64 | 72 | V | 4608 | Grand Total Memory Gibibytes | 13056 |
Each storage controller has main memory that is used for the operating system and caching filesystem data. Each controller also has NVRAM; See "Stable Storage" for more information.
The AFF A800 utilizes non-volatile battery-backed memory (NVRAM) for write caching. When a file-modifying operation is processed by the filesystem (WAFL) it is written to system memory and journaled into a non-volatile memory region backed by the NVRAM. This memory region is often referred to as the WAFL NVLog (non-volatile log). The NVLog is mirrored between nodes in an HA pair and protects the filesystem from any SPOF (single-point-of-failure) until the data is de-staged to disk via a WAFL consistency point (CP). In the event of an abrupt failure, data which was committed to the NVLog but has not yet reached its final destination (disk) is read back from the NVLog and subsequently written to disk via a CP.
All clients accessed the FlexGroup from all the available network
interfaces.
Unlike a general-purpose operating system, ONTAP does not
provide mechanisms for non-administrative users to run third-party code. Due to
this behavior, ONTAP is not affected by either the Spectre or Meltdown
vulnerabilities. The same is true of all ONTAP variants including both ONTAP
running on FAS/AFF hardware as well as virtualized ONTAP products such as ONTAP
Select and ONTAP Cloud. In addition, FAS/AFF BIOS firmware does not provide a
mechanism to run arbitrary code and thus is not susceptible to either the
Spectre or Meltdown attacks. More information is available from
https://security.netapp.com/advisory/ntap-20180104-0001/.
None of the
components used to perform the test were patched with Spectre or Meltdown
patches (CVE-2017-5754,CVE-2017-5753,CVE-2017-5715).
ONTAP Storage Efficiency techniques including inline compression and inline deduplication were enabled by default, and were active during this test. Standard data protection features, including background RAID and media error scrubbing, software validated RAID checksum, and double disk failure protection via double parity RAID (RAID-DP) were enabled during the test. WARMUP_TIME has been set to a value of 600 seconds.
Please reference the configuration diagram. 72 clients were used to generate the workload; 1 client acted as Prime Client to control the 71 other clients. Each client used one 25 GbE connection, through a Cisco Nexus C9516 switch. Each storage HA pair had 4x 100 GbE connections to the data switch. The filesystem consisted of one ONTAP FlexGroup. The clients mounted the FlexGroup volume as an NFSv3 filesystem. The ONTAP cluster provided access to the FlexGroup volume on every 100 GbE port connected to the data switch (16 ports total). Each cluster node had 2 Logical Interfaces (LIFs) per 100GbE Port, for a total of 4 LIFs per node, for a total of 48 LIFs for the AFF A800 cluster. Each client created mount points across those 48 LIFs symmetrically.
None
NetApp is a registered trademark and "Data ONTAP", "FlexGroup", and "WAFL" are trademarks of NetApp, Inc. in the United States and other countries. All other trademarks belong to their respective owners and should be treated as such.
Generated on Wed Mar 13 16:19:32 2019 by SpecReport
Copyright © 2016-2019 Standard Performance Evaluation Corporation
