SPEC SFS®2014_swbuild Result
Copyright © 2016-2019 Standard Performance Evaluation Corporation
|
SPEC SFS®2014_swbuild ResultCopyright © 2016-2019 Standard Performance Evaluation Corporation |
| Oracle | SPEC SFS2014_swbuild = 1120 Builds |
|---|---|
| Oracle ZFS Storage ZS7-2 | Overall Response Time = 1.07 msec |
|
|
| Oracle ZFS Storage ZS7-2 | |
|---|---|
| Tested by | Oracle | Hardware Available | November 13, 2018 | Software Available | November 13, 2018 | Date Tested | October 2018 | License Number | 00073 | Licensee Locations | Redwood Shores, CA, USA |
The Oracle ZFS Storage ZS7-2 is a high-end high-performance all-flash storage system that offers enterprise-class NAS and SAN capabilities with industry-leading Oracle Database integration, in a cost-effective high-availability configuration. The Oracle ZFS Storage ZS7-2 provides simplified set up, management, and industry-leading storage analytics. The performance-optimized platform uses specialized Read and Write Flash caching devices in the hybrid storage configuration, for high-performance throughput and latency. The Oracle ZFS Storage ZS7-2 high-end can scale to 1.5TB Memory, 48 CPU cores per controller and 3.6 PB of all-flash storage. Oracle ZFS Storage Appliances deliver economic value with bundled data services for file and block-level protocols with connectivity over 40GbE, 10GbE, InfiniBand, and 32Gb FC. Data may be managed using Compression, Deduplication, Encryption, Thin provisioning, Real-Time Analytics, Virus Scan, Snapshots, ZFS RAID Data Protection, Snapshots, Remote Replication, NDMP, and High Availability Clustering.
| Item No | Qty | Type | Vendor | Model/Name | Description |
|---|---|---|---|---|---|
| 1 | 2 | Storage Controller | Oracle | Oracle ZFS Storage ZS7-2 | Oracle ZFS Storage ZS7-2, 2 x 2.10GHz Intel Xeon Platinum 8160 CPU. 1.5TB DDR4-2666 LRDIMM. 2 x 10TB SAS3 HGST boot drives. Support for SAS3, IB, 10GbE. |
| 2 | 48 | Memory | Oracle | Oracle ZFS Storage ZS7-2 | Oracle ZFS Storage ZS7-2, 48 x 64GB DDR4-2666 LRDIMM. Memory is order configurable, a total of 1.5TB was installed in each storage controller. |
| 3 | 6 | Storage Drive Enclosure | Oracle | Oracle Storage Drive Enclosure DE3-24P | 24 drive slot enclosure, SAS3 connected, 24 x 3TB HGST SSD. Dual PSU. |
| 4 | 6 | Storage Drive Enclosure | Oracle | Oracle Storage Drive Enclosure DE3-24P | 24 drive slot enclosure, SAS3 connected, 20 x 3TB HGST SSD and 4 X 200GB HGST SSD. Dual PSU. |
| 5 | 264 | SAS3 SSD | Oracle | 7118008 | 3TB HGST SSD. Drive selection is order configurable, a total of 264 x 3TB HGST SSD drives were installed across all Oracle Storage Drive Enclosure DE3-24P. |
| 6 | 24 | SAS3 SSD | Oracle | 7115942 | 200GB HGST SSD. Drive selection is order configurable, a total of 24 x 200GB HGST SSD drives were installed across all Oracle Storage Drive Enclosure DE3-24P. These Drives are used for write flash accelerators. |
| 7 | 8 | Client | Oracle | Oracle X6-2 | Oracle X6-2 Client Node, 2 x 2.10GHz Intel Xeon CPU E5-2699 v4. 512GB RAM. 2 x 10GbE. Used for benchmark load generation. |
| 8 | 8 | OS Drive | Oracle | 7093013 | 600GB HGST hard drive. 8 x 600GB HGST hard drives, one for each Oracle X6-2 Client Node was installed for OS boot drive. |
| 9 | 1 | Switch | Oracle | Oracle Switch ES2-64 | Oracle Switch ES2-64, high-performance, low-latency 10/40 Gb/sec Ethernet switch. |
| Item No | Component | Type | Name and Version | Description |
|---|---|---|---|---|
| 1 | Oracle ZFS Storage | Storage Controller OS | 8.8 | Oracle ZFS Storage OS for storage controllers. |
| 2 | Oracle Linux | Client Node OS | 7.3 | Oracle Linux OS for client nodes. |
| Oracle ZFS Storage ZS7-2 | Parameter Name | Value | Description |
|---|---|---|
| MTU | 9000 | Network Jumbo Frames |
| svcadm enable power | enabled | Oracle ZFS Storage Power Service |
| poweradm set administrative-authority=none | none | Oracle ZFS Storage Power Service | Oracle X6 Client Node | Parameter Name | Value | Description |
| MTU | 9000 | Network Jumbo Frames |
Oracle ZFS Storage ZS7-2 controllers and Oracle X6-2 client nodes both had 10Gb
ethernet ports set up to MTU of 9000 jumbo frames.
Power management
which controls Intel processor power states was set to administrative-authority
equal "none". Power management is controlled through the Oracle ZFS Storage
ZS7-2 managment BUI.
| Oracle X6-2 Client Nodes | Parameter Name | Value | Description |
|---|---|---|
| vers | 3 | NFS mount option set to version 3 |
| rsize,wsize | 1048576 | NFS mount option for data block size |
| sync | sync | NFS mount option set to sync io |
| net.ipv4.tcp_rmem, net.ipv4.tcp_wmem | 10000000 | Linux kernel tcp send and receive buffers |
| net.core.somaxconn | 65536 | Linux kernel maximum socket connections |
Tune the communications between Oracle X6-2 client nodes and the Oracle ZFS Storage ZS7-2 controllers over the 10Gb ethernet by optimizing amount of data transfer and minimum overhead. This includes setting the Oracle X6-2 clients mounts of the Oracle ZFS Storage ZS7-2 files systems to use sync io, read and write sizes to 1048576, along with increasing the Oracle X6-2 client send and receive buffers sizes to 10000000.
None
| Item No | Description | Data Protection | Stable Storage | Qty |
|---|---|---|---|---|
| 1 | 3.2TB SSD Oracle ZFS Storage ZS7-2 Data Pool Drives | RAID-10 | Yes | 264 |
| 2 | 200GB SSD Oracle ZFS Storage ZS7-2 Log Drives | None | Yes | 24 |
| 3 | 10TB HGST Oracle ZFS Storage ZS7-2 OS Drives | Mirrored | No | 4 |
| 4 | 600GB HGST Oracle X6-2 Client Node OS Drives | None | No | 8 |
| Number of Filesystems | 64 | Total Capacity | 366TiB | Filesystem Type | ZFS |
|---|
Two ZFS storage pools are created overall in the SUT (1 storage pool per Oracle ZFS Storage ZS7-2 controller). Each of the controller's storage pools are configured with 128 ssd drives, 12 write flash accelerator (log device) and 4 spare ssd drives. When configuring the storage pool via the administrative html interface of each Oracle ZFS Storage ZS7-2 storage controller, at the start you will be asked to select the number of disk drives and log devices to use per tray. The storage pools are set up to mirror the data (RAID-10) across all 128 data ssd drives (Note: When configuring storage pools on the Oracle ZFS Storage ZS7-2 controllers this is a data profile of Mirrored). The write flash accelerator in each storage pool is used for the ZFS Intent Log (ZIL). Each of the storage pools are configured with 32 ZFS filesystems. Since each controller is configured with 1 storage pool and each storage pool contains 32 ZFS filesystems, in total the SUT has 64 ZFS filesystems. There are 2 internal mirrored system disk drives per Oracle ZFS Storage ZS7-2 controller and are used only for the controllers core operating system. These drives are not used for data cache or storing user data.
All filesystems on both Oracle ZFS Storage ZS7-2 controllers are created with setting of the Database Record Size of 128KB. The logbias setting is set to latency for each filesystem. This is a common practice for storage solutions with the Oracle ZFS Storage ZS7-2 storage controllers.
| Item No | Transport Type | Number of Ports Used | Notes |
|---|---|---|---|
| 1 | 10GbE Ethernet | 16 | Each Oracle ZFS Storage ZS7-2 Controller uses 8x 10GbE Ethernet physical ports for dataflow |
| 2 | 10GbE Ethernet | 2 | Each Oracle ZFS Storage ZS7-2 Controller uses 1x 10GbE Ethernet physical port for management |
| 3 | 10GbE Ethernet | 16 | Each Oracle X6-2 Client Node uses 2x 10GbE Ethernet physical ports for dataflow |
| 4 | 10GbE Ethernet | 8 | Each Oracle X6-2 Client Node uses 1x 10GbE Ethernet physical port for management |
Each Oracle ZFS Storage ZS7-2 uses 8 active 10GbE Ethernet ports. Total Oracle
ZFS Storage ZS7-2 controllers use 16 ports active. In the event of controller
failure IP address will be taken over by surviving controller. All ports are
setup with the MTU size of 9000 on each of the 10 GbE ports. There is 1x 10GbE
port per controller assigned to the managment interface, this interface is only
used manage the controller and does not take part in dataflow.
The
Oracle X6-2 client nodes uses 2x 10 GbE Ethernet cards each for dataflow. Each
port is set to MTU of 9000. The Oracle X6-2 client nodes each use 1x 10GbE
Ethernet port for managment, these interfaces are not used for
dataflow.
Each of the 16 active physical 10GbE Ethernet ports are
assigned 4 vnic IP addresses on the Oracle X6-2 client nodes. Each of the 16
active physical 10GbE Ethernet ports on the Oracle ZFS Storage ZS7-2 are also
assigned 4 vnic IP addresses. On the Oracle ZFS Storage ZS7-2, vnics are
configured through the management BUI. On the Oracle X6-2 client nodes, vnics
are configured in Linux OS /etc/sysconfig/network-scripts. Please reference the
vnic diagram for IP layout.
| Item No | Switch Name | Switch Type | Total Port Count | Used Port Count | Notes |
|---|---|---|---|---|---|
| 1 | Oracle Switch ES2-64 | 10/40GbE Ethernet Switch | 46 | 32 | All ports set up for jumbo frame support |
| Item No | Qty | Type | Location | Description | Processing Function |
|---|---|---|---|---|---|
| 1 | 4 | CPU | Oracle ZFS Storage ZS7-2 | 2.10GHz Intel Xeon Platinum 8160 CPU | ZFS, TCP/IP, RAID/Storage Drivers, NFS |
| 2 | 16 | CPU | Oracle X6-2 Client Node | 2.20GHz Intel Xeon CPU E5-2699 v4 | TCP/IP, NFS |
Each Oracle ZFS Storage ZS7-2 controller contains 2 physical processors, each
with 24 processing cores.
Oracle X6-2 client contains 2 physical
processors, each with 22 processing cores.
| Description | Size in GiB | Number of Instances | Nonvolatile | Total GiB |
|---|---|---|---|---|
| Memory in Oracle ZFS Storage ZS7-2 | 1500 | 2 | V | 3000 |
| Memory in Oracle X6-2 clients | 768 | 8 | V | 6144 | Grand Total Memory Gibibytes | 9144 |
The Oracle ZFS Storage ZS7-2 controllers' main memory is used for the Adaptive
Replacement Cache (ARC), the data cache, and operating system memory.
Oracle X6-2 client memory is not used for storage or cache of the Oracle ZFS
Storage ZS7-2 controllers, just for the client use.
The Stable Storage requirement is guaranteed by the ZFS Intent Log (ZIL) which logs writes and other filesystem changing transactions to either a write flash accelerator or a disk drive. Writes and other filesystem changing transactions are not acknowledged until the data is written to stable storage. Since this is an active-active cluster high availability system, in the event of a controller failing or power loss, the other active controller can take over for the failed controller. Since the write flash accelerators or disk drives are located in the disk shelves and can be accessed via the 4 backend SAS channels from both controllers, the remaining active controller can complete any outstanding transactions using the ZIL. In the event of power loss to both controllers, the ZIL is used after power is restored to reinstate any writes and other filesystem changes.
The system under test are Oracle ZFS Storage ZS7-2, high end storage
controllers, setup in an active-active cluster configuration with failover
capabilities.
The Oracle X6-2 client nodes became EOL in February,
2018. For general availability third parties still sell the model with original
Oracle warranty and support. In addition this model line has been refreshed and
is available from Oracle as the Oracle X7-2.
The non-default WARMUP_TIME of 500 seconds was use for this benchmark run.
Please reference the SUT diagram. The 8 Oracle X6-2 client nodes are used for benchmark load generation. The Oracle X6-2 client nodes each mount 8 of the total 64 ZFS filesystems of the Oracle ZFS Storage ZS7-2 controllers via NFSv3. Half of the filesystems are shared from each Oracle ZFS Storage ZS7-2 controller. Each of the two Oracle ZFS Storage ZS7-2 controllers has 8x 10GbE Ethernet active ports for io dataflow, all are assigned separate subnets. Each Oracle X6-2 client node has 2x 10GbE Ethernet ports, accessing half of its nfs mounts through each ethernet port. There is a one-to-one match between the 16 total 10GbE Ethernet client ports to the 16 total 10GbE Ethernet controller ports used for io dataflow (non-management ports). So in effect, this spreads io load evenly across the filesystems mounts, network interfaces, and storage pools of the Oracle ZFS Storage ZS7-2 Cluster SUT.
Oracle and ZFS are registered trademarks of Oracle Corporation in the U.S. and/or other countries. Intel and Xeon are registered trademarks of the Intel Corporation in the U.S. and/or other countries.
The test sponsor attests, as of date of publication, that CVE-2017-5754 (Meltdown), CVE-2017-5753 (Spectre variant 1), and CVE-2017-5715 (Spectre variant 2) are mitigated in the system as tested and documented.
Generated on Wed Mar 13 16:27:02 2019 by SpecReport
Copyright © 2016-2019 Standard Performance Evaluation Corporation
