SPECstorage(TM) Solution 2020_vda Result SPECstorage(TM) : Reference Submission in the Public Cloud Subcommittee SPECstorage Solution = 200 Streams (Overall Response Time = 1.98 msec) 2020_vda =============================================================================== Performance =========== Business Average Metric Latency Streams Streams (Streams) (msec) Ops/Sec MB/Sec ------------ ------------ ------------ ------------ 10 1.6 100 46 20 1.6 200 91 30 1.7 300 138 40 1.8 400 184 50 1.8 500 229 60 1.9 600 277 70 1.9 700 322 80 1.9 800 369 90 2.0 900 414 100 2.0 1000 460 110 2.0 1100 506 120 2.1 1200 552 130 2.1 1300 599 140 2.1 1400 644 150 2.1 1501 691 160 2.2 1601 737 170 2.2 1701 783 180 2.2 1801 827 190 2.2 1901 874 200 2.3 2001 921 =============================================================================== Product and Test Information ============================ +---------------------------------------------------------------+ | Reference Submission in the Public Cloud | +---------------------------------------------------------------+ Tested by SPECstorage(TM) Subcommittee Hardware Available November 2020 Software Available October 2020 Date Tested November 2020 License Number 4553 Licensee Locations Campbell, California Weka has developed a software-defined storage architecture that offers complete hardware independence while delivering an extremely performant solution that offers the flexibility of multiple protocol support. The design philosophy behind the Weka file system (WekaFS) was to create a single storage architecture, that runs on industry standard servers or on instances in the public cloud, with the performance of all-flash arrays, the simplicity and feature set of network-attached storage (NAS), and the scalability and economics of the cloud. At the heart of the design is WekaFS, a fully distributed, parallel file system that was written entirely from scratch to deliver the highest-performance file and object services leveraging NVMe flash as its primary storage tier between the application and the storage. Internally, and transparent to the application layer, the software seamlessly expands the namespace to a hard disk drive (HDD) layer built on any S3 compliant object storage system. Solution Under Test Bill of Materials ===================================== Item No Qty Type Vendor Model/Name Description ---- ---- ---------- ---------- ---------- ----------------------------------- 1 8 AWS EC2 AWS c5n.9xlarg Amazon C5n EC2 instances are ideal Instances e for high compute applications (including High Performance Computing (HPC) workloads, data lakes, and network appliances, c5n.9xlarge instances have 36 vCPU, 96GiB of memory, 50Gbps networking 2 8 AWS EC2 AWS i3en.12xla Amazon i3en family of EC2 instances Instances rge provides dense Non-Volatile Memory Express (NVMe)SSD instance storage optimized for low latency, high random I/O performance, high sequential disk throughput, i3en.12xlarge instances have 48 vCPU, 384GiB of memory, 4x7500GB NVMe SSDs, 50Gbps networking Configuration Diagrams ====================== 1) storage2020-20201210-00011.config1.png (see SPECstorage Solution 2020 results webpage) Component Software ================== Item Name and No Component Type Version Description ---- ------------ ------------ ------------ ----------------------------------- 1 WekaFS File System Version WekaFS is a parallel and 3.8.3 distributed POSIX compliant file system that scales across NVMe enabled compute nodes, creating a single storage cluster that delivers high performance access to data. Each node in the cluster contains equal amount of data and metadata for optimal parallel file access and performance. 2 Amazon Linux Operating Version The Amazon Linux AMI is a supported AMI System 2017.09 and maintained Linux image provided by Amazon Web Services for use on Amazon Elastic Compute Cloud (Amazon EC2).It is deployed on the 8 i3en.12xlarge storage nodes. 3 Amazon Linux Operating Version Amazon Linux 2 is the next 2 System 2017.09 generation of Amazon Linux, from Amazon Web Services (AWS). It is deployed on the 8 c5n.9xlarge compute nodes. Hardware Configuration and Tuning - Virtual =========================================== +----------------------------------------------------------------------+ | Storage Nodes | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- SR-IOV Enabled Enables CPU virtualization technology Hardware Configuration and Tuning Notes --------------------------------------- SR-IOV was enabled on EC2 instance BIOS. No additional hardware tuning was required. The MTU is required and valid for all environments. Software Configuration and Tuning - Virtual =========================================== +----------------------------------------------------------------------+ | Component Name | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Jumbo Frames 9001 Enables jumbo frames up to 9001 bytes of Ethernet frames Software Configuration and Tuning Notes --------------------------------------- None Service SLA Notes ----------------- AWS uses commercially reasonable efforts to make the Included Products and Services each available with a Monthly Uptime Percentage of at least 99.99%, in each case during any monthly billing cycle. (Monthly Uptime Percentage) is calculated by subtracting from 100% the percentage of minutes during the month in which any of the Included Products and Services, as applicable, was in the state of "Region Unavailable". Storage and Filesystems ======================= Item Stable No Description Data Protection Storage Qty ---- ------------------------------------- ------------------ -------- ----- 1 Amazon EC2 i3en.12xlarge instances 4+2 Yes 8 with WekaFS as a shared file system, each with 4 x 7,500GB NVMe SSDs Number of Filesystems 1 Total Capacity 98 TiB Filesystem Type WekaFS Filesystem Creation Notes ------------------------- WekaFS is deployed on Amazon through a cloud formation template that can be launched from Marketplace. The Weka software is installed and the file system is configured as part of the cloud formation process. No additional file system creation steps are required. Storage and Filesystem Notes ---------------------------- None Transport Configuration - Virtual ================================= Item Number of No Transport Type Ports Used Notes ---- --------------- ---------- ----------------------------------------------- 1 50Gbps Ethernet 16 Virtual network Virtual NIC Transport Configuration Notes ----------------------------- None Switches - Virtual ================== Total Used Item Port Port No Switch Name Switch Type Count Count Notes ---- -------------------- --------------- ------ ----- ------------------------ 1 AWS 50Gbs Ethernet 16 16 Virtual network with Enhanced Networking Processing Elements - Virtual ============================= Item No Qty Type Location Description Processing Function ---- ---- -------- -------------- ------------------------- ------------------- 1 384 vCPU i3en.12xlarge 3.1 GHz all core turbo WekaFS, RAID, storage Intel® Xeon® Scalable Network instances (Skylake) processors communication, Storage functions 2 288 vCPU c5n.9xlarge 3.0 GHz Intel® Xeon® WekaFS Client and client Platinum processors benchmark instances Processing Element Notes ------------------------ None Memory - Virtual ================ Size in Number of Description GiB Instances Nonvolatile Total GiB ------------------------- ---------- ---------- ------------ ------------ AWS EC2 i3en.12xlarge 384 8 V 3072 instance memory AWS EC2 c5n.9xlarge 96 8 V 768 instance memory Grand Total Memory Gibibytes 3840 Memory Notes ------------ None Stable Storage ============== WekaFS uses the EC2 instances which provide stable storage. The data is journeled as well as UPS protected across multiple layers of the AWS infrastructure. Solution Under Test Configuration Notes ======================================= The solution under test was a standard WekaFS enabled cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. None of the components used to perform the test were patched with Spectre or Meltdown patches Other Solution Notes ==================== None Dataflow ======== None Other Notes =========== None Other Report Notes ================== None =============================================================================== Generated on Thu Dec 17 10:48:53 2020 by SpecReport Copyright (C) 2016-2020 Standard Performance Evaluation Corporation