SPECstorage(TM) Solution 2020_ai_image Result SPECstorage(TM) : Reference Submission in the Public Cloud Subcommittee SPECstorage Solution = 190 AI_Jobs (Overall Response Time = 1.26 msec) 2020_ai_image =============================================================================== Performance =========== Business Average Metric Latency AI_Jobs AI_Jobs (AI_Jobs) (msec) Ops/Sec MB/Sec ------------ ------------ ------------ ------------ 10 0.6 4350 978 20 0.6 8701 1955 30 0.6 13051 2935 40 0.7 17402 3912 50 0.7 21752 4892 60 0.7 26103 5868 70 0.8 30454 6847 80 0.9 34804 7823 90 1.0 39155 8802 100 1.1 43505 9780 110 1.2 47856 10759 120 1.2 52206 11737 130 1.4 56557 12716 140 1.4 60908 13692 150 1.6 65258 14668 160 1.8 69609 15647 170 2.2 73959 16624 180 2.5 78310 17601 190 4.4 82409 18560 =============================================================================== Product and Test Information ============================ +---------------------------------------------------------------+ | Reference Submission in the Public Cloud | +---------------------------------------------------------------+ Tested by SPECstorage(TM) Subcommittee Hardware Available November 2020 Software Available October 2020 Date Tested November 2020 License Number 4553 Licensee Locations Campbell, California Weka has developed a software-defined storage architecture that offers complete hardware independence while delivering an extremely performant solution that offers the flexibility of multiple protocol support. The design philosophy behind the Weka file system (WekaFS) was to create a single storage architecture, that runs on industry standard servers or on instances in the public cloud, with the performance of all-flash arrays, the simplicity and feature set of network-attached storage (NAS), and the scalability and economics of the cloud. At the heart of the design is WekaFS, a fully distributed, parallel file system that was written entirely from scratch to deliver the highest-performance file and object services leveraging NVMe flash as its primary storage tier between the application and the storage. Internally, and transparent to the application layer, the software seamlessly expands the namespace to a hard disk drive (HDD) layer built on any S3 compliant object storage system. Solution Under Test Bill of Materials ===================================== Item No Qty Type Vendor Model/Name Description ---- ---- ---------- ---------- ---------- ----------------------------------- 1 8 AWS EC2 AWS c5n.9xlarg Amazon C5n EC2 instances are ideal Instances e for high compute applications (including High Performance Computing (HPC) workloads, data lakes, and network appliances, c5n.9xlarge instances have 36 vCPU, 96GiB of memory, 50Gbps networking 2 8 AWS EC2 AWS i3en.12xla Amazon i3en family of EC2 instances Instances rge provides dense Non-Volatile Memory Express (NVMe)SSD instance storage optimized for low latency, high random I/O performance, high sequential disk throughput, i3en.12xlarge instances have 48 vCPU, 384GiB of memory, 4x7500GB NVMe SSDs, 50Gbps networking Configuration Diagrams ====================== 1) storage2020-20201210-00007.config1.png (see SPECstorage Solution 2020 results webpage) Component Software ================== Item Name and No Component Type Version Description ---- ------------ ------------ ------------ ----------------------------------- 1 WekaFS File System Version WekaFS is a parallel and 3.8.3 distributed POSIX compliant file system that scales across NVMe enabled compute nodes, creating a single storage cluster that delivers high performance access to data. Each node in the cluster contains equal amount of data and metadata for optimal parallel file access and performance. 2 Amazon Linux Operating Version The Amazon Linux AMI is a supported AMI System 2017.09 and maintained Linux image provided by Amazon Web Services for use on Amazon Elastic Compute Cloud (Amazon EC2).It is deployed on the 8 i3en.12xlarge storage nodes. 3 Amazon Linux Operating Version Amazon Linux 2 is the next 2 System 2017.09 generation of Amazon Linux, from Amazon Web Services (AWS). It is deployed on the 8 c5n.9xlarge compute nodes. Hardware Configuration and Tuning - Virtual =========================================== +----------------------------------------------------------------------+ | Storage Nodes | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- SR-IOV Enabled Enables CPU virtualization technology Hardware Configuration and Tuning Notes --------------------------------------- SR-IOV was enabled on EC2 instance BIOS. No additional hardware tuning was required. The MTU is required and valid for all environments. Software Configuration and Tuning - Virtual =========================================== +----------------------------------------------------------------------+ | Component Name | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Jumbo Frames 9001 Enables jumbo frames up to 9001 bytes of Ethernet frames Software Configuration and Tuning Notes --------------------------------------- None Service SLA Notes ----------------- AWS uses commercially reasonable efforts to make the Included Products and Services each available with a Monthly Uptime Percentage of at least 99.99%, in each case during any monthly billing cycle. (Monthly Uptime Percentage) is calculated by subtracting from 100% the percentage of minutes during the month in which any of the Included Products and Services, as applicable, was in the state of "Region Unavailable". Storage and Filesystems ======================= Item Stable No Description Data Protection Storage Qty ---- ------------------------------------- ------------------ -------- ----- 1 Amazon EC2 i3en.12xlarge instances 4+2 Yes 8 with WekaFS as a shared file system, each with 4 x 7,500GB NVMe SSDs Number of Filesystems 1 Total Capacity 98 TiB Filesystem Type WekaFS Filesystem Creation Notes ------------------------- WekaFS is deployed on Amazon through a cloud formation template that can be launched from Marketplace. The Weka software is installed and the file system is configured as part of the cloud formation process. No additional file system creation steps are required. Storage and Filesystem Notes ---------------------------- None Transport Configuration - Virtual ================================= Item Number of No Transport Type Ports Used Notes ---- --------------- ---------- ----------------------------------------------- 1 50Gbps Ethernet 16 Virtual network Virtual NIC Transport Configuration Notes ----------------------------- None Switches - Virtual ================== Total Used Item Port Port No Switch Name Switch Type Count Count Notes ---- -------------------- --------------- ------ ----- ------------------------ 1 AWS 50Gbs Ethernet 16 16 Virtual network with Enhanced Networking Processing Elements - Virtual ============================= Item No Qty Type Location Description Processing Function ---- ---- -------- -------------- ------------------------- ------------------- 1 384 vCPU i3en.12xlarge 3.1 GHz all core turbo WekaFS, RAID, storage Intel® Xeon® Scalable Network instances (Skylake) processors communication, Storage functions 2 288 vCPU c5n.9xlarge 3.0 GHz Intel® Xeon® WekaFS Client and client Platinum processors benchmark instances Processing Element Notes ------------------------ None Memory - Virtual ================ Size in Number of Description GiB Instances Nonvolatile Total GiB ------------------------- ---------- ---------- ------------ ------------ AWS EC2 i3en.12xlarge 384 8 V 3072 instance memory AWS EC2 c5n.9xlarge 96 8 V 768 instance memory Grand Total Memory Gibibytes 3840 Memory Notes ------------ None Stable Storage ============== WekaFS uses the EC2 instances which provide stable storage. The data is journeled as well as UPS protected across multiple layers of the AWS infrastructure. Solution Under Test Configuration Notes ======================================= The solution under test was a standard WekaFS enabled cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. None of the components used to perform the test were patched with Spectre or Meltdown patches Other Solution Notes ==================== None Dataflow ======== None Other Notes =========== None Other Report Notes ================== None =============================================================================== Generated on Thu Dec 17 10:40:40 2020 by SpecReport Copyright (C) 2016-2020 Standard Performance Evaluation Corporation