SPECstorage(TM) Solution 2020_genomics Result WEKAIO, Inc. : WEKA - Public Cloud Reference SPECstorage Solution = 2200 Jobs (Overall Response Time = 0.59 msec) 2020_genomics =============================================================================== Performance =========== Business Average Metric Latency (Jobs) (msec) Jobs Ops/Sec Jobs MB/Sec ------------ ------------ ------------ ------------ 220 0.4 220006 18680 440 0.4 440012 37378 660 0.4 660019 56062 880 0.4 880025 74747 1100 0.5 1100032 93424 1320 0.5 1320041 112129 1540 0.6 1540048 130807 1760 0.6 1760057 149492 1980 0.8 1980066 168195 2200 1.7 2200089 186853 =============================================================================== Product and Test Information ============================ +---------------------------------------------------------------+ | WEKA - Public Cloud Reference | +---------------------------------------------------------------+ Tested by WEKAIO, Inc. Hardware Available February 2024 Software Available February 2024 Date Tested February 2024 License Number 4553 Licensee Locations Campbell, CA USA The WEKA Data Platform is purpose-built to seamlessly and sustainably deliver speed, simplicity, and scale that meets the needs of modern enterprises and research organizations without compromise. Its advanced, software-defined architecture supports next-generation workloads in virtually any location with cloud simplicity and on-premises performance. Solution Under Test Bill of Materials ===================================== Item No Qty Type Vendor Model/Name Description ---- ---- ---------- ---------- ---------- ----------------------------------- 1 40 AWS AWS Standard i WEKA storage backends - AWS Virtual 3en.24xlar Standard i3en.24xlarge Virtual Machine ge Machine (96 vCPUs, 768 GiB memory, 100 Gbps networking) 2 40 AWS AWS Standard c Amazon Linux clients with WEKA Virtual 5n.18xlarg POSIX-compliant filesystem driver - Machine e AWS Standard C5n.18xlarge Virtual Machine (72 vCPUs, 192 GiB memory, 100 Gbps networking) Configuration Diagrams ====================== 1) storage2020-20240229-00069.config1.png (see SPECstorage Solution 2020 results webpage) Component Software ================== Item Name and No Component Type Version Description ---- ------------ ------------ ------------ ----------------------------------- 1 WekaFS File System Version The WEKA filesystem (WekaFS) 4.2.8.66 redefines storage solutions with its software-only approach and addresses common storage challenges by removing performance bottlenecks, making it suitable for environments requiring low latency, high performance, and cloud scalability. 2 Weka Backend Storage OS Amazon Linux Operating System (OS) for 40 Node v2 kernel storage virtual machines using AWS 5.10 i3en.24xlarge instances 3 Clients Client OS Amazon Linux Operating System (OS) for 40 client v2 kernel virtual machines using AWS 5.10 c5n.18xlarge instances Hardware Configuration and Tuning - Virtual =========================================== +----------------------------------------------------------------------+ | Component Name | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Enhanced Enabled Both c5n.18xlarge and i3en.24xlarge Networking instances have 100 Gbps connectivity Hardware Configuration and Tuning Notes --------------------------------------- None Software Configuration and Tuning - Virtual =========================================== +----------------------------------------------------------------------+ | Networking | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Jumbo Frames 9001 WekaFS recommended MTU for AWS. num_cores 7 WekaFS mount option to designate the number of frontend cores to allocate for the client. Software Configuration and Tuning Notes --------------------------------------- The single filesystem was attached via a single mount per client. The mount string used was "mount -t wekafs -o net=eth1 -o net=eth2 -o net=eth3 -o net=eth4 -o net=eth5 -o net=eth6 -o net=eth7 -o num_cores=7 -o readahead_kb=4096 -o mgmt_ip=10.0.3.153 internal-spec-boni-lb-344769224.us-east-1.elb.amazonaws.com/default /mnt/weka" The mount specifies that 7 WEKA front ends (cores) were used on the clients. Service SLA Notes ----------------- None Storage and Filesystems ======================= Item Stable No Description Data Protection Storage Qty ---- ------------------------------------- ------------------ -------- ----- 1 WEKA Filesystem with 1.7 PB of usable WEKA 16+2 Amazon 320 storage distributed data EC2 7TB protection NVMe Drives Number of Filesystems 1 Total Capacity 1.66 PiB Filesystem Type WekaFS Filesystem Creation Notes ------------------------- Each storage backend has eight 7TB Amazon EC2 NVMe drives attached to it, which are dedicated to the WEKA filesystem created. The storage cluster consisted of 40 storage servers with a single WEKA filesystem created in the cluster. Each client mounted the WEKA filesystem with a single mount. Storage and Filesystem Notes ---------------------------- None Transport Configuration - Virtual ================================= Item Number of No Transport Type Ports Used Notes ---- --------------- ---------- ----------------------------------------------- 1 100 Gbps 40 Used by WEKA Backend Server Nodes Virtual Nic 2 100 Gbps 40 Used by WEKA Frontend Client Nodes Virtual Nic Transport Configuration Notes ----------------------------- None Switches - Virtual ================== Total Used Item Port Port No Switch Name Switch Type Count Count Notes ---- -------------------- --------------- ------ ----- ------------------------ 1 AWS 100 Gbps 40 40 Used by WEKA Backend Ethernet with Servers Enhanced Networking 2 AWS 100 Gbps 40 40 Used by WEKA Frontend Ethernet with Clients Enhanced Networking Processing Elements - Virtual ============================= Item No Qty Type Location Description Processing Function ---- ---- -------- -------------- ------------------------- ------------------- 1 3840 vCPU i3en.24xlarge 2.5 GHz Intel(R) Xeon(R) WekaFS, Network WEKA Backend Platinum 8259CL Communications, processors Storage Functions 2 2880 vCPU c5n.18xlarge 3 GHz Intel(R) Xeon(R) WekaFS, Network WEKA Frontend Platinum 8124M processors Communications, Storage Functions Processing Element Notes ------------------------ None Memory - Virtual ================ Size in Number of Description GiB Instances Nonvolatile Total GiB ------------------------- ---------- ---------- ------------ ------------ AWS i3en.24xlarge Virtual 768 40 V 30720 Machine Memory AWS c5n.18xlarge Virtual 192 40 V 7680 Machine Memory Grand Total Memory Gibibytes 38400 Memory Notes ------------ None Stable Storage ============== WekaFS does not use any internal memory to temporarily cache write data to the underlying storage system. All writes are commited directly to the storage disk, and protected via WekaFS Distributed Data Protection (16+2 in this case). There is no need for any RAM battery protection. In an event of a power failure a write in transit would not be acknowledged. Solution Under Test Configuration Notes ======================================= The system under test consisted of 40 x c5n.18xlarge AWS client virtual machines and 40 x i3en.24xlarge backend virtual machines. Both client and backend have 100Gbps network connectivity. Other Solution Notes ==================== Entire cloud deployment is automated using Weka Terraform Module. See https://docs.weka.io/install/aws/weka-installation-on-aws-using-terraform for more information. Dataflow ======== None Other Notes =========== weka debug override list: heartbeat_grace_msecs 20000 heartbeat_timeout_msecs 5000 Other Report Notes ================== The solution under test was a standard WekaFS cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. SUT not vulnerable to Spectre or Meltdown (CVE-2017-5754, CVE-2017-5753, CVE-2017-5715). =============================================================================== Generated on Wed Mar 6 17:56:10 2024 by SpecReport Copyright (C) 2016-2024 Standard Performance Evaluation Corporation