SPECstorage™ Solution 2020_eda_blended Result

Copyright © 2016-2024 Standard Performance Evaluation Corporation

WEKAIO, Inc. SPECstorage Solution 2020_eda_blended = 6310 Job_Sets
WEKA - Public Cloud Reference Overall Response Time = 0.87 msec


Performance

Business
Metric
(Job_Sets)
Average
Latency
(msec)
Job_Sets
Ops/Sec
Job_Sets
MB/Sec
6310.4662839654582
12620.4985679319163
18930.49385189513744
25240.534113586818329
31550.589141982622908
37860.665170379827491
44170.756198776332073
50480.875227172736653
56791.233255569341239
63104.022283674945757
Performance Graph


Product and Test Information

WEKA - Public Cloud Reference
Tested byWEKAIO, Inc.
Hardware AvailableFebruary 2024
Software AvailableFebruary 2024
Date TestedFebruary 2024
License Number4553
Licensee LocationsCampbell, CA USA

The WEKA Data Platform is purpose-built to seamlessly and sustainably deliver speed, simplicity, and scale that meets the needs of modern enterprises and research organizations without compromise. Its advanced, software-defined architecture supports next-generation workloads in virtually any location with cloud simplicity and on-premises performance.

Solution Under Test Bill of Materials

Item NoQtyTypeVendorModel/NameDescription
140AWS Virtual MachineAWSStandard i3en.24xlargeWEKA storage backends - AWS Standard i3en.24xlarge Virtual Machine (96 vCPUs, 768 GiB memory, 100 Gbps networking)
240AWS Virtual MachineAWSStandard c5n.18xlargeAmazon Linux clients with WEKA POSIX-compliant filesystem driver - AWS Standard C5n.18xlarge Virtual Machine (72 vCPUs, 192 GiB memory, 100 Gbps networking)

Configuration Diagrams

  1. WEKA in AWS

Component Software

Item NoComponentTypeName and VersionDescription
1WekaFSFile SystemVersion 4.2.8.66The WEKA filesystem (WekaFS) redefines storage solutions with its software-only approach and addresses common storage challenges by removing performance bottlenecks, making it suitable for environments requiring low latency, high performance, and cloud scalability.
2Weka Backend NodeStorage OSAmazon Linux v2 kernel 5.10Operating System (OS) for 40 storage virtual machines using AWS i3en.24xlarge instances
3ClientsClient OSAmazon Linux v2 kernel 5.10Operating System (OS) for 40 client virtual machines using AWS c5n.18xlarge instances

Hardware Configuration and Tuning - Virtual

Component Name
Parameter NameValueDescription
Enhanced NetworkingEnabledBoth c5n.18xlarge and i3en.24xlarge instances have 100 Gbps connectivity

Hardware Configuration and Tuning Notes

None

Software Configuration and Tuning - Virtual

Networking
Parameter NameValueDescription
Jumbo Frames9001WekaFS recommended MTU for AWS.
num_cores7WekaFS mount option to designate the number of frontend cores to allocate for the client.

Software Configuration and Tuning Notes

The single filesystem was attached via a single mount per client. The mount string used was "mount -t wekafs -o net=eth1 -o net=eth2 -o net=eth3 -o net=eth4 -o net=eth5 -o net=eth6 -o net=eth7 -o num_cores=7 -o readahead_kb=4096 -o mgmt_ip=10.0.3.153 internal-spec-boni-lb-344769224.us-east-1.elb.amazonaws.com/default /mnt/weka" The mount specifies that 7 WEKA front ends (cores) were used on the clients.

Service SLA Notes

None

Storage and Filesystems

Item NoDescriptionData ProtectionStable StorageQty
1WEKA Filesystem with 1.66 PiB of usable storageWEKA 16+2 distributed data protectionAmazon EC2 7TB NVMe Drives320
Number of Filesystems1
Total Capacity1.66 PiB
Filesystem TypeWekaFS

Filesystem Creation Notes

Each storage backend has eight 7TB Amazon EC2 NVMe drives attached to it, which are dedicated to the WEKA filesystem created. The storage cluster consisted of 40 storage servers with a single WEKA filesystem created in the cluster. Each client mounted the WEKA filesystem with a single mount.

Storage and Filesystem Notes

None

Transport Configuration - Virtual

Item NoTransport TypeNumber of Ports UsedNotes
1100 Gbps Virtual Nic40Used by WEKA Backend Server Nodes
2100 Gbps Virtual Nic40Used by WEKA Frontend Client Nodes

Transport Configuration Notes

None

Switches - Virtual

Item NoSwitch NameSwitch TypeTotal Port CountUsed Port CountNotes
1AWS100 Gbps Ethernet with Enhanced Networking4040Used by WEKA Backend Servers
2AWS100 Gbps Ethernet with Enhanced Networking4040Used by WEKA Frontend Clients

Processing Elements - Virtual

Item NoQtyTypeLocationDescriptionProcessing Function
13840vCPUi3en.24xlarge WEKA Backend2.5 GHz Intel(R) Xeon(R) Platinum 8259CL processorsWekaFS, Network Communications, Storage Functions
22880vCPUc5n.18xlarge WEKA Frontend3 GHz Intel(R) Xeon(R) Platinum 8124M processorsWekaFS, Network Communications, Storage Functions

Processing Element Notes

None

Memory - Virtual

DescriptionSize in GiBNumber of InstancesNonvolatileTotal GiB
AWS i3en.24xlarge Virtual Machine Memory76840V30720
AWS c5n.18xlarge Virtual Machine Memory19240V7680
Grand Total Memory Gibibytes38400

Memory Notes

None

Stable Storage

WekaFS does not use any internal memory to temporarily cache write data to the underlying storage system. All writes are commited directly to the storage disk, and protected via WekaFS Distributed Data Protection (16+2 in this case). There is no need for any RAM battery protection. In an event of a power failure a write in transit would not be acknowledged.

Solution Under Test Configuration Notes

The system under test consisted of 40 x c5n.18xlarge AWS client virtual machines and 40 x i3en.24xlarge backend virtual machines. Both client and backend have 100Gbps network connectivity.

Other Solution Notes

Entire cloud deployment is automated using Weka Terraform Module. See https://docs.weka.io/install/aws/weka-installation-on-aws-using-terraform for more information.

Dataflow

None

Other Notes

weka debug override list: heartbeat_grace_msecs 20000 heartbeat_timeout_msecs 5000

Other Report Notes

The solution under test was a standard WekaFS cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. SUT not vulnerable to Spectre or Meltdown (CVE-2017-5754, CVE-2017-5753, CVE-2017-5715).


Generated on Wed Mar 6 17:45:16 2024 by SpecReport
Copyright © 2016-2024 Standard Performance Evaluation Corporation