SPEC Cloud IaaS 2016 Benchmark
Copyright © 2016 Standard Performance Evaluation Corporation
Cloud Vendor: Dell Inc.
Cloud Type / SUT Type: private/whitebox
Hardware Platform: x86_64
Hypervisor: VMware vSphere 6.5
Cloud Infrastructure: VMware Integrated Openstack 3.1 (Mitaka)		 Scalability: 78.5 @ 72 Application Instances
Elasticity: 87.4%
Mean Instance Provisioning Time: 25s
Tested by: Dell Inc. SPEC Licence Number : 55 Test Date : May-2017
Performance Sections
Performance Summary
Performance Details
Validation, Errors, and Issues
Glossary of Terms		 SUT Configuration Sections
Instance Configuration
Cloud Configuration
Network Configuration
Storage Configuration		 Elasticity + Scalability Phase Date/Time and Test Region
Elasticity + Scalability Start Time: 2017-04-14_04:28:24_UTC
Elasticity + Scalability End Time: 2017-04-14_09:02:48_UTC
Test Region: US Central Time Zone
Cloud Informational Metrics
AI Provisioning Success: 100.00%
AI Run Success: 94.52
Total Instances: 468
Reference ID: ALLRUN04140320

Performance Summary

Baseline Summary Results for YCSB
Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) AI Provisioning Time (s)
Average 11887.9 1.830 2.963 121.2

Baseline Summary Results for KMeans
Completion Time (s) AI Provisioning Time (s)
Average 151.9 102.0

Scalability/Elasticity Summary Results for YCSB for 36 Valid AIs
Av. Throughput (ops/s) Av. Insert Latency 99% (ms) Av. Read Latency 99% (ms) Av. Provisioning Time (s) Scalability Elasticity (%)
9588.1 1.984 3.257 113.5 42.24 89.59

Scalability/Elasticity Summary Results for KMeans for 36 Valid AIs
Av. Completion Time (s) Av. Provisioning Time (s) Scalability Elasticity (%)
189.0 91.6 36.30 85.26

YCSB Summary Results by AI
AI name Run count Run code Av. Throughput (ops/s) Av. Insert Latency 99% (ms) Av. Read Latency 99% (ms) AI. Provisioning time (s) AI. Prov. Initiated from Elasticity Start (s)
ai_11 66 1 10782.8 1.795 2.929 104.0 3.0
ai_14 66 1 10464.3 1.686 2.929 109.0 570.0
ai_16 67 1 10799.2 1.699 2.962 109.0 931.0
ai_17 65 0 10207.1 1.733 2.727 108.0 1410.0
ai_19 64 1 10479.6 1.637 2.842 119.0 1766.0
ai_21 65 0 10885.7 1.626 2.483 110.0 2095.0
ai_24 64 0 10338.0 1.558 2.700 110.0 2671.0
ai_26 63 0 10361.9 1.883 2.897 111.0 3174.0
ai_28 60 0 10107.4 1.844 3.052 106.0 3572.0
ai_30 60 0 10012.6 1.778 3.138 111.0 3886.0
ai_32 55 0 9651.2 1.776 3.098 107.0 4479.0
ai_34 62 0 11681.5 1.567 2.962 109.0 4905.0
ai_35 49 0 9453.7 2.049 3.291 110.0 5397.0
ai_38 46 0 9500.3 1.875 2.982 105.0 5935.0
ai_40 44 0 9514.3 2.050 3.403 111.0 6506.0
ai_42 41 0 9374.8 2.065 3.309 118.0 7042.0
ai_44 40 0 9748.9 1.867 2.923 116.0 7408.0
ai_46 37 0 9159.4 2.114 3.641 108.0 7954.0
ai_48 35 0 9262.5 2.004 3.386 123.0 8313.0
ai_50 34 0 9748.0 1.918 3.240 113.0 8884.0
ai_52 30 0 8982.5 2.100 3.261 110.0 9275.0
ai_54 30 0 9408.6 1.956 3.358 108.0 9674.0
ai_56 27 0 9331.5 1.939 3.283 117.0 10056.0
ai_58 26 0 9236.7 1.968 3.385 120.0 10440.0
ai_60 23 0 8900.9 2.205 3.701 120.0 10861.0
ai_62 25 0 11174.6 1.513 2.752 113.0 11447.0
ai_64 18 0 9301.0 1.903 3.194 115.0 12034.0
ai_66 15 0 8857.7 2.162 3.691 119.0 12608.0
ai_68 13 0 8708.7 2.171 3.507 114.0 13102.0
ai_70 11 0 8650.7 2.225 3.571 111.0 13513.0
ai_72 9 0 8827.4 2.311 3.403 121.0 13994.0
ai_74 8 0 8661.6 2.440 3.673 116.0 14412.0
ai_76 6 0 9332.0 2.118 3.436 114.0 14781.0
ai_78 4 0 8020.6 2.547 4.052 128.0 15190.0
ai_80 2 0 8333.4 2.487 3.848 120.0 15739.0
ai_82 1 0 7911.5 2.871 4.239 122.0 16063.0

KMeans Summary Results by AI
AI name Run count Run code Av. Completion Time (s) Av. Provisioning time (s) AI. Prov. Initiated from Elasticity Start (s)
ai_12 72 0 176.6 85.0 10.0
ai_13 71 0 174.3 84.0 416.0
ai_15 71 0 167.9 80.0 898.0
ai_18 67 0 171.2 90.0 1447.0
ai_20 61 0 182.5 81.0 1908.0
ai_22 59 0 189.3 81.0 2209.0
ai_23 57 0 187.7 87.0 2576.0
ai_25 57 0 180.9 82.0 2904.0
ai_27 55 0 180.1 79.0 3447.0
ai_29 53 0 176.3 88.0 3816.0
ai_31 52 0 175.5 86.0 4307.0
ai_33 49 0 179.3 87.0 4886.0
ai_36 44 0 189.7 88.0 5442.0
ai_37 45 0 179.1 85.0 5755.0
ai_39 40 0 195.0 84.0 6353.0
ai_41 42 0 173.4 91.0 6672.0
ai_43 37 0 188.6 104.0 7254.0
ai_45 34 0 195.2 90.0 7651.0
ai_47 32 0 197.7 85.0 8203.0
ai_49 31 0 188.7 100.0 8587.0
ai_51 28 0 199.2 92.0 9140.0
ai_53 27 0 192.8 92.0 9612.0
ai_55 25 0 194.0 93.0 9931.0
ai_57 22 0 205.9 90.0 10340.0
ai_59 21 0 197.2 93.0 10839.0
ai_61 19 0 199.2 107.0 11403.0
ai_63 17 0 194.9 102.0 11999.0
ai_65 15 0 184.6 95.0 12471.0
ai_67 13 0 212.4 100.0 12798.0
ai_69 11 0 188.5 106.0 13374.0
ai_71 8 0 225.5 94.0 13863.0
ai_73 8 0 211.1 96.0 14211.0
ai_75 7 0 168.1 103.0 14613.0
ai_77 5 0 174.2 96.0 15099.0
ai_79 3 0 250.3 94.0 15438.0
ai_81 1 0 157.0 108.0 15949.0
ai_83 0 2 0.0 99.0 16310.0

Validation, Errors, and Issues

Glossary of "Run code": 0 no error, 3 if an AI failed to provision, 2 if an AI that was provisioned but did not report any runs when the benchmark terminated - the AIs that are provisioning towards the end of elasticity + scalability phase may have zero runs, 1 for all other errors including AIs that were provisioned but did not report any result or reported results after QoS limits.

AIs that were issued but their provisioning was inprogress when the benchmark terminated are not shown.

Instance Configuration

Instance Configuration Hadoop Master
Parameter Value
instance_type_name m1.medium
cpu 2
memory 4GB
disk_size 43GB
disk_backed block storage
image_name cb_speccloud_kmeans_271
image_OS Ubuntu 14.04
metadata__hw_vif_model virtualVmxnet3
metadata__vmware_adaptertype lsiLogic
metadata__vmware_disktype streamOptimized
metadata__vmware_ostype ubuntu64Guest
Notes Flavor disk size modified to match image size of 40GiB

Instance Configuration Hadoop Slave
Parameter Value
instance_type_name m1.medium
cpu 2
memory 4GB
disk_size 43GB
disk_backed block storage
image_name cb_speccloud_kmeans_271
image_OS Ubuntu 14.04
metadata__hw_vif_model virtualVmxnet3
metadata__vmware_adaptertype lsiLogic
metadata__vmware_disktype streamOptimized
metadata__vmware_ostype ubuntu64Guest
Notes Flavor disk size modified to match image size of 40GiB

Instance Configuration YCSB
Parameter Value
instance_type_name m1.medium
cpu 2
memory 4GB
disk_size 43GB
disk_backed block storage
image_name cb_speccloud_cassandra_2111
image_OS Ubuntu 14.04
metadata__hw_vif_model virtualVmxnet3
metadata__vmware_adaptertype lsiLogic
metadata__vmware_disktype streamOptimized
metadata__vmware_ostype ubuntu64Guest
Notes Flavor disk size modified to match image size of 40GiB

Instance Configuration Cassandra
Parameter Value
instance_type_name m1.medium
cpu 2
memory 4GB
disk_size 43GB
image_name cb_speccloud_cassandra_2111
image_OS Ubuntu 14.04
metadata__hw_vif_model virtualVmxnet3
metadata__vmware_adaptertype lsiLogic
metadata__vmware_disktype streamOptimized
metadata__vmware_ostype ubuntu64Guest
Notes Flavor disk size modified to match image size of 40GiB

Cloud Configuration

Cloud High Level Information and Availability
Parameter Value
Name VMware Integrated Openstack 3.1
Type private
Sut_Type whitebox
Virtualized VMware vSphere 6.5
Manager_Protocol VMware Integrated Openstack 3.1
Manager_Version (Mitaka)
Hardware_Availability Jun-2016
Software/Service_Availability Feb-2017
Geographic_Distribution 1
Notes The SUT consists of - 1) VIO_Mgmt_Cluster where VIO 3.1 is installed across 3 of 4 FC6030's. 2) Two Nova Compute clusters of 4 FC630's each, the first set of nodes has 512GB RAM and the second set has 384GB RAM. DRS is enabled and set to Partially Automated. 3) A cluster of 4 C6220's, where vCenter Server Appliance runs on one host, Cloudbench Harness VM runs on another host, and a third VM runs on another host is use to access Horizon GUI. All VMs on all hosts use one or more Portgroups on a Distributed Virtual Switch (DVS) for communications. Network consist of Two PC8024 stacked via ports 2, one Switch is PC8024 (Ethernet) the other PC8024F (Fiber).

Cloud Under Test Software Levels
Parameter Value
Python_version 2.7.6
JVM_version 1.8.0_85
Hadoop_version 2.7.1
Cassandra_version 2.1.11
Notes None

Benchmark Harness Node
Parameter Value
Vendor Dell Inc.
Platform PowerEdge FX2 Chassis with 4 FC630
CPU_Description Intel Xeon, E5-2698v4
CPU_Frequency_MHz 2200
RAM_Memory 256 GB, ECC, DDR4, 2133MHz RDIMM
Local_Storage 2x 8GB, IDSDM, RAID1
Quantity 4
Hw_Availability_Date June-2016
OS_Version ESXi 6.5.0-4564106.x86_64_Dell_Customized_A00
OS_Availability_Date DEC-2016
Network_Adapter_1 1 x Qlogic QLE2562 Fiber Channel 8GB 2P
Network_Adapter_2 1 x Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet.quad-port
Network_port_type_1 1000BASE-T
Network_port_type_2 10GBASE-T
Network_speed_port_1 1 Gb/s
Network_speed_port_2 10 Gb/s
Notes Cloud infrastructure and management nodes all reside in a single datacenter located in Austin, TX. Cloud installation followed standard VIO 3.1 installation guide.

Cloud Manager Node
Parameter Value
Hw_Vendor Dell Inc.
Hw_Platform PowerEdge FX2 Chassis with 4 FC630
CPU_Type Intel Xeon, E5-2698v4
CPU_Count 2
CPU_Cores_Per 20
CPU_Frequency_MHz 2200
RAM_Memory_GB 256 GB
Primary_Cache_KB 64 KB I + 32 KB D per core, on chip
Secondary_Cache_KB 256 KB I+D per core, on chip
RAM_Memory_Type ECC, DDR4, 2133MHz RDIMM
Hw_Availability_Date June-2016
Local_Storage 2x 8GB, IDSDM, RAID1
Quantity 4
Virtualized VMware ESXi 6.5
Network_Adapter_1 1 x Qlogic QLE2562 Fiber Channel 8GB 2P
Network_Adapter_2 1 x Broadcom Corporation NetXtreme BCM57810 Gigabit Ethernet.Dual-port
Network_port_type_1 1000BASE-T
Network_port_type_2 10GBASE-SR
Network_speed_port_1 1 Gb/s
Network_speed_port_2 10 Gb/s
OS_Version ESXi 6.5.0-4564106.x86_64_Dell_Customized_A00
OS_Availability_Date Dec-2016
Notes None

Cloud Computes Nodes
Parameter Value
type 1_Compute_nodes 4
type 1_Hw_Vendor Dell Inc.
type 1_Hw_Platform PowerEdge FX2 Chassis with 4 FC630
type 1_Quantity 4
type 1_CPU_Type Intel Xeon, E5-2698v4 (Nodes 1-3), E5-2699v4 (Node 4)
type 1_CPU_Count 8
type 1_CPU_Cores_Per 20 (Nodes 1-3), 22 (Node 4)
type 1_CPU_Frequency_MHz 2200
type 1_RAM_Memory_GB 512
type 1_Primary_Cache_KB 64 KB I + 32 KB D per core, on chip
type 1_Secondary_Cache_KB 256 KB I+D per core, on chip
type 1_RAM_Memory_Type ECC, DDR4, 2400MHz RDIMM
type 1_Hw_Availability_Date June-2016
type 1_Local_Storage 2x 8GB, IDSDM, RAID1
type 1_Virtualized VMware ESXi 6.5
type 1_Network_Adapter_1 1 x Qlogic QLE2562 Fiber Channel 8GB 2Port
type 1_Network_Adapter_2 1 x Broadcom Corporation NetXtreme BCM57810 Gigabit Ethernet.Dual-port
type 1_Network_port_type_1 1000BASE-T
type 1_Network_port_type_2 10GBASE-SR
type 1_Network_speed_port_1 1 Gb/s
type 1_Network_speed_port_2 10 Gb/s
type 1_OS_Version ESXi 6.5.0-4564106.x86_64_Dell_Customized_A00
type 1_OS_Availability_Date Dec-2016
type 1_Notes Each server has a single 3.2TB NVMe (DPN# 61DHT) in PCIe slots attached to each server. Xeon E5-2698v4 used in 3 nodes and E5-2699v4 used in 1 node.
type 2_Compute_nodes 4
type 2_Hw_Vendor Dell Inc.
type 2_Hw_Platform PowerEdge FX2 Chassis with 4 FC630
type 2_Quantity 4
type 2_CPU_Type Intel Xeon, E5-2698v4
type 2_CPU_Count 8
type 2_CPU_Cores_Per 20
type 2_CPU_Frequency_MHz 2200
type 2_RAM_Memory_GB 384
type 2_Primary_Cache_KB 64 KB I + 32 KB D per core, on chip
type 2_Secondary_Cache_KB 256 KB I+D per core, on chip
type 2_RAM_Memory_Type ECC, DDR4, 2400MHz RDIMM
type 2_Hw_Availability_Date June-2016
type 2_Local_Storage 2x 8GB, IDSDM, RAID1
type 2_Virtualized VMware ESXi 6.5
type 2_Network_Adapter_1 1 x Qlogic QLE2562 Fiber Channel 8GB 2Port
type 2_Network_Adapter_2 1 x Broadcom Corporation NetXtreme BCM57810 Gigabit Ethernet.Dual-port
type 2_Network_port_type_1 1000BASE-T
type 2_Network_port_type_2 10GBASE-SR
type 2_Network_speed_port_1 1 Gb/s
type 2_Network_speed_port_2 10 Gb/s
type 2_OS_Version ESXi 6.5.0-4564106.x86_64_Dell_Customized_A00
type 2_OS_Availability_Date Dec-2016
type 2_Notes Each server has a single 3.2TB NVMe (DPN# 61DHT) in PCIe slots attached each server.

Cloud Network Information

Cloud Network Information
Parameter Value
network 1_Technology ethernet
network 1_Protocol TCP/IP
network 1_Protocol_version IPV4
network 1_Speed_Mbps 10000
network 1_Function data
network 1_Notes network_1 related notes
network 2_Technology ethernet
network 2_Protocol TCP/IP
network 2_Protocol_version IPV4
network 2_Speed_Mbps 1000
network 2_Function management
network 2_Notes network_2 related notes

Cloud Schematic

Storage Configuration

Cloud Storage
Parameter Value
Storage 1_Attach_Type local
Storage 1_Capacity 3.2 TB
Storage 1_Technology NVMe
Storage 1_Notes Each server has a single 3.2TB NVMe (DPN# 61DHT) in PCIe slots attached each server.

Performance Details

YCSB Detailed Results for Application Instance Runs
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_11 1 11921.1 1.794 2.903 85.0 0 0
2 12178.8 1.506 2.663 84.0 0 0
3 12664.6 1.568 2.537 72.0 0 0
4 11942.3 1.705 2.817 77.0 0 0
5 12042.5 1.776 2.677 80.0 0 0
6 12436.0 1.654 2.497 70.0 0 0
7 12493.3 1.574 2.545 73.0 0 0
8 12173.4 1.672 2.643 75.0 0 0
9 12503.1 1.537 2.639 72.0 0 0
10 12174.3 1.692 2.469 77.0 0 0
11 12290.1 1.633 2.631 75.0 0 0
12 12585.6 1.486 2.517 71.0 0 0
13 12549.7 1.559 2.437 77.0 0 0
14 12219.7 1.536 2.537 71.0 0 0
15 12328.5 1.655 2.477 73.0 0 0
16 11483.2 1.790 2.799 74.0 0 0
17 11392.4 1.740 2.661 78.0 0 0
18 11512.8 1.521 2.591 80.0 0 0
19 11953.0 1.620 2.577 74.0 0 0
20 11836.4 1.619 2.635 76.0 0 0
21 11560.8 1.659 2.607 77.0 0 0
22 11432.0 1.636 2.739 79.0 0 0
23 11494.9 1.706 2.779 79.0 0 0
24 11331.4 1.728 2.801 81.0 0 0
25 11471.4 1.544 2.531 81.0 0 0
26 11348.0 1.658 2.669 79.0 0 0
27 11505.2 1.651 2.529 75.0 0 0
28 11398.3 1.617 2.705 76.0 0 0
29 11216.7 1.633 2.703 78.0 0 0
30 11563.5 1.594 2.637 79.0 0 0
31 11694.3 1.728 2.687 77.0 0 0
32 11249.6 1.638 2.687 76.0 0 0
33 11180.2 1.698 2.811 79.0 0 0
34 10858.6 1.654 2.807 81.0 0 0
35 11035.9 1.725 2.857 82.0 0 0
36 10523.5 1.698 2.835 78.0 0 0
37 10619.7 1.742 2.783 82.0 0 0
38 10803.7 1.811 2.807 82.0 0 0
39 10792.7 1.877 2.891 84.0 0 0
40 10842.3 1.669 2.753 89.0 0 0
41 11290.8 1.735 2.771 85.0 0 0
42 10367.1 1.787 3.041 81.0 0 0
43 10612.0 1.703 2.799 81.0 0 0
44 10366.8 1.720 2.901 89.0 0 0
45 10199.0 1.731 2.875 87.0 0 0
46 9532.0 1.753 3.161 90.0 0 0
47 10005.0 1.781 3.573 91.0 0 0
48 10566.7 1.759 2.897 80.0 0 0
49 10145.9 1.875 2.923 90.0 0 0
50 10270.6 2.029 3.237 90.0 0 0
51 10243.5 1.735 3.009 90.0 0 0
52 9823.0 1.910 3.187 90.0 0 0
53 10010.8 1.948 3.355 91.0 0 0
54 9813.5 2.021 3.293 93.0 0 0
55 9591.0 2.039 3.349 93.0 0 0
56 9194.9 2.329 3.789 98.0 0 0
57 9266.8 2.113 3.569 95.0 0 0
58 9603.3 2.097 3.563 97.0 0 0
59 8776.8 2.375 3.879 102.0 0 0
60 8827.8 1.992 3.595 101.0 0 0
61 9406.1 2.032 3.433 103.0 0 0
62 8756.3 2.171 3.593 102.0 0 0
63 8375.0 2.573 4.003 103.0 0 0
64 8966.4 2.159 3.493 108.0 0 0
65 8139.5 2.487 4.021 107.0 0 0
66 2913.1 2.287 3.119 213.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_14 1 11997.9 1.583 2.745 86.0 0 0
2 12247.5 1.403 2.611 73.0 0 0
3 12321.6 1.407 2.305 71.0 0 0
4 12715.0 1.433 2.639 76.0 0 0
5 12529.9 1.304 2.305 73.0 0 0
6 12653.4 1.346 2.329 72.0 0 0
7 12389.6 1.422 2.377 77.0 0 0
8 12012.6 1.400 2.295 77.0 0 0
9 11998.6 1.336 2.461 76.0 0 0
10 12658.5 1.175 2.267 73.0 0 0
11 11809.7 1.361 2.537 77.0 0 0
12 11850.0 1.476 2.547 70.0 0 0
13 12101.1 1.467 2.485 76.0 0 0
14 12558.2 1.342 2.313 73.0 0 0
15 11841.0 1.460 2.447 81.0 0 0
16 12124.9 1.358 2.445 77.0 0 0
17 12341.4 1.292 2.409 75.0 0 0
18 12418.2 1.309 2.195 72.0 0 0
19 12191.0 1.279 2.485 72.0 0 0
20 12096.7 1.366 2.379 77.0 0 0
21 12238.4 1.294 2.265 76.0 0 0
22 11264.3 1.606 2.751 72.0 0 0
23 11558.4 1.614 2.539 80.0 0 0
24 11645.2 1.423 2.249 78.0 0 0
25 11726.6 1.430 2.617 77.0 0 0
26 11402.8 1.589 2.803 76.0 0 0
27 11672.2 1.515 2.715 83.0 0 0
28 12019.2 1.294 2.325 83.0 0 0
29 11649.0 1.380 2.599 81.0 0 0
30 11026.7 1.522 2.671 79.0 0 0
31 10577.2 1.668 2.579 80.0 0 0
32 10885.8 1.621 2.833 81.0 0 0
33 10918.8 1.524 2.553 83.0 0 0
34 10096.3 1.627 2.923 83.0 0 0
35 11156.9 1.524 2.807 86.0 0 0
36 10619.2 1.665 3.027 78.0 0 0
37 10593.4 1.611 2.921 85.0 0 0
38 10114.5 1.744 3.201 98.0 0 0
39 9740.6 1.831 3.161 90.0 0 0
40 10577.6 1.628 3.077 92.0 0 0
41 10656.8 1.586 2.943 89.0 0 0
42 10452.9 1.742 2.957 93.0 0 0
43 9710.0 1.809 3.109 91.0 0 0
44 9423.1 1.936 3.271 93.0 0 0
45 9703.3 1.759 3.189 98.0 0 0
46 9158.6 1.970 3.647 94.0 0 0
47 9354.2 2.051 3.379 98.0 0 0
48 9347.8 1.846 3.437 91.0 0 0
49 9091.5 1.834 3.481 91.0 0 0
50 9102.9 1.927 3.511 96.0 0 0
51 9333.4 1.981 3.457 97.0 0 0
52 8978.1 2.069 3.571 92.0 0 0
53 8713.3 2.147 3.817 104.0 0 0
54 8783.1 2.107 3.795 99.0 0 0
55 8312.8 2.433 3.963 101.0 0 0
56 8523.6 2.285 3.755 101.0 0 0
57 8701.3 2.279 3.749 106.0 0 0
58 8713.3 2.079 3.801 98.0 0 0
59 8817.0 2.129 3.737 99.0 0 0
60 8867.1 2.161 3.699 99.0 0 0
61 8531.7 2.173 3.865 103.0 0 0
62 8470.6 2.121 3.623 101.0 0 0
63 8336.5 2.259 3.759 101.0 0 0
64 2497.0 1.793 2.719 191.0 0 0
65 9810.4 2.165 3.145 300.0 0 0
66 2912.9 2.011 2.769 301.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_16 1 11512.0 1.712 2.957 86.0 0 0
2 12376.4 1.559 2.691 77.0 0 0
3 12217.8 1.477 2.435 76.0 0 0
4 12821.7 1.298 2.523 73.0 0 0
5 12500.9 1.476 2.459 73.0 0 0
6 12036.9 1.613 2.485 75.0 0 0
7 12583.0 1.379 2.421 73.0 0 0
8 12475.5 1.463 2.499 78.0 0 0
9 12412.6 1.535 2.497 78.0 0 0
10 12007.1 1.431 2.403 73.0 0 0
11 11941.0 1.578 2.611 73.0 0 0
12 11998.5 1.519 2.413 81.0 0 0
13 12631.8 1.432 2.523 73.0 0 0
14 12411.3 1.464 2.487 77.0 0 0
15 12514.7 1.544 2.655 72.0 0 0
16 11593.8 1.541 2.533 73.0 0 0
17 11919.5 1.449 2.537 76.0 0 0
18 12180.0 1.357 2.617 74.0 0 0
19 11679.8 1.636 2.535 80.0 0 0
20 11727.0 1.544 2.621 74.0 0 0
21 11703.8 1.477 2.537 74.0 0 0
22 11871.1 1.447 2.637 76.0 0 0
23 11954.6 1.365 2.565 82.0 0 0
24 11956.1 1.463 2.437 82.0 0 0
25 11762.6 1.405 2.649 75.0 0 0
26 12145.4 1.376 2.585 77.0 0 0
27 11408.9 1.563 2.581 79.0 0 0
28 15630.9 1.013 1.812 58.0 0 0
29 11464.7 1.462 2.537 78.0 0 0
30 11417.3 1.493 2.705 76.0 0 0
31 11520.9 1.398 2.533 79.0 0 0
32 11381.7 1.483 2.501 78.0 0 0
33 10253.0 1.645 2.865 81.0 0 0
34 11170.4 1.473 2.431 78.0 0 0
35 11611.8 1.440 2.601 81.0 0 0
36 10471.4 1.619 2.869 81.0 0 0
37 10854.6 1.677 3.021 85.0 0 0
38 10630.5 1.629 2.973 91.0 0 0
39 9811.3 1.816 3.475 86.0 0 0
40 10291.1 1.668 3.153 89.0 0 0
41 12854.6 1.122 1.495 89.0 0 0
42 10254.0 1.677 2.887 92.0 0 0
43 10029.2 1.700 3.055 96.0 0 0
44 10180.1 1.703 3.143 100.0 0 0
45 9736.4 1.997 3.175 91.0 0 0
46 9832.3 1.755 3.239 91.0 0 0
47 10034.6 1.772 3.097 89.0 0 0
48 10471.9 1.550 2.977 92.0 0 0
49 9551.7 1.889 3.207 83.0 0 0
50 9391.4 1.927 3.233 91.0 0 0
51 10261.3 1.713 3.071 88.0 0 0
52 9231.7 1.944 3.355 89.0 0 0
53 9254.4 1.764 3.391 95.0 0 0
54 9182.1 1.999 3.339 93.0 0 0
55 9618.4 1.754 3.095 94.0 0 0
56 9158.3 1.899 3.357 92.0 0 0
57 9626.5 1.728 3.193 101.0 0 0
58 9030.4 2.001 3.417 90.0 0 0
59 9331.7 1.870 3.615 100.0 0 0
60 9595.9 1.887 3.353 103.0 0 0
61 9104.5 1.912 3.319 98.0 0 0
62 9296.1 1.779 3.223 99.0 0 0
63 8732.5 2.105 3.475 97.0 0 0
64 9106.5 1.812 3.637 103.0 0 0
65 9175.2 1.902 3.483 98.0 0 0
66 8650.2 2.119 3.771 103.0 0 0
67 0.0 6.610 10.494 203.0 1 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_17 1 11396.5 1.778 2.787 96.0 0 0
2 12264.1 1.476 2.359 80.0 0 0
3 12292.3 1.349 2.119 77.0 0 0
4 11570.7 1.519 2.363 79.0 0 0
5 12513.3 1.278 2.159 83.0 0 0
6 12517.1 1.465 2.241 79.0 0 0
7 12465.9 1.319 1.865 77.0 0 0
8 12832.4 1.348 2.071 81.0 0 0
9 11566.6 1.615 2.297 80.0 0 0
10 12135.2 1.334 2.095 80.0 0 0
11 12752.3 1.250 1.855 80.0 0 0
12 12389.4 1.282 1.933 79.0 0 0
13 12105.8 1.509 2.183 79.0 0 0
14 13053.5 1.092 1.769 82.0 0 0
15 12560.4 1.297 2.073 75.0 0 0
16 11811.3 1.555 2.501 80.0 0 0
17 12032.0 1.446 2.327 79.0 0 0
18 12618.3 1.131 1.760 78.0 0 0
19 12787.2 1.183 1.959 79.0 0 0
20 12340.2 1.348 2.131 83.0 0 0
21 12257.3 1.396 2.241 84.0 0 0
22 12073.6 1.299 2.025 76.0 0 0
23 11686.2 1.655 2.381 79.0 0 0
24 12007.5 1.425 2.227 79.0 0 0
25 11820.8 1.393 2.191 84.0 0 0
26 10571.2 1.621 2.655 86.0 0 0
27 10479.3 1.559 2.379 94.0 0 0
28 10302.5 1.561 2.501 85.0 0 0
29 9828.6 1.692 2.867 84.0 0 0
30 10355.4 1.777 2.729 88.0 0 0
31 9880.9 1.882 3.121 92.0 0 0
32 9427.5 2.073 3.183 90.0 0 0
33 10378.4 1.621 2.817 94.0 0 0
34 8830.8 2.042 3.099 96.0 0 0
35 9611.1 1.855 3.043 99.0 0 0
36 9151.0 1.977 3.115 92.0 0 0
37 9077.9 1.949 2.963 112.0 0 0
38 9227.6 2.055 3.421 107.0 0 0
39 9215.1 1.700 2.917 108.0 0 0
40 9227.1 2.073 3.217 101.0 0 0
41 8969.7 2.014 3.249 106.0 0 0
42 9229.9 2.107 3.261 99.0 0 0
43 8790.9 2.121 3.353 107.0 0 0
44 8656.7 1.996 3.143 103.0 0 0
45 8983.9 2.037 3.219 103.0 0 0
46 9276.6 1.865 3.023 103.0 0 0
47 9046.2 1.926 3.021 103.0 0 0
48 9150.8 1.864 3.115 108.0 0 0
49 8790.5 1.901 3.047 108.0 0 0
50 8709.5 2.181 3.415 110.0 0 0
51 8669.1 2.221 3.389 113.0 0 0
52 8927.9 1.915 2.941 109.0 0 0
53 8841.2 1.953 3.425 110.0 0 0
54 8760.5 2.233 3.643 106.0 0 0
55 8705.2 2.073 3.563 109.0 0 0
56 8684.1 2.343 3.527 108.0 0 0
57 8767.1 2.023 3.461 105.0 0 0
58 9517.6 1.833 2.897 100.0 0 0
59 8949.7 1.947 3.185 100.0 0 0
60 9161.7 2.071 3.227 105.0 0 0
61 9266.3 2.016 3.023 104.0 0 0
62 9570.1 1.804 2.715 102.0 0 0
63 8796.7 1.845 2.955 102.0 0 0
64 2913.0 2.117 2.857 212.0 0 0
65 2913.2 2.069 2.685 301.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_19 1 12031.2 1.519 2.721 91.0 0 0
2 11388.4 1.593 2.611 74.0 0 0
3 11862.8 1.369 2.359 82.0 0 0
4 12216.7 1.275 2.209 78.0 0 0
5 11614.3 1.519 2.515 75.0 0 0
6 12116.5 1.445 2.471 77.0 0 0
7 12291.4 1.438 2.511 80.0 0 0
8 11932.6 1.336 2.211 75.0 0 0
9 12332.3 1.455 2.509 82.0 0 0
10 11623.9 1.488 2.543 77.0 0 0
11 11731.6 1.322 2.401 84.0 0 0
12 11012.9 1.531 2.497 78.0 0 0
13 11422.3 1.478 2.605 82.0 0 0
14 11096.2 1.608 2.655 80.0 0 0
15 11404.5 1.541 2.581 82.0 0 0
16 10883.1 1.530 2.821 85.0 0 0
17 10816.7 1.523 2.669 90.0 0 0
18 10986.0 1.415 2.569 94.0 0 0
19 10856.7 1.602 2.769 86.0 0 0
20 11227.3 1.519 2.487 84.0 0 0
21 11129.9 1.576 2.697 87.0 0 0
22 11049.8 1.439 2.521 87.0 0 0
23 10949.5 1.471 2.583 83.0 0 0
24 10942.0 1.479 2.469 86.0 0 0
25 10541.2 1.580 2.721 82.0 0 0
26 10568.5 1.643 2.853 86.0 0 0
27 10881.2 1.607 2.695 84.0 0 0
28 9647.6 1.718 2.835 85.0 0 0
29 10303.0 1.606 2.773 89.0 0 0
30 10237.8 1.733 3.189 94.0 0 0
31 9822.6 1.808 3.077 89.0 0 0
32 10339.2 1.637 2.783 87.0 0 0
33 10899.2 1.524 2.603 87.0 0 0
34 11060.0 1.601 2.783 91.0 0 0
35 10106.2 1.813 3.053 84.0 0 0
36 10600.1 1.577 3.031 85.0 0 0
37 11131.5 1.451 2.607 88.0 0 0
38 10845.0 1.528 2.799 88.0 0 0
39 10847.6 1.715 2.895 91.0 0 0
40 10419.4 1.606 2.709 89.0 0 0
41 10250.0 1.706 3.167 89.0 0 0
42 10107.2 1.731 3.001 90.0 0 0
43 10379.5 1.591 2.687 87.0 0 0
44 9752.2 1.744 3.081 88.0 0 0
45 9961.0 1.696 2.943 91.0 0 0
46 9606.1 1.626 2.929 92.0 0 0
47 9943.2 1.661 3.091 93.0 0 0
48 9731.9 1.741 3.235 95.0 0 0
49 9746.9 1.581 2.885 94.0 0 0
50 9858.5 1.728 2.951 95.0 0 0
51 9847.9 1.617 2.997 99.0 0 0
52 9478.0 1.869 3.125 96.0 0 0
53 9966.4 1.657 3.067 93.0 0 0
54 9512.8 1.861 2.925 91.0 0 0
55 9659.2 1.828 2.973 96.0 0 0
56 9303.1 1.786 3.095 96.0 0 0
57 9099.8 1.900 3.201 97.0 0 0
58 8703.9 1.965 3.233 100.0 0 0
59 8747.2 2.181 3.549 102.0 0 0
60 8936.9 1.916 3.395 109.0 0 0
61 9052.4 1.905 3.599 100.0 0 0
62 8780.9 1.991 3.439 97.0 0 0
63 8646.2 1.925 3.517 101.0 0 0
64 8457.7 1.945 3.431 106.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_21 1 11352.0 1.783 2.905 91.0 0 0
2 12824.8 1.365 2.169 73.0 0 0
3 13128.9 1.328 1.997 76.0 0 0
4 11742.5 1.745 2.397 79.0 0 0
5 12924.9 1.317 1.995 76.0 0 0
6 12364.1 1.231 2.267 72.0 0 0
7 12955.0 1.406 2.143 75.0 0 0
8 13138.2 1.408 1.995 73.0 0 0
9 13002.9 1.329 2.025 73.0 0 0
10 13254.2 1.235 1.893 71.0 0 0
11 12556.7 1.377 2.159 79.0 0 0
12 12425.8 1.401 2.129 77.0 0 0
13 11739.3 1.560 2.263 74.0 0 0
14 13078.2 1.231 2.030 77.0 0 0
15 12378.2 1.329 1.881 82.0 0 0
16 12465.3 1.356 2.117 75.0 0 0
17 12582.6 1.452 2.197 77.0 0 0
18 12564.2 1.405 2.171 76.0 0 0
19 12823.0 1.168 1.940 79.0 0 0
20 11825.4 1.458 2.231 78.0 0 0
21 12386.2 1.431 2.221 78.0 0 0
22 12634.1 1.319 2.125 80.0 0 0
23 11977.5 1.416 2.217 77.0 0 0
24 12712.6 1.228 2.009 78.0 0 0
25 11975.9 1.450 2.097 85.0 0 0
26 12141.8 1.433 2.235 77.0 0 0
27 12369.0 1.424 2.181 75.0 0 0
28 12288.8 1.475 2.395 82.0 0 0
29 11975.5 1.543 2.279 79.0 0 0
30 12566.9 1.292 2.085 85.0 0 0
31 11679.2 1.504 2.339 79.0 0 0
32 12467.9 1.381 2.201 77.0 0 0
33 12300.3 1.357 2.017 87.0 0 0
34 11591.8 1.546 2.671 80.0 0 0
35 11943.4 1.357 2.171 86.0 0 0
36 12493.9 1.413 2.033 93.0 0 0
37 11291.0 1.569 2.353 78.0 0 0
38 11887.4 1.336 2.193 94.0 0 0
39 11685.7 1.485 2.165 85.0 0 0
40 11451.3 1.551 2.505 85.0 0 0
41 11566.4 1.492 2.291 88.0 0 0
42 10849.6 1.612 2.507 89.0 0 0
43 10138.8 1.656 2.861 85.0 0 0
44 10204.7 1.990 2.841 93.0 0 0
45 9804.3 1.877 2.801 96.0 0 0
46 10494.3 1.659 2.643 93.0 0 0
47 10311.0 1.552 2.681 96.0 0 0
48 9395.3 1.972 3.169 98.0 0 0
49 9250.5 1.876 2.997 93.0 0 0
50 9302.6 1.937 2.919 98.0 0 0
51 9277.9 2.020 2.983 102.0 0 0
52 9665.1 1.679 2.755 97.0 0 0
53 9856.6 1.739 2.721 92.0 0 0
54 8868.7 1.936 3.111 103.0 0 0
55 8725.0 2.101 3.361 95.0 0 0
56 8519.0 2.359 3.555 102.0 0 0
57 8327.4 2.287 3.519 106.0 0 0
58 8580.9 2.249 3.361 101.0 0 0
59 8379.5 2.275 3.461 107.0 0 0
60 8631.8 1.890 3.143 101.0 0 0
61 8960.7 1.952 3.083 107.0 0 0
62 1664.5 1.893 2.195 190.0 0 0
63 9269.3 2.079 3.029 301.0 0 0
64 2496.8 2.577 2.891 301.0 0 0
65 2080.7 2.649 3.151 301.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_24 1 10957.2 1.836 2.945 79.0 0 0
2 12141.5 1.476 2.649 70.0 0 0
3 12519.1 1.396 2.411 77.0 0 0
4 12099.5 1.288 2.329 80.0 0 0
5 12097.2 1.368 2.441 72.0 0 0
6 12220.3 1.337 2.419 78.0 0 0
7 12887.8 1.254 2.299 75.0 0 0
8 12684.6 1.257 2.377 74.0 0 0
9 12390.3 1.282 2.409 76.0 0 0
10 12219.3 1.259 2.337 80.0 0 0
11 11267.2 1.515 2.621 86.0 0 0
12 11433.3 1.334 2.471 77.0 0 0
13 11824.7 1.311 2.427 85.0 0 0
14 10718.7 1.418 2.519 82.0 0 0
15 11185.6 1.389 2.363 77.0 0 0
16 10824.5 1.844 2.985 82.0 0 0
17 10783.8 1.518 2.573 82.0 0 0
18 11450.6 1.359 2.339 83.0 0 0
19 11326.7 1.356 2.355 78.0 0 0
20 10970.8 1.517 2.527 82.0 0 0
21 11446.0 1.411 2.557 84.0 0 0
22 10485.6 1.451 2.509 81.0 0 0
23 11163.3 1.277 2.381 82.0 0 0
24 11226.0 1.391 2.493 81.0 0 0
25 11673.4 1.268 2.253 84.0 0 0
26 11618.7 1.298 2.331 87.0 0 0
27 10372.0 1.501 2.575 90.0 0 0
28 10793.5 1.458 2.593 87.0 0 0
29 10697.6 1.503 2.647 84.0 0 0
30 11301.6 1.485 2.503 84.0 0 0
31 10699.5 1.476 2.709 86.0 0 0
32 11415.4 1.292 2.327 88.0 0 0
33 10663.9 1.344 2.335 84.0 0 0
34 10706.8 1.437 2.505 91.0 0 0
35 10464.5 1.453 2.589 84.0 0 0
36 9915.8 1.707 2.763 91.0 0 0
37 10889.7 1.407 2.591 84.0 0 0
38 10991.3 1.407 2.489 91.0 0 0
39 10072.7 1.681 2.999 81.0 0 0
40 10311.5 1.463 2.649 87.0 0 0
41 10275.7 1.862 2.953 85.0 0 0
42 9872.9 1.798 3.269 84.0 0 0
43 10459.3 1.586 2.783 87.0 0 0
44 9548.4 1.849 3.099 86.0 0 0
45 9484.2 1.974 3.383 93.0 0 0
46 10402.8 1.491 2.619 95.0 0 0
47 9985.4 1.704 2.803 93.0 0 0
48 9817.7 1.696 3.283 94.0 0 0
49 9747.8 1.648 2.893 99.0 0 0
50 9726.7 1.618 3.105 105.0 0 0
51 9640.9 1.666 2.903 95.0 0 0
52 9361.0 1.969 3.361 99.0 0 0
53 9425.3 1.810 3.183 103.0 0 0
54 9491.5 1.817 3.155 98.0 0 0
55 9476.1 1.834 3.345 99.0 0 0
56 8794.5 1.856 3.383 103.0 0 0
57 9509.9 1.841 3.269 103.0 0 0
58 9029.9 1.854 3.211 101.0 0 0
59 9523.7 1.752 3.077 94.0 0 0
60 9142.0 2.033 3.479 103.0 0 0
61 10097.0 1.942 2.561 195.0 0 0
62 2913.3 1.706 2.237 300.0 0 0
63 2497.0 1.525 2.195 117.0 0 0
64 2497.1 1.872 2.681 109.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_26 1 11550.4 1.753 2.925 87.0 0 0
2 11793.3 1.662 2.545 78.0 0 0
3 11672.2 1.590 2.363 84.0 0 0
4 12449.6 1.521 2.337 77.0 0 0
5 12111.4 1.662 2.405 77.0 0 0
6 12276.7 1.631 2.375 76.0 0 0
7 11815.2 1.692 2.475 76.0 0 0
8 11869.3 1.736 2.445 79.0 0 0
9 11725.1 1.401 2.277 77.0 0 0
10 11150.9 1.798 2.617 85.0 0 0
11 11394.6 1.808 2.583 84.0 0 0
12 11965.0 1.459 2.201 82.0 0 0
13 11436.5 1.601 2.309 83.0 0 0
14 11758.8 1.504 2.377 87.0 0 0
15 11818.9 1.250 1.930 85.0 0 0
16 11758.6 1.669 2.425 88.0 0 0
17 11229.8 1.742 2.581 85.0 0 0
18 11295.4 1.770 2.645 89.0 0 0
19 11572.6 1.530 2.347 81.0 0 0
20 11172.1 1.732 2.479 85.0 0 0
21 11328.0 1.623 2.417 84.0 0 0
22 11317.6 1.537 2.525 82.0 0 0
23 10587.5 1.621 2.601 84.0 0 0
24 10092.9 2.087 3.069 90.0 0 0
25 11301.5 1.712 2.607 89.0 0 0
26 10358.4 1.645 2.675 95.0 0 0
27 9928.3 1.992 3.127 93.0 0 0
28 10368.6 1.704 2.685 89.0 0 0
29 9791.8 2.129 3.431 93.0 0 0
30 10052.5 1.969 2.929 94.0 0 0
31 10398.7 1.845 2.761 88.0 0 0
32 10168.4 1.821 2.887 87.0 0 0
33 10782.3 1.761 2.737 94.0 0 0
34 10915.6 1.722 2.565 88.0 0 0
35 10034.8 1.900 2.841 94.0 0 0
36 9943.7 2.007 3.071 92.0 0 0
37 10499.7 1.539 2.473 92.0 0 0
38 10865.3 1.844 2.747 91.0 0 0
39 9914.3 1.999 3.047 92.0 0 0
40 10422.4 1.941 3.033 96.0 0 0
41 10545.5 1.678 2.713 93.0 0 0
42 10048.9 1.868 2.977 95.0 0 0
43 9990.0 1.739 2.917 95.0 0 0
44 9690.2 2.265 3.447 93.0 0 0
45 10124.1 1.825 2.875 92.0 0 0
46 9442.5 2.267 3.277 96.0 0 0
47 10316.9 1.729 2.751 100.0 0 0
48 9680.9 2.101 3.179 102.0 0 0
49 9782.1 1.942 2.925 96.0 0 0
50 9605.4 2.005 3.069 96.0 0 0
51 8754.0 2.417 3.699 98.0 0 0
52 8741.1 2.113 3.463 103.0 0 0
53 8357.4 2.303 3.551 111.0 0 0
54 9059.6 1.989 3.359 119.0 0 0
55 8353.1 2.897 4.335 114.0 0 0
56 8405.3 2.287 3.661 114.0 0 0
57 9130.9 2.038 3.283 110.0 0 0
58 8842.7 2.315 3.445 113.0 0 0
59 8871.1 1.937 3.223 110.0 0 0
60 8337.2 2.541 3.891 101.0 0 0
61 8317.5 2.495 3.773 107.0 0 0
62 7532.8 2.755 4.163 121.0 0 0
63 7983.7 2.233 3.655 121.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_28 1 10884.7 1.988 3.073 95.0 0 0
2 11241.8 1.718 2.811 81.0 0 0
3 11646.6 1.488 2.555 78.0 0 0
4 11831.7 1.434 2.335 78.0 0 0
5 11729.0 1.613 2.649 81.0 0 0
6 11124.2 1.593 2.733 87.0 0 0
7 11375.0 1.413 2.515 88.0 0 0
8 11270.1 1.590 2.631 86.0 0 0
9 11333.4 1.374 2.473 86.0 0 0
10 11497.7 1.453 2.421 83.0 0 0
11 11344.4 1.638 2.463 88.0 0 0
12 11385.4 1.671 2.731 80.0 0 0
13 11088.4 1.564 2.747 84.0 0 0
14 11419.4 1.549 2.533 83.0 0 0
15 11083.6 1.711 2.769 84.0 0 0
16 11043.3 1.694 2.587 82.0 0 0
17 10747.7 1.555 2.623 83.0 0 0
18 11066.8 1.667 2.589 88.0 0 0
19 10358.5 1.775 3.065 83.0 0 0
20 9866.4 1.709 3.139 90.0 0 0
21 10489.1 1.696 2.643 88.0 0 0
22 10165.1 1.770 3.139 89.0 0 0
23 10471.2 1.852 2.909 90.0 0 0
24 11111.4 1.439 2.425 88.0 0 0
25 10661.7 1.632 2.653 100.0 0 0
26 10186.9 1.824 3.437 89.0 0 0
27 10507.1 1.664 2.971 90.0 0 0
28 10161.6 1.694 2.825 88.0 0 0
29 10666.0 1.644 2.827 92.0 0 0
30 10324.1 1.693 3.071 87.0 0 0
31 10351.2 1.838 2.855 89.0 0 0
32 10163.9 1.762 2.975 91.0 0 0
33 10285.9 1.743 2.721 90.0 0 0
34 9745.4 2.055 3.151 97.0 0 0
35 9753.0 1.844 3.237 89.0 0 0
36 9471.1 1.965 3.255 98.0 0 0
37 9458.1 1.766 3.103 95.0 0 0
38 9684.8 1.872 3.147 98.0 0 0
39 9422.3 2.033 3.271 96.0 0 0
40 9238.0 2.010 3.293 94.0 0 0
41 9758.7 1.784 3.229 98.0 0 0
42 8965.8 2.189 3.563 103.0 0 0
43 9238.3 2.008 3.291 97.0 0 0
44 8974.6 2.095 3.335 96.0 0 0
45 8602.7 2.063 3.483 99.0 0 0
46 8846.2 1.993 3.451 109.0 0 0
47 8801.0 2.173 3.501 104.0 0 0
48 8838.3 2.303 3.553 106.0 0 0
49 8805.3 2.317 3.647 104.0 0 0
50 8921.0 2.071 3.367 99.0 0 0
51 8834.4 2.341 3.513 105.0 0 0
52 9097.6 2.085 3.409 105.0 0 0
53 9191.7 2.251 3.653 103.0 0 0
54 9196.0 1.988 3.491 104.0 0 0
55 9279.3 2.263 3.525 105.0 0 0
56 9648.1 1.891 3.147 94.0 0 0
57 9687.0 1.896 3.279 101.0 0 0
58 9092.6 2.107 3.781 107.0 0 0
59 9024.2 2.163 3.453 105.0 0 0
60 7986.8 2.645 4.099 101.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_30 1 11090.2 1.767 2.975 95.0 0 0
2 11293.7 1.559 2.673 72.0 0 0
3 11619.0 1.489 2.437 80.0 0 0
4 11464.3 1.504 2.633 80.0 0 0
5 11053.1 1.632 2.621 76.0 0 0
6 10672.5 1.563 2.713 76.0 0 0
7 10641.0 1.720 2.703 85.0 0 0
8 11255.1 1.515 2.557 84.0 0 0
9 11185.9 1.546 2.619 84.0 0 0
10 11029.7 1.577 2.663 83.0 0 0
11 11012.0 1.634 2.671 79.0 0 0
12 11039.5 1.600 2.623 80.0 0 0
13 11452.3 1.578 2.617 85.0 0 0
14 11065.4 1.539 2.641 79.0 0 0
15 11203.0 1.459 2.719 85.0 0 0
16 10538.2 1.677 2.853 86.0 0 0
17 11432.8 1.397 2.697 87.0 0 0
18 10648.8 1.855 3.429 76.0 0 0
19 9389.3 2.042 3.463 85.0 0 0
20 10703.8 1.576 2.845 83.0 0 0
21 10153.0 1.801 3.069 84.0 0 0
22 10532.6 1.638 3.143 87.0 0 0
23 9622.7 1.647 3.123 89.0 0 0
24 10067.5 1.553 2.917 84.0 0 0
25 9129.8 2.355 3.587 83.0 0 0
26 10507.1 1.552 2.711 83.0 0 0
27 10518.1 1.650 3.047 86.0 0 0
28 10093.1 1.757 2.979 87.0 0 0
29 10395.6 1.586 3.081 83.0 0 0
30 10009.8 1.763 3.125 86.0 0 0
31 9977.1 1.702 3.265 91.0 0 0
32 9890.1 1.801 3.317 94.0 0 0
33 10562.7 1.624 3.009 86.0 0 0
34 9602.4 1.949 3.561 87.0 0 0
35 9783.8 1.784 3.355 88.0 0 0
36 10086.6 1.797 3.211 88.0 0 0
37 9729.1 1.876 3.051 97.0 0 0
38 9875.7 1.689 3.167 89.0 0 0
39 9963.3 1.746 3.103 95.0 0 0
40 9551.6 1.862 3.485 99.0 0 0
41 9546.4 1.850 3.173 90.0 0 0
42 8940.9 1.932 3.623 96.0 0 0
43 9113.9 2.000 3.413 96.0 0 0
44 8889.4 2.095 3.683 102.0 0 0
45 8642.1 2.057 3.505 104.0 0 0
46 8810.3 1.982 3.629 107.0 0 0
47 9185.4 1.977 3.473 100.0 0 0
48 8858.0 1.907 3.547 97.0 0 0
49 8938.7 2.117 3.539 95.0 0 0
50 9062.6 1.972 3.455 96.0 0 0
51 9475.4 1.664 3.441 100.0 0 0
52 9279.6 1.950 3.501 99.0 0 0
53 9281.3 1.784 3.427 105.0 0 0
54 8692.1 2.027 3.749 93.0 0 0
55 8644.0 2.002 3.461 98.0 0 0
56 8998.4 1.956 3.259 102.0 0 0
57 8741.6 2.223 3.727 103.0 0 0
58 9572.9 1.856 3.459 106.0 0 0
59 9089.9 1.983 3.381 107.0 0 0
60 9154.8 1.980 3.393 98.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_32 1 10428.6 1.857 3.137 103.0 0 0
2 10362.4 1.640 2.913 87.0 0 0
3 10101.8 1.662 2.749 95.0 0 0
4 10301.5 1.579 2.755 90.0 0 0
5 10394.8 1.604 2.731 83.0 0 0
6 10301.8 1.530 2.631 87.0 0 0
7 10420.1 1.660 2.685 88.0 0 0
8 10695.1 1.479 2.719 88.0 0 0
9 10163.8 1.611 2.657 91.0 0 0
10 10295.9 1.633 2.875 90.0 0 0
11 10074.2 1.677 3.109 90.0 0 0
12 10194.6 1.572 2.859 91.0 0 0
13 10593.7 1.496 2.729 84.0 0 0
14 9964.7 1.602 2.873 90.0 0 0
15 9687.9 1.603 2.777 91.0 0 0
16 9851.1 1.608 2.815 92.0 0 0
17 9038.9 1.859 3.485 91.0 0 0
18 9971.6 1.514 2.641 93.0 0 0
19 9833.2 1.747 3.083 90.0 0 0
20 10092.1 1.455 2.701 90.0 0 0
21 9871.4 1.570 2.859 89.0 0 0
22 10252.4 1.551 2.839 85.0 0 0
23 10134.4 1.489 2.663 88.0 0 0
24 10310.4 1.540 2.835 89.0 0 0
25 10356.8 1.515 2.833 84.0 0 0
26 9938.5 1.619 2.753 86.0 0 0
27 9820.9 1.687 3.049 90.0 0 0
28 10430.6 1.497 2.639 89.0 0 0
29 10085.0 1.695 3.169 89.0 0 0
30 9761.8 1.980 3.339 95.0 0 0
31 10082.9 1.677 3.169 94.0 0 0
32 9864.9 1.722 3.087 90.0 0 0
33 9981.2 1.610 2.783 91.0 0 0
34 9307.7 1.799 2.951 91.0 0 0
35 9366.6 1.890 3.381 95.0 0 0
36 9234.3 2.003 3.331 97.0 0 0
37 9157.3 1.909 3.457 94.0 0 0
38 9819.2 1.656 3.073 102.0 0 0
39 8959.4 1.988 3.331 94.0 0 0
40 8941.7 1.844 3.157 100.0 0 0
41 8928.4 2.000 3.407 104.0 0 0
42 9237.9 1.991 3.435 102.0 0 0
43 8674.2 2.083 3.357 100.0 0 0
44 9331.1 1.747 3.173 100.0 0 0
45 8639.7 2.087 3.435 101.0 0 0
46 8788.3 2.111 3.577 100.0 0 0
47 9044.9 1.907 3.453 99.0 0 0
48 8941.1 2.105 3.569 101.0 0 0
49 8576.1 2.157 3.581 99.0 0 0
50 9256.3 1.943 3.371 95.0 0 0
51 8756.8 2.141 3.509 101.0 0 0
52 8591.1 2.145 3.697 99.0 0 0
53 8771.3 2.131 3.695 93.0 0 0
54 8580.2 2.207 3.701 99.0 0 0
55 8255.1 2.275 3.831 110.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_34 1 12497.3 1.481 3.001 89.0 0 0
2 12269.6 1.333 2.801 76.0 0 0
3 11793.6 1.512 2.749 79.0 0 0
4 12288.0 1.439 2.773 78.0 0 0
5 12617.5 1.266 2.369 77.0 0 0
6 12717.5 1.362 2.667 79.0 0 0
7 11545.6 1.738 2.983 76.0 0 0
8 12667.8 1.542 2.867 72.0 0 0
9 12881.0 1.276 2.667 79.0 0 0
10 11997.0 1.472 2.823 77.0 0 0
11 12662.6 1.386 2.759 76.0 0 0
12 12254.6 1.475 2.773 76.0 0 0
13 13122.8 1.239 2.519 72.0 0 0
14 12239.0 1.530 2.849 78.0 0 0
15 12759.3 1.292 2.541 76.0 0 0
16 12076.3 1.452 2.783 79.0 0 0
17 12103.5 1.531 2.885 79.0 0 0
18 12592.1 1.401 2.899 81.0 0 0
19 11410.6 1.859 3.661 77.0 0 0
20 11992.6 1.536 2.833 78.0 0 0
21 12106.0 1.409 2.847 83.0 0 0
22 12342.2 1.647 3.047 79.0 0 0
23 11696.6 1.620 3.053 73.0 0 0
24 11745.8 1.644 2.849 76.0 0 0
25 12554.6 1.357 2.695 73.0 0 0
26 12378.1 1.438 2.841 77.0 0 0
27 12278.2 1.488 2.805 78.0 0 0
28 12235.7 1.480 2.777 76.0 0 0
29 12319.4 1.425 2.659 74.0 0 0
30 11896.7 1.421 2.779 79.0 0 0
31 12496.7 1.441 2.821 77.0 0 0
32 12102.7 1.551 2.857 79.0 0 0
33 11301.7 1.771 3.025 77.0 0 0
34 12079.6 1.476 2.863 77.0 0 0
35 12229.6 1.479 2.827 79.0 0 0
36 12211.5 1.350 2.683 81.0 0 0
37 12425.1 1.422 2.833 78.0 0 0
38 11581.7 1.673 2.867 77.0 0 0
39 11799.4 1.627 2.965 80.0 0 0
40 11413.0 1.667 3.119 79.0 0 0
41 11714.8 1.569 3.031 77.0 0 0
42 11447.7 1.627 2.953 79.0 0 0
43 11607.8 1.561 2.857 82.0 0 0
44 11945.7 1.462 2.811 79.0 0 0
45 11726.9 1.590 2.795 80.0 0 0
46 11721.8 1.517 2.937 81.0 0 0
47 11662.6 1.686 3.087 83.0 0 0
48 11810.3 1.400 2.743 80.0 0 0
49 11779.4 1.590 3.049 85.0 0 0
50 10375.9 1.705 3.221 91.0 0 0
51 10338.7 1.797 3.387 93.0 0 0
52 9252.5 2.171 3.909 84.0 0 0
53 10172.3 1.767 3.197 88.0 0 0
54 9218.5 2.193 3.859 93.0 0 0
55 9719.9 1.795 3.491 92.0 0 0
56 10994.9 1.422 2.701 102.0 0 0
57 10656.4 1.596 3.149 100.0 0 0
58 9779.6 1.845 3.667 93.0 0 0
59 10870.5 1.708 3.077 94.0 0 0
60 9570.5 2.041 3.641 88.0 0 0
61 9795.2 1.965 3.551 90.0 0 0
62 10405.5 1.630 3.103 89.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_35 1 11315.5 1.763 3.033 91.0 0 0
2 11067.1 1.722 2.629 83.0 0 0
3 11061.0 1.545 2.545 81.0 0 0
4 11339.9 1.662 2.541 81.0 0 0
5 10732.8 1.712 2.711 91.0 0 0
6 10494.9 1.902 2.879 86.0 0 0
7 10119.1 1.740 2.939 89.0 0 0
8 9471.6 2.046 3.097 96.0 0 0
9 9805.5 1.840 2.913 98.0 0 0
10 9320.0 2.199 3.167 97.0 0 0
11 9425.6 1.919 3.339 96.0 0 0
12 9604.7 1.718 2.849 99.0 0 0
13 9320.3 2.010 3.387 96.0 0 0
14 9733.9 1.777 2.945 99.0 0 0
15 9582.2 2.087 3.311 98.0 0 0
16 8395.0 2.617 4.319 102.0 0 0
17 9225.7 2.016 3.141 102.0 0 0
18 9138.0 2.151 3.461 93.0 0 0
19 9705.8 2.073 3.439 106.0 0 0
20 9439.8 2.207 3.419 103.0 0 0
21 8830.2 2.161 3.627 105.0 0 0
22 8996.7 2.351 3.691 104.0 0 0
23 9168.3 2.261 3.741 91.0 0 0
24 9290.8 2.017 3.463 95.0 0 0
25 9242.9 2.163 3.425 89.0 0 0
26 8866.4 2.249 3.667 97.0 0 0
27 9290.2 1.872 3.341 100.0 0 0
28 8975.5 2.089 3.387 105.0 0 0
29 8868.4 2.337 3.727 102.0 0 0
30 9317.5 2.129 3.537 101.0 0 0
31 9248.0 1.989 3.199 104.0 0 0
32 9154.9 2.139 3.317 105.0 0 0
33 9031.1 2.159 3.515 107.0 0 0
34 8937.1 2.017 3.347 103.0 0 0
35 9317.8 2.022 3.259 106.0 0 0
36 9509.9 2.032 3.207 108.0 0 0
37 9266.3 1.936 3.151 99.0 0 0
38 9679.0 1.890 3.115 103.0 0 0
39 9464.0 1.993 3.119 101.0 0 0
40 9136.5 2.013 3.277 101.0 0 0
41 9168.9 2.145 3.435 100.0 0 0
42 9174.3 1.951 3.239 105.0 0 0
43 9420.6 2.006 3.331 99.0 0 0
44 8993.9 2.059 3.211 100.0 0 0
45 8926.6 2.273 3.411 93.0 0 0
46 9118.0 2.263 3.287 97.0 0 0
47 8757.5 2.473 3.973 106.0 0 0
48 9169.8 2.309 3.509 99.0 0 0
49 8613.3 2.391 3.695 100.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_38 1 10091.3 1.933 3.057 91.0 0 0
2 10787.5 1.318 2.221 90.0 0 0
3 10777.3 1.489 2.547 88.0 0 0
4 10332.9 1.536 2.687 91.0 0 0
5 10812.9 1.433 2.165 93.0 0 0
6 10006.7 1.786 2.557 93.0 0 0
7 10371.1 1.631 2.449 95.0 0 0
8 10484.5 1.456 2.401 98.0 0 0
9 10066.7 1.697 2.651 104.0 0 0
10 9784.5 1.749 2.699 96.0 0 0
11 9746.9 1.585 2.339 107.0 0 0
12 9436.0 1.912 2.931 103.0 0 0
13 9189.5 2.071 3.375 98.0 0 0
14 9228.3 2.057 3.205 96.0 0 0
15 9636.9 1.868 3.245 100.0 0 0
16 10737.9 1.479 2.739 97.0 0 0
17 9298.7 2.167 3.111 104.0 0 0
18 8864.6 2.051 3.375 100.0 0 0
19 8979.0 2.101 3.273 99.0 0 0
20 8655.0 1.967 3.237 98.0 0 0
21 9183.8 1.808 2.809 104.0 0 0
22 9288.9 1.787 2.871 103.0 0 0
23 9128.8 1.988 3.137 105.0 0 0
24 9029.3 1.948 3.243 117.0 0 0
25 9005.7 2.097 3.361 112.0 0 0
26 9517.1 1.693 2.743 110.0 0 0
27 9179.1 1.917 2.981 103.0 0 0
28 9339.7 1.933 3.141 110.0 0 0
29 9412.2 1.895 3.131 107.0 0 0
30 8805.9 2.267 3.391 115.0 0 0
31 8608.5 2.237 3.487 107.0 0 0
32 9142.8 1.919 3.137 112.0 0 0
33 9371.1 1.844 2.937 105.0 0 0
34 9154.9 2.035 3.153 108.0 0 0
35 9481.4 1.738 2.843 98.0 0 0
36 9159.1 1.794 3.091 103.0 0 0
37 9323.6 1.757 2.909 106.0 0 0
38 9151.4 1.813 2.905 102.0 0 0
39 9713.7 1.835 3.039 110.0 0 0
40 9391.4 1.808 2.817 111.0 0 0
41 9300.6 1.980 3.133 103.0 0 0
42 8822.8 2.335 3.487 106.0 0 0
43 9538.9 1.990 3.081 106.0 0 0
44 9487.2 2.030 3.289 107.0 0 0
45 9231.1 2.221 3.249 106.0 0 0
46 8956.6 2.299 3.551 113.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_40 1 10228.3 1.928 3.249 95.0 0 0
2 10688.8 1.586 2.845 89.0 0 0
3 9309.9 2.087 3.309 84.0 0 0
4 9347.3 1.890 3.489 89.0 0 0
5 9722.6 1.976 3.157 89.0 0 0
6 10171.3 1.905 3.383 85.0 0 0
7 10161.1 1.864 2.873 88.0 0 0
8 10381.3 1.711 3.105 94.0 0 0
9 9639.6 2.245 3.365 88.0 0 0
10 9714.2 2.022 3.261 88.0 0 0
11 9792.9 1.903 3.239 94.0 0 0
12 9805.6 1.860 3.253 88.0 0 0
13 10202.3 1.718 3.067 94.0 0 0
14 10351.3 1.899 3.201 92.0 0 0
15 9944.8 1.876 3.285 91.0 0 0
16 10351.8 1.822 3.113 92.0 0 0
17 10082.7 1.761 3.175 89.0 0 0
18 9780.0 2.005 3.321 90.0 0 0
19 9827.0 1.907 3.351 92.0 0 0
20 9432.5 2.083 3.345 99.0 0 0
21 9672.8 1.948 3.439 106.0 0 0
22 9506.2 2.097 3.525 107.0 0 0
23 9535.7 2.012 3.415 102.0 0 0
24 9160.4 2.029 3.431 105.0 0 0
25 8837.1 2.379 3.743 103.0 0 0
26 9263.0 2.225 3.505 109.0 0 0
27 8817.3 2.575 3.893 99.0 0 0
28 8862.4 2.239 3.787 99.0 0 0
29 8576.9 2.435 3.619 103.0 0 0
30 8748.6 2.237 3.561 97.0 0 0
31 9282.2 2.239 3.611 100.0 0 0
32 9387.5 1.951 3.215 101.0 0 0
33 8922.0 2.157 3.511 105.0 0 0
34 9060.3 2.091 3.497 105.0 0 0
35 9319.3 2.155 3.527 100.0 0 0
36 9266.6 2.020 3.377 97.0 0 0
37 8729.4 2.247 3.577 102.0 0 0
38 9514.8 2.063 3.267 99.0 0 0
39 9278.6 1.992 3.623 98.0 0 0
40 9420.1 2.016 3.411 103.0 0 0
41 9067.3 2.201 3.739 99.0 0 0
42 8803.8 2.371 3.843 102.0 0 0
43 9285.7 2.295 3.595 102.0 0 0
44 9375.0 2.183 3.629 96.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_42 1 9902.3 1.908 3.183 106.0 0 0
2 9491.0 2.065 3.231 93.0 0 0
3 9737.0 1.971 3.525 98.0 0 0
4 10001.6 1.913 2.985 90.0 0 0
5 9670.2 1.814 3.045 97.0 0 0
6 10208.7 1.597 2.621 90.0 0 0
7 9375.8 1.898 3.115 98.0 0 0
8 9872.6 1.762 2.879 98.0 0 0
9 9704.2 2.013 3.155 102.0 0 0
10 9786.5 1.887 3.077 94.0 0 0
11 9555.8 1.933 3.197 100.0 0 0
12 9226.7 2.013 3.365 104.0 0 0
13 9935.5 1.831 3.103 103.0 0 0
14 9274.3 2.187 3.539 103.0 0 0
15 10382.9 1.583 2.827 95.0 0 0
16 10061.4 1.866 2.971 99.0 0 0
17 9680.2 1.919 3.199 98.0 0 0
18 9466.7 1.875 3.155 101.0 0 0
19 8808.7 2.125 3.339 112.0 0 0
20 9466.2 2.095 3.319 106.0 0 0
21 8922.2 2.101 3.521 109.0 0 0
22 9109.5 2.321 3.523 108.0 0 0
23 8929.8 2.041 3.183 111.0 0 0
24 9197.2 2.217 3.397 106.0 0 0
25 8920.4 2.295 3.499 103.0 0 0
26 9010.9 2.275 3.575 108.0 0 0
27 8857.6 2.211 3.613 108.0 0 0
28 8741.6 2.291 3.465 101.0 0 0
29 8995.5 2.243 3.461 101.0 0 0
30 9485.3 2.191 3.349 102.0 0 0
31 9120.3 2.139 3.359 98.0 0 0
32 9749.1 2.117 3.385 104.0 0 0
33 8892.6 2.305 3.585 103.0 0 0
34 8613.3 2.351 3.965 107.0 0 0
35 9912.1 1.775 3.005 101.0 0 0
36 9062.7 2.321 3.519 99.0 0 0
37 9143.0 2.135 3.145 99.0 0 0
38 9155.2 2.129 3.303 103.0 0 0
39 8640.4 2.495 3.927 100.0 0 0
40 9151.1 2.187 3.501 97.0 0 0
41 9148.0 2.269 3.579 103.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_44 1 9733.1 2.219 3.395 105.0 0 0
2 9486.9 1.881 2.841 92.0 0 0
3 10179.0 1.601 2.567 98.0 0 0
4 11511.9 1.184 1.542 88.0 0 0
5 9784.3 1.939 2.827 98.0 0 0
6 10428.9 1.509 2.655 101.0 0 0
7 10237.9 1.613 2.539 102.0 0 0
8 10045.7 1.702 2.623 97.0 0 0
9 10198.9 1.629 2.581 90.0 0 0
10 10057.6 1.833 2.865 93.0 0 0
11 10659.8 1.663 2.809 93.0 0 0
12 9809.0 1.883 3.155 100.0 0 0
13 9918.5 1.809 2.895 96.0 0 0
14 10516.0 1.773 2.655 92.0 0 0
15 9609.4 1.917 3.017 99.0 0 0
16 9785.0 1.913 3.189 94.0 0 0
17 10211.2 1.726 2.643 98.0 0 0
18 10630.5 1.488 2.577 101.0 0 0
19 10297.3 1.821 2.817 94.0 0 0
20 9964.9 1.899 2.879 100.0 0 0
21 10241.9 1.669 2.643 96.0 0 0
22 8964.8 2.327 3.427 105.0 0 0
23 10046.4 1.807 2.877 104.0 0 0
24 10376.2 1.769 2.849 99.0 0 0
25 10062.1 1.891 3.031 100.0 0 0
26 9788.9 1.991 3.049 97.0 0 0
27 8859.0 2.171 3.173 102.0 0 0
28 8998.7 2.053 2.957 101.0 0 0
29 9174.2 1.931 3.011 103.0 0 0
30 8776.4 1.966 3.251 101.0 0 0
31 8729.0 2.161 3.273 94.0 0 0
32 8868.2 1.926 3.057 106.0 0 0
33 9073.5 1.788 3.167 102.0 0 0
34 9421.1 1.799 3.043 110.0 0 0
35 9289.1 1.952 3.405 99.0 0 0
36 9415.0 2.020 3.247 107.0 0 0
37 9467.2 2.073 3.083 106.0 0 0
38 8794.2 2.135 3.275 108.0 0 0
39 10006.6 1.756 2.417 92.0 0 0
40 8537.6 2.505 3.631 129.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_46 1 9837.7 1.997 3.339 101.0 0 0
2 8915.6 1.923 3.539 84.0 0 0
3 9572.6 1.943 3.367 86.0 0 0
4 9459.7 1.804 3.435 91.0 0 0
5 9833.7 1.822 3.279 89.0 0 0
6 9522.2 2.095 3.341 87.0 0 0
7 9670.0 1.874 3.321 89.0 0 0
8 9992.1 1.738 3.083 97.0 0 0
9 9707.6 1.914 3.497 92.0 0 0
10 9477.1 1.846 3.349 88.0 0 0
11 10080.2 1.891 3.305 93.0 0 0
12 9366.9 1.922 3.355 97.0 0 0
13 9475.7 1.830 3.303 98.0 0 0
14 9616.4 1.801 3.377 92.0 0 0
15 9877.0 1.819 3.235 87.0 0 0
16 9017.6 2.155 3.599 94.0 0 0
17 9540.9 1.885 3.469 88.0 0 0
18 9335.4 1.938 3.447 93.0 0 0
19 9713.7 1.927 3.279 93.0 0 0
20 9625.8 1.873 3.695 91.0 0 0
21 10048.4 1.798 3.427 94.0 0 0
22 9486.7 1.964 3.483 97.0 0 0
23 9866.6 1.921 3.135 88.0 0 0
24 10008.7 1.789 3.339 92.0 0 0
25 9179.7 2.032 3.809 97.0 0 0
26 8453.2 2.227 3.735 100.0 0 0
27 8249.5 2.505 3.869 104.0 0 0
28 8522.8 2.331 3.831 97.0 0 0
29 8086.6 2.597 4.227 102.0 0 0
30 8355.9 2.455 3.949 100.0 0 0
31 8808.3 2.261 3.863 100.0 0 0
32 8347.5 2.399 3.909 103.0 0 0
33 8173.8 2.791 4.243 95.0 0 0
34 8452.4 2.257 3.943 103.0 0 0
35 8503.5 2.389 3.953 102.0 0 0
36 7202.4 3.203 5.151 104.0 0 0
37 7515.7 3.305 5.247 110.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_48 1 9918.8 2.289 3.215 105.0 0 0
2 9540.7 2.267 3.457 98.0 0 0
3 9661.2 1.880 3.297 92.0 0 0
4 9697.4 1.802 3.023 98.0 0 0
5 9498.1 1.885 3.171 93.0 0 0
6 9940.4 1.764 3.177 94.0 0 0
7 9721.2 1.886 2.975 98.0 0 0
8 9590.1 1.733 3.151 101.0 0 0
9 9548.9 1.888 3.407 96.0 0 0
10 10069.8 1.695 2.879 100.0 0 0
11 9673.7 1.771 3.117 95.0 0 0
12 9298.4 1.883 3.347 105.0 0 0
13 8969.8 2.010 3.661 103.0 0 0
14 9700.9 1.818 3.267 104.0 0 0
15 9252.9 1.916 3.413 99.0 0 0
16 9227.0 1.969 3.341 101.0 0 0
17 8799.6 2.181 3.751 102.0 0 0
18 8977.0 1.970 3.357 103.0 0 0
19 8806.2 2.383 3.753 104.0 0 0
20 9140.5 1.998 3.383 100.0 0 0
21 9172.6 2.057 3.531 103.0 0 0
22 9135.8 1.856 3.395 101.0 0 0
23 8570.2 2.187 3.415 92.0 0 0
24 9280.1 1.887 3.229 98.0 0 0
25 8994.9 1.964 3.375 103.0 0 0
26 9400.9 1.849 3.149 92.0 0 0
27 9067.6 1.953 3.469 102.0 0 0
28 8668.8 2.269 3.755 99.0 0 0
29 8830.6 2.165 3.413 91.0 0 0
30 9539.4 1.965 3.183 96.0 0 0
31 9521.3 1.889 3.271 102.0 0 0
32 8923.4 2.245 3.927 109.0 0 0
33 8685.2 2.235 3.843 97.0 0 0
34 8852.0 2.221 3.591 95.0 0 0
35 8510.6 2.425 3.831 105.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_50 1 10696.4 1.856 3.265 95.0 0 0
2 11066.1 1.427 2.747 93.0 0 0
3 10668.9 1.816 2.923 80.0 0 0
4 11389.1 1.382 2.489 81.0 0 0
5 10432.5 1.550 2.833 93.0 0 0
6 10871.9 1.529 2.777 92.0 0 0
7 10649.9 1.744 2.959 82.0 0 0
8 10422.6 1.687 2.845 90.0 0 0
9 10360.2 1.688 2.963 94.0 0 0
10 10084.0 1.631 2.943 94.0 0 0
11 9292.9 2.107 3.595 102.0 0 0
12 9375.7 2.027 3.549 101.0 0 0
13 9436.3 2.103 3.383 99.0 0 0
14 9772.7 2.024 3.385 96.0 0 0
15 9321.3 1.931 3.285 90.0 0 0
16 9044.9 2.163 3.423 94.0 0 0
17 8691.9 2.241 3.763 94.0 0 0
18 8963.8 2.269 3.589 101.0 0 0
19 8828.6 2.275 3.901 96.0 0 0
20 8713.5 2.507 3.569 102.0 0 0
21 9127.1 2.119 3.471 97.0 0 0
22 8966.0 2.089 3.573 92.0 0 0
23 9171.5 2.145 3.709 98.0 0 0
24 9413.2 2.020 3.467 96.0 0 0
25 9590.1 2.055 3.369 99.0 0 0
26 8888.7 2.217 3.489 91.0 0 0
27 10474.7 1.575 2.969 91.0 0 0
28 9368.2 1.890 3.243 101.0 0 0
29 9830.3 1.743 3.023 92.0 0 0
30 9430.0 1.817 2.959 99.0 0 0
31 9380.0 1.820 3.279 96.0 0 0
32 9966.9 1.961 3.105 103.0 0 0
33 9903.6 1.885 3.125 100.0 0 0
34 9838.8 1.912 3.181 100.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_52 1 10325.5 1.838 2.991 104.0 0 0
2 10054.4 1.740 2.885 92.0 0 0
3 9717.3 2.024 2.947 86.0 0 0
4 9632.1 1.836 2.921 92.0 0 0
5 9653.1 1.752 2.759 97.0 0 0
6 9469.5 2.171 3.147 98.0 0 0
7 9409.2 2.051 2.987 92.0 0 0
8 9828.3 1.630 2.709 94.0 0 0
9 9823.5 1.854 2.947 97.0 0 0
10 9235.3 2.013 3.011 96.0 0 0
11 9108.1 2.046 3.141 97.0 0 0
12 9237.1 2.067 3.095 94.0 0 0
13 8917.3 2.131 3.337 91.0 0 0
14 9272.4 1.946 3.013 98.0 0 0
15 9102.2 2.079 3.117 97.0 0 0
16 8564.6 2.123 3.345 100.0 0 0
17 8763.0 2.185 3.459 100.0 0 0
18 8507.2 2.137 3.383 103.0 0 0
19 8188.1 2.331 3.603 102.0 0 0
20 9067.3 2.045 3.203 111.0 0 0
21 8189.5 2.487 3.751 109.0 0 0
22 8470.8 2.397 3.659 110.0 0 0
23 8260.0 2.335 3.613 109.0 0 0
24 7950.6 2.475 3.715 110.0 0 0
25 8182.0 2.255 3.677 109.0 0 0
26 8527.7 2.217 3.443 110.0 0 0
27 8949.3 1.812 3.079 109.0 0 0
28 8375.8 2.391 3.621 106.0 0 0
29 8548.0 2.249 3.513 115.0 0 0
30 8145.2 2.383 3.761 114.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_54 1 9716.9 2.251 3.771 103.0 0 0
2 9817.2 1.888 3.455 89.0 0 0
3 10212.2 1.724 2.779 86.0 0 0
4 9564.8 1.898 3.301 89.0 0 0
5 9797.8 1.819 3.227 97.0 0 0
6 9499.8 1.956 3.413 97.0 0 0
7 9812.2 1.698 3.115 95.0 0 0
8 9333.7 1.920 3.413 100.0 0 0
9 9985.2 1.789 3.091 99.0 0 0
10 9783.8 1.834 3.139 96.0 0 0
11 9183.9 1.987 3.343 103.0 0 0
12 9595.3 1.761 3.063 98.0 0 0
13 9366.8 2.037 3.379 98.0 0 0
14 8881.9 2.105 3.659 100.0 0 0
15 8948.9 1.989 3.349 102.0 0 0
16 8633.7 2.227 3.575 106.0 0 0
17 8934.8 2.077 3.543 109.0 0 0
18 9374.8 1.819 3.229 101.0 0 0
19 9233.9 2.021 3.195 102.0 0 0
20 9898.9 1.695 2.925 93.0 0 0
21 9388.9 2.065 3.281 97.0 0 0
22 9827.5 1.892 3.231 98.0 0 0
23 9526.5 1.846 3.287 104.0 0 0
24 9395.3 2.071 3.525 98.0 0 0
25 9054.0 2.083 3.799 101.0 0 0
26 9337.2 1.976 3.293 95.0 0 0
27 8976.5 2.209 3.619 99.0 0 0
28 9388.3 1.820 3.519 94.0 0 0
29 8543.6 2.079 3.717 98.0 0 0
30 9242.8 2.129 3.495 99.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_56 1 9333.7 2.325 3.507 99.0 0 0
2 9864.9 1.768 3.129 97.0 0 0
3 9676.2 1.782 2.971 91.0 0 0
4 9649.2 2.017 3.215 95.0 0 0
5 10138.5 1.859 2.873 93.0 0 0
6 9712.2 1.756 3.179 86.0 0 0
7 9576.3 1.890 3.251 96.0 0 0
8 9795.5 1.869 3.111 92.0 0 0
9 9463.6 1.718 3.019 92.0 0 0
10 9483.3 1.836 3.227 92.0 0 0
11 9910.6 1.830 2.751 98.0 0 0
12 9911.0 1.750 3.007 95.0 0 0
13 9455.4 1.859 3.043 98.0 0 0
14 9455.6 1.921 3.257 96.0 0 0
15 9147.8 1.946 3.307 99.0 0 0
16 8977.1 2.045 3.473 106.0 0 0
17 9028.9 2.089 3.391 109.0 0 0
18 9026.5 1.979 3.297 107.0 0 0
19 9302.8 1.914 3.373 110.0 0 0
20 8761.9 1.950 3.553 103.0 0 0
21 8893.1 1.915 3.671 107.0 0 0
22 8761.9 2.032 3.523 110.0 0 0
23 9159.7 1.970 3.489 105.0 0 0
24 9033.6 2.127 3.399 110.0 0 0
25 9225.4 1.991 3.231 109.0 0 0
26 8760.9 2.069 3.477 106.0 0 0
27 8443.7 2.147 3.915 107.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_58 1 9312.0 2.137 3.713 103.0 0 0
2 9490.9 2.005 3.485 98.0 0 0
3 9186.8 1.841 3.365 104.0 0 0
4 9481.5 1.857 3.413 97.0 0 0
5 9367.0 1.962 3.179 101.0 0 0
6 9378.8 1.967 3.383 92.0 0 0
7 9671.5 2.015 3.115 105.0 0 0
8 9123.3 2.041 3.407 101.0 0 0
9 8680.2 2.165 3.841 102.0 0 0
10 8622.5 2.001 3.717 104.0 0 0
11 8973.8 2.087 3.377 108.0 0 0
12 8275.8 2.227 3.883 106.0 0 0
13 9009.9 1.916 3.477 103.0 0 0
14 9024.7 2.057 3.265 101.0 0 0
15 9317.7 1.915 3.203 109.0 0 0
16 9496.5 1.784 3.133 98.0 0 0
17 9942.8 1.678 3.159 94.0 0 0
18 9323.1 1.879 3.225 102.0 0 0
19 9731.2 1.897 3.165 105.0 0 0
20 9585.1 1.917 3.373 103.0 0 0
21 9273.2 1.959 3.085 98.0 0 0
22 9100.1 1.943 3.279 95.0 0 0
23 9089.9 2.181 3.417 98.0 0 0
24 9768.0 1.645 3.219 101.0 0 0
25 8986.3 2.053 3.647 100.0 0 0
26 8941.3 2.051 3.475 98.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_60 1 8433.9 2.517 4.159 120.0 0 0
2 8451.2 2.289 4.083 102.0 0 0
3 8666.4 2.281 3.801 109.0 0 0
4 8996.9 2.149 3.609 104.0 0 0
5 8738.4 2.441 3.987 99.0 0 0
6 8595.6 2.235 3.667 99.0 0 0
7 8770.2 2.177 3.787 102.0 0 0
8 8924.4 2.311 3.825 100.0 0 0
9 8643.3 2.195 3.883 104.0 0 0
10 8833.5 1.957 3.485 100.0 0 0
11 9469.2 2.129 3.437 96.0 0 0
12 9154.2 2.111 3.569 99.0 0 0
13 8868.6 2.205 3.611 96.0 0 0
14 9083.0 2.133 3.635 98.0 0 0
15 9023.2 2.091 3.605 96.0 0 0
16 8213.6 2.289 4.099 98.0 0 0
17 9139.2 1.975 3.467 97.0 0 0
18 9179.4 2.137 3.247 94.0 0 0
19 9375.9 2.038 3.511 97.0 0 0
20 9088.7 2.251 3.515 95.0 0 0
21 8935.9 2.447 3.765 96.0 0 0
22 9203.4 2.269 3.751 96.0 0 0
23 8932.0 2.083 3.617 104.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_62 1 11019.8 2.023 3.285 95.0 0 0
2 11482.5 1.276 2.489 83.0 0 0
3 11270.0 1.462 2.765 81.0 0 0
4 12288.8 1.219 2.209 85.0 0 0
5 11498.6 1.396 2.431 85.0 0 0
6 11459.2 1.476 2.755 83.0 0 0
7 11657.3 1.663 2.941 84.0 0 0
8 12181.6 1.230 2.279 81.0 0 0
9 12366.4 1.196 2.219 87.0 0 0
10 10497.3 1.422 2.573 82.0 0 0
11 11198.5 1.504 2.713 93.0 0 0
12 10626.3 1.649 3.103 86.0 0 0
13 11531.9 1.287 2.413 85.0 0 0
14 11373.6 1.590 2.683 90.0 0 0
15 10818.2 1.599 2.991 90.0 0 0
16 10722.9 1.577 2.857 91.0 0 0
17 10835.6 1.485 2.675 87.0 0 0
18 10705.8 1.673 3.009 88.0 0 0
19 10586.7 1.694 3.035 85.0 0 0
20 11562.8 1.353 2.555 83.0 0 0
21 10439.4 1.740 3.233 85.0 0 0
22 11088.2 1.455 2.719 89.0 0 0
23 11387.4 1.442 2.435 85.0 0 0
24 10044.7 1.798 3.255 85.0 0 0
25 10722.0 1.615 3.175 85.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_64 1 8703.4 2.335 3.587 120.0 0 0
2 9099.7 1.965 3.373 99.0 0 0
3 9115.8 1.959 3.141 104.0 0 0
4 9201.6 1.837 3.261 100.0 0 0
5 8825.5 2.083 3.659 101.0 0 0
6 8691.9 2.203 3.869 102.0 0 0
7 9049.4 2.063 3.359 98.0 0 0
8 9164.1 1.988 3.181 100.0 0 0
9 9819.6 1.685 2.733 98.0 0 0
10 9773.1 1.731 2.923 104.0 0 0
11 9368.5 1.733 3.067 99.0 0 0
12 9418.5 1.795 3.149 98.0 0 0
13 9708.1 1.704 3.129 110.0 0 0
14 9020.0 1.952 3.329 99.0 0 0
15 9502.4 1.793 3.025 100.0 0 0
16 9617.7 1.795 2.907 106.0 0 0
17 9784.0 1.782 2.761 103.0 0 0
18 9555.3 1.851 3.031 98.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_66 1 8354.4 2.365 4.013 131.0 0 0
2 8310.3 2.319 4.029 104.0 0 0
3 8484.7 2.249 3.969 100.0 0 0
4 8693.6 2.185 3.893 105.0 0 0
5 8567.4 2.247 3.785 100.0 0 0
6 8390.5 2.085 3.745 110.0 0 0
7 8454.1 2.205 3.781 106.0 0 0
8 8770.5 2.183 3.639 102.0 0 0
9 9064.0 2.271 3.749 105.0 0 0
10 9582.5 1.974 3.405 99.0 0 0
11 9472.5 2.014 3.291 100.0 0 0
12 9409.1 2.145 3.443 100.0 0 0
13 9006.1 2.193 3.661 103.0 0 0
14 9477.1 1.873 3.393 107.0 0 0
15 8829.4 2.117 3.565 106.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_68 1 8498.1 2.419 3.941 107.0 0 0
2 8424.2 2.285 3.955 99.0 0 0
3 8379.2 2.309 3.831 115.0 0 0
4 8740.6 2.173 3.549 106.0 0 0
5 8437.4 2.221 3.531 104.0 0 0
6 8671.7 2.159 3.435 105.0 0 0
7 9103.7 2.045 3.235 105.0 0 0
8 8647.4 2.137 3.561 97.0 0 0
9 9198.3 1.892 3.169 102.0 0 0
10 9268.4 1.888 2.983 112.0 0 0
11 8643.9 2.217 3.553 109.0 0 0
12 8704.1 2.191 3.311 110.0 0 0
13 8496.2 2.285 3.533 106.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_70 1 9070.1 2.215 3.487 138.0 0 0
2 8651.9 2.375 3.765 99.0 0 0
3 8499.1 2.093 3.533 105.0 0 0
4 8629.1 2.209 3.811 108.0 0 0
5 8303.5 2.441 3.801 104.0 0 0
6 8517.3 2.359 3.779 105.0 0 0
7 8709.2 2.219 3.573 110.0 0 0
8 8945.9 2.131 3.273 113.0 0 0
9 8834.6 1.956 3.243 107.0 0 0
10 8411.4 2.291 3.563 111.0 0 0
11 8585.8 2.189 3.449 111.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_72 1 7997.5 2.969 4.077 121.0 0 0
2 8517.2 2.363 3.687 106.0 0 0
3 8565.7 2.169 3.637 104.0 0 0
4 8973.6 2.207 3.149 114.0 0 0
5 9590.1 1.776 2.879 114.0 0 0
6 9157.3 2.107 3.079 105.0 0 0
7 9563.6 1.940 2.831 100.0 0 0
8 8807.5 2.411 3.289 112.0 0 0
9 8273.8 2.853 4.003 113.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_74 1 7835.1 3.155 4.455 123.0 0 0
2 8789.1 2.389 3.865 108.0 0 0
3 8693.9 2.561 3.711 100.0 0 0
4 9097.7 2.075 3.237 102.0 0 0
5 9173.0 2.133 3.277 101.0 0 0
6 8848.9 2.273 3.557 110.0 0 0
7 8486.9 2.583 3.769 105.0 0 0
8 8368.2 2.351 3.515 102.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_76 1 9457.0 2.141 3.567 101.0 0 0
2 9148.5 2.040 3.377 99.0 0 0
3 9604.2 2.043 3.379 103.0 0 0
4 8980.9 1.994 3.331 106.0 0 0
5 9667.6 2.217 3.419 102.0 0 0
6 9133.8 2.275 3.545 103.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_78 1 8397.2 2.573 4.159 113.0 0 0
2 8501.2 2.024 3.669 103.0 0 0
3 8244.6 2.551 3.805 104.0 0 0
4 6939.2 3.041 4.575 118.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_80 1 7954.8 2.603 4.143 134.0 0 0
2 8712.1 2.371 3.553 150.0 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_82 1 7911.5 2.871 4.239 133.0 0 0

KMeans Detailed Results for Application Instance Runs
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_12 1 90.0 32.0 0 0
2 82.0 30.0 0 0
3 100.0 36.0 0 0
4 113.0 36.0 0 0
5 155.0 30.0 0 0
6 186.0 30.0 0 0
7 86.0 30.0 0 0
8 81.0 31.0 0 0
9 180.0 29.0 0 0
10 92.0 34.0 0 0
11 93.0 37.0 0 0
12 131.0 36.0 0 0
13 189.0 37.0 0 0
14 155.0 50.0 0 0
15 127.0 36.0 0 0
16 161.0 32.0 0 0
17 122.0 33.0 0 0
18 180.0 33.0 0 0
19 212.0 32.0 0 0
20 237.0 43.0 0 0
21 200.0 37.0 0 0
22 194.0 43.0 0 0
23 219.0 44.0 0 0
24 139.0 29.0 0 0
25 179.0 37.0 0 0
26 212.0 36.0 0 0
27 204.0 37.0 0 0
28 143.0 52.0 0 0
29 131.0 44.0 0 0
30 113.0 53.0 0 0
31 190.0 37.0 0 0
32 221.0 38.0 0 0
33 138.0 37.0 0 0
34 216.0 44.0 0 0
35 174.0 45.0 0 0
36 191.0 37.0 0 0
37 208.0 38.0 0 0
38 222.0 37.0 0 0
39 167.0 44.0 0 0
40 123.0 40.0 0 0
41 132.0 55.0 0 0
42 227.0 47.0 0 0
43 226.0 49.0 0 0
44 135.0 48.0 0 0
45 190.0 47.0 0 0
46 107.0 41.0 0 0
47 180.0 45.0 0 0
48 295.0 41.0 0 0
49 282.0 48.0 0 0
50 161.0 48.0 0 0
51 238.0 42.0 0 0
52 187.0 40.0 0 0
53 250.0 48.0 0 0
54 230.0 57.0 0 0
55 170.0 40.0 0 0
56 197.0 41.0 0 0
57 218.0 43.0 0 0
58 168.0 41.0 0 0
59 211.0 40.0 0 0
60 146.0 40.0 0 0
61 150.0 43.0 0 0
62 210.0 63.0 0 0
63 312.0 46.0 0 0
64 125.0 62.0 0 0
65 219.0 37.0 0 0
66 173.0 56.0 0 0
67 242.0 44.0 0 0
68 202.0 46.0 0 0
69 193.0 48.0 0 0
70 220.0 54.0 0 0
71 246.0 44.0 0 0
72 120.0 53.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_13 1 198.0 40.0 0 0
2 126.0 31.0 0 0
3 166.0 32.0 0 0
4 128.0 37.0 0 0
5 69.0 31.0 0 0
6 158.0 37.0 0 0
7 174.0 38.0 0 0
8 101.0 33.0 0 0
9 177.0 42.0 0 0
10 105.0 33.0 0 0
11 103.0 43.0 0 0
12 95.0 40.0 0 0
13 127.0 44.0 0 0
14 201.0 44.0 0 0
15 131.0 44.0 0 0
16 134.0 36.0 0 0
17 177.0 36.0 0 0
18 210.0 44.0 0 0
19 163.0 44.0 0 0
20 230.0 37.0 0 0
21 181.0 44.0 0 0
22 233.0 44.0 0 0
23 162.0 39.0 0 0
24 151.0 39.0 0 0
25 211.0 36.0 0 0
26 111.0 49.0 0 0
27 109.0 36.0 0 0
28 213.0 36.0 0 0
29 133.0 51.0 0 0
30 200.0 36.0 0 0
31 182.0 35.0 0 0
32 127.0 51.0 0 0
33 135.0 46.0 0 0
34 226.0 43.0 0 0
35 198.0 29.0 0 0
36 188.0 38.0 0 0
37 215.0 37.0 0 0
38 139.0 36.0 0 0
39 190.0 36.0 0 0
40 212.0 37.0 0 0
41 195.0 37.0 0 0
42 106.0 44.0 0 0
43 234.0 36.0 0 0
44 187.0 54.0 0 0
45 195.0 45.0 0 0
46 248.0 41.0 0 0
47 232.0 33.0 0 0
48 192.0 54.0 0 0
49 126.0 42.0 0 0
50 166.0 45.0 0 0
51 201.0 41.0 0 0
52 170.0 38.0 0 0
53 205.0 42.0 0 0
54 268.0 51.0 0 0
55 131.0 44.0 0 0
56 125.0 41.0 0 0
57 179.0 44.0 0 0
58 201.0 55.0 0 0
59 125.0 53.0 0 0
60 211.0 42.0 0 0
61 149.0 47.0 0 0
62 178.0 45.0 0 0
63 97.0 44.0 0 0
64 273.0 65.0 0 0
65 248.0 53.0 0 0
66 260.0 47.0 0 0
67 174.0 52.0 0 0
68 172.0 54.0 0 0
69 121.0 57.0 0 0
70 202.0 55.0 0 0
71 316.0 50.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_15 1 113.0 29.0 0 0
2 169.0 36.0 0 0
3 95.0 35.0 0 0
4 141.0 36.0 0 0
5 91.0 32.0 0 0
6 167.0 36.0 0 0
7 119.0 31.0 0 0
8 149.0 37.0 0 0
9 175.0 34.0 0 0
10 104.0 40.0 0 0
11 144.0 41.0 0 0
12 130.0 41.0 0 0
13 213.0 34.0 0 0
14 137.0 34.0 0 0
15 191.0 35.0 0 0
16 186.0 47.0 0 0
17 183.0 50.0 0 0
18 124.0 43.0 0 0
19 130.0 35.0 0 0
20 129.0 42.0 0 0
21 111.0 50.0 0 0
22 213.0 43.0 0 0
23 228.0 34.0 0 0
24 105.0 37.0 0 0
25 190.0 34.0 0 0
26 171.0 36.0 0 0
27 212.0 35.0 0 0
28 175.0 36.0 0 0
29 200.0 36.0 0 0
30 145.0 35.0 0 0
31 151.0 43.0 0 0
32 186.0 36.0 0 0
33 153.0 41.0 0 0
34 185.0 36.0 0 0
35 189.0 43.0 0 0
36 156.0 43.0 0 0
37 157.0 35.0 0 0
38 127.0 36.0 0 0
39 164.0 36.0 0 0
40 166.0 42.0 0 0
41 137.0 37.0 0 0
42 160.0 44.0 0 0
43 109.0 37.0 0 0
44 210.0 38.0 0 0
45 130.0 40.0 0 0
46 78.0 37.0 0 0
47 216.0 46.0 0 0
48 113.0 42.0 0 0
49 224.0 52.0 0 0
50 243.0 45.0 0 0
51 108.0 45.0 0 0
52 121.0 51.0 0 0
53 175.0 47.0 0 0
54 206.0 47.0 0 0
55 193.0 43.0 0 0
56 234.0 53.0 0 0
57 191.0 46.0 0 0
58 211.0 60.0 0 0
59 147.0 45.0 0 0
60 116.0 45.0 0 0
61 242.0 43.0 0 0
62 193.0 40.0 0 0
63 178.0 49.0 0 0
64 247.0 43.0 0 0
65 227.0 51.0 0 0
66 279.0 60.0 0 0
67 220.0 39.0 0 0
68 151.0 48.0 0 0
69 223.0 47.0 0 0
70 213.0 40.0 0 0
71 153.0 41.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_18 1 175.0 35.0 0 0
2 147.0 32.0 0 0
3 141.0 41.0 0 0
4 239.0 34.0 0 0
5 111.0 34.0 0 0
6 208.0 35.0 0 0
7 197.0 49.0 0 0
8 200.0 37.0 0 0
9 150.0 50.0 0 0
10 124.0 42.0 0 0
11 86.0 51.0 0 0
12 175.0 43.0 0 0
13 121.0 43.0 0 0
14 159.0 37.0 0 0
15 107.0 36.0 0 0
16 200.0 37.0 0 0
17 149.0 36.0 0 0
18 132.0 44.0 0 0
19 218.0 50.0 0 0
20 203.0 37.0 0 0
21 107.0 36.0 0 0
22 206.0 43.0 0 0
23 168.0 37.0 0 0
24 193.0 44.0 0 0
25 148.0 35.0 0 0
26 206.0 50.0 0 0
27 159.0 43.0 0 0
28 226.0 44.0 0 0
29 107.0 36.0 0 0
30 206.0 36.0 0 0
31 141.0 34.0 0 0
32 139.0 37.0 0 0
33 102.0 36.0 0 0
34 98.0 37.0 0 0
35 208.0 45.0 0 0
36 140.0 43.0 0 0
37 164.0 44.0 0 0
38 103.0 35.0 0 0
39 153.0 45.0 0 0
40 223.0 47.0 0 0
41 112.0 39.0 0 0
42 182.0 40.0 0 0
43 238.0 38.0 0 0
44 232.0 40.0 0 0
45 103.0 38.0 0 0
46 110.0 39.0 0 0
47 150.0 42.0 0 0
48 201.0 39.0 0 0
49 244.0 58.0 0 0
50 139.0 46.0 0 0
51 184.0 50.0 0 0
52 254.0 43.0 0 0
53 141.0 44.0 0 0
54 127.0 37.0 0 0
55 265.0 56.0 0 0
56 174.0 55.0 0 0
57 242.0 47.0 0 0
58 204.0 57.0 0 0
59 133.0 47.0 0 0
60 271.0 49.0 0 0
61 90.0 53.0 0 0
62 269.0 45.0 0 0
63 192.0 64.0 0 0
64 197.0 66.0 0 0
65 230.0 43.0 0 0
66 205.0 46.0 0 0
67 142.0 65.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_20 1 95.0 38.0 0 0
2 162.0 36.0 0 0
3 187.0 42.0 0 0
4 212.0 35.0 0 0
5 188.0 43.0 0 0
6 187.0 36.0 0 0
7 145.0 36.0 0 0
8 192.0 42.0 0 0
9 168.0 38.0 0 0
10 99.0 44.0 0 0
11 202.0 37.0 0 0
12 181.0 37.0 0 0
13 151.0 37.0 0 0
14 196.0 36.0 0 0
15 171.0 51.0 0 0
16 187.0 37.0 0 0
17 152.0 37.0 0 0
18 226.0 38.0 0 0
19 233.0 38.0 0 0
20 271.0 45.0 0 0
21 110.0 53.0 0 0
22 175.0 38.0 0 0
23 175.0 44.0 0 0
24 167.0 38.0 0 0
25 167.0 46.0 0 0
26 192.0 45.0 0 0
27 113.0 38.0 0 0
28 149.0 38.0 0 0
29 166.0 38.0 0 0
30 227.0 39.0 0 0
31 135.0 45.0 0 0
32 227.0 52.0 0 0
33 120.0 54.0 0 0
34 115.0 45.0 0 0
35 176.0 46.0 0 0
36 148.0 47.0 0 0
37 163.0 40.0 0 0
38 208.0 46.0 0 0
39 147.0 41.0 0 0
40 227.0 43.0 0 0
41 216.0 46.0 0 0
42 310.0 43.0 0 0
43 156.0 41.0 0 0
44 258.0 53.0 0 0
45 255.0 57.0 0 0
46 160.0 43.0 0 0
47 171.0 56.0 0 0
48 121.0 46.0 0 0
49 159.0 47.0 0 0
50 283.0 57.0 0 0
51 189.0 42.0 0 0
52 191.0 47.0 0 0
53 138.0 55.0 0 0
54 128.0 44.0 0 0
55 134.0 55.0 0 0
56 220.0 46.0 0 0
57 276.0 57.0 0 0
58 192.0 44.0 0 0
59 203.0 46.0 0 0
60 184.0 47.0 0 0
61 279.0 45.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_22 1 115.0 41.0 0 0
2 135.0 36.0 0 0
3 108.0 29.0 0 0
4 185.0 41.0 0 0
5 217.0 43.0 0 0
6 173.0 50.0 0 0
7 118.0 36.0 0 0
8 115.0 42.0 0 0
9 107.0 34.0 0 0
10 206.0 35.0 0 0
11 179.0 36.0 0 0
12 219.0 44.0 0 0
13 158.0 43.0 0 0
14 239.0 43.0 0 0
15 126.0 37.0 0 0
16 203.0 35.0 0 0
17 179.0 36.0 0 0
18 134.0 37.0 0 0
19 173.0 37.0 0 0
20 199.0 37.0 0 0
21 112.0 43.0 0 0
22 168.0 39.0 0 0
23 190.0 36.0 0 0
24 120.0 36.0 0 0
25 231.0 45.0 0 0
26 128.0 45.0 0 0
27 197.0 36.0 0 0
28 191.0 37.0 0 0
29 186.0 38.0 0 0
30 223.0 38.0 0 0
31 218.0 37.0 0 0
32 201.0 40.0 0 0
33 245.0 38.0 0 0
34 105.0 54.0 0 0
35 239.0 47.0 0 0
36 156.0 41.0 0 0
37 247.0 43.0 0 0
38 209.0 44.0 0 0
39 135.0 37.0 0 0
40 127.0 60.0 0 0
41 213.0 51.0 0 0
42 222.0 38.0 0 0
43 279.0 45.0 0 0
44 206.0 44.0 0 0
45 139.0 40.0 0 0
46 240.0 43.0 0 0
47 238.0 53.0 0 0
48 171.0 62.0 0 0
49 279.0 56.0 0 0
50 257.0 46.0 0 0
51 236.0 67.0 0 0
52 241.0 39.0 0 0
53 289.0 34.0 0 0
54 135.0 45.0 0 0
55 167.0 41.0 0 0
56 258.0 42.0 0 0
57 267.0 49.0 0 0
58 240.0 42.0 0 0
59 175.0 36.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_23 1 105.0 52.0 0 0
2 123.0 35.0 0 0
3 237.0 44.0 0 0
4 199.0 35.0 0 0
5 196.0 36.0 0 0
6 185.0 37.0 0 0
7 75.0 44.0 0 0
8 148.0 44.0 0 0
9 232.0 37.0 0 0
10 163.0 44.0 0 0
11 143.0 38.0 0 0
12 208.0 45.0 0 0
13 233.0 36.0 0 0
14 167.0 50.0 0 0
15 199.0 41.0 0 0
16 155.0 43.0 0 0
17 198.0 37.0 0 0
18 106.0 35.0 0 0
19 204.0 43.0 0 0
20 172.0 43.0 0 0
21 240.0 37.0 0 0
22 138.0 53.0 0 0
23 172.0 37.0 0 0
24 235.0 46.0 0 0
25 217.0 46.0 0 0
26 154.0 34.0 0 0
27 162.0 48.0 0 0
28 231.0 39.0 0 0
29 133.0 47.0 0 0
30 170.0 39.0 0 0
31 280.0 35.0 0 0
32 215.0 39.0 0 0
33 194.0 41.0 0 0
34 154.0 38.0 0 0
35 184.0 55.0 0 0
36 148.0 41.0 0 0
37 279.0 39.0 0 0
38 188.0 49.0 0 0
39 157.0 40.0 0 0
40 174.0 39.0 0 0
41 232.0 40.0 0 0
42 119.0 51.0 0 0
43 138.0 41.0 0 0
44 226.0 48.0 0 0
45 146.0 46.0 0 0
46 152.0 42.0 0 0
47 217.0 52.0 0 0
48 190.0 61.0 0 0
49 189.0 57.0 0 0
50 265.0 47.0 0 0
51 138.0 46.0 0 0
52 186.0 41.0 0 0
53 272.0 50.0 0 0
54 289.0 44.0 0 0
55 180.0 48.0 0 0
56 216.0 47.0 0 0
57 272.0 52.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_25 1 256.0 43.0 0 0
2 188.0 44.0 0 0
3 160.0 37.0 0 0
4 117.0 48.0 0 0
5 102.0 37.0 0 0
6 119.0 38.0 0 0
7 140.0 35.0 0 0
8 155.0 37.0 0 0
9 238.0 52.0 0 0
10 239.0 35.0 0 0
11 145.0 36.0 0 0
12 72.0 52.0 0 0
13 93.0 43.0 0 0
14 159.0 38.0 0 0
15 136.0 44.0 0 0
16 197.0 38.0 0 0
17 117.0 40.0 0 0
18 200.0 41.0 0 0
19 121.0 38.0 0 0
20 144.0 38.0 0 0
21 162.0 40.0 0 0
22 195.0 48.0 0 0
23 234.0 38.0 0 0
24 121.0 42.0 0 0
25 303.0 46.0 0 0
26 188.0 38.0 0 0
27 120.0 43.0 0 0
28 164.0 60.0 0 0
29 129.0 52.0 0 0
30 119.0 41.0 0 0
31 231.0 52.0 0 0
32 227.0 59.0 0 0
33 231.0 32.0 0 0
34 239.0 42.0 0 0
35 159.0 41.0 0 0
36 275.0 53.0 0 0
37 222.0 48.0 0 0
38 167.0 43.0 0 0
39 164.0 41.0 0 0
40 151.0 43.0 0 0
41 172.0 53.0 0 0
42 156.0 48.0 0 0
43 197.0 43.0 0 0
44 218.0 40.0 0 0
45 237.0 40.0 0 0
46 187.0 47.0 0 0
47 170.0 49.0 0 0
48 303.0 46.0 0 0
49 256.0 47.0 0 0
50 207.0 57.0 0 0
51 180.0 50.0 0 0
52 158.0 50.0 0 0
53 200.0 49.0 0 0
54 231.0 44.0 0 0
55 158.0 43.0 0 0
56 237.0 58.0 0 0
57 145.0 47.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_27 1 168.0 38.0 0 0
2 149.0 37.0 0 0
3 105.0 38.0 0 0
4 85.0 37.0 0 0
5 92.0 37.0 0 0
6 118.0 34.0 0 0
7 149.0 37.0 0 0
8 191.0 32.0 0 0
9 149.0 32.0 0 0
10 169.0 51.0 0 0
11 205.0 41.0 0 0
12 157.0 37.0 0 0
13 210.0 37.0 0 0
14 141.0 36.0 0 0
15 110.0 43.0 0 0
16 198.0 37.0 0 0
17 167.0 52.0 0 0
18 123.0 52.0 0 0
19 247.0 55.0 0 0
20 218.0 64.0 0 0
21 163.0 49.0 0 0
22 175.0 49.0 0 0
23 282.0 46.0 0 0
24 144.0 38.0 0 0
25 204.0 42.0 0 0
26 154.0 53.0 0 0
27 227.0 39.0 0 0
28 198.0 45.0 0 0
29 185.0 51.0 0 0
30 138.0 42.0 0 0
31 142.0 39.0 0 0
32 189.0 52.0 0 0
33 229.0 41.0 0 0
34 132.0 40.0 0 0
35 229.0 49.0 0 0
36 148.0 42.0 0 0
37 170.0 49.0 0 0
38 264.0 41.0 0 0
39 185.0 51.0 0 0
40 116.0 39.0 0 0
41 157.0 54.0 0 0
42 183.0 52.0 0 0
43 241.0 43.0 0 0
44 231.0 46.0 0 0
45 152.0 50.0 0 0
46 260.0 51.0 0 0
47 133.0 53.0 0 0
48 255.0 45.0 0 0
49 245.0 44.0 0 0
50 253.0 44.0 0 0
51 189.0 45.0 0 0
52 119.0 42.0 0 0
53 215.0 52.0 0 0
54 248.0 48.0 0 0
55 199.0 54.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_29 1 187.0 49.0 0 0
2 203.0 45.0 0 0
3 215.0 51.0 0 0
4 134.0 36.0 0 0
5 112.0 36.0 0 0
6 79.0 37.0 0 0
7 208.0 46.0 0 0
8 140.0 37.0 0 0
9 154.0 45.0 0 0
10 137.0 44.0 0 0
11 139.0 45.0 0 0
12 146.0 36.0 0 0
13 138.0 37.0 0 0
14 117.0 38.0 0 0
15 161.0 36.0 0 0
16 149.0 44.0 0 0
17 121.0 54.0 0 0
18 116.0 47.0 0 0
19 243.0 53.0 0 0
20 220.0 47.0 0 0
21 187.0 39.0 0 0
22 220.0 40.0 0 0
23 136.0 55.0 0 0
24 163.0 41.0 0 0
25 288.0 40.0 0 0
26 121.0 39.0 0 0
27 121.0 48.0 0 0
28 103.0 46.0 0 0
29 221.0 55.0 0 0
30 245.0 42.0 0 0
31 116.0 49.0 0 0
32 270.0 42.0 0 0
33 131.0 40.0 0 0
34 114.0 46.0 0 0
35 270.0 40.0 0 0
36 122.0 50.0 0 0
37 167.0 48.0 0 0
38 188.0 40.0 0 0
39 179.0 48.0 0 0
40 242.0 49.0 0 0
41 160.0 64.0 0 0
42 125.0 41.0 0 0
43 275.0 56.0 0 0
44 198.0 46.0 0 0
45 270.0 47.0 0 0
46 268.0 45.0 0 0
47 240.0 47.0 0 0
48 111.0 50.0 0 0
49 109.0 45.0 0 0
50 185.0 47.0 0 0
51 281.0 66.0 0 0
52 242.0 63.0 0 0
53 156.0 64.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_31 1 121.0 51.0 0 0
2 161.0 44.0 0 0
3 220.0 45.0 0 0
4 130.0 54.0 0 0
5 151.0 38.0 0 0
6 116.0 43.0 0 0
7 114.0 38.0 0 0
8 144.0 50.0 0 0
9 237.0 46.0 0 0
10 207.0 38.0 0 0
11 123.0 39.0 0 0
12 173.0 46.0 0 0
13 164.0 57.0 0 0
14 113.0 40.0 0 0
15 148.0 46.0 0 0
16 175.0 55.0 0 0
17 132.0 38.0 0 0
18 229.0 39.0 0 0
19 172.0 40.0 0 0
20 187.0 42.0 0 0
21 186.0 42.0 0 0
22 122.0 46.0 0 0
23 136.0 42.0 0 0
24 233.0 58.0 0 0
25 192.0 40.0 0 0
26 213.0 60.0 0 0
27 184.0 60.0 0 0
28 236.0 55.0 0 0
29 149.0 42.0 0 0
30 159.0 40.0 0 0
31 192.0 40.0 0 0
32 123.0 48.0 0 0
33 223.0 50.0 0 0
34 124.0 57.0 0 0
35 116.0 38.0 0 0
36 216.0 42.0 0 0
37 164.0 40.0 0 0
38 173.0 49.0 0 0
39 214.0 49.0 0 0
40 200.0 40.0 0 0
41 188.0 47.0 0 0
42 187.0 48.0 0 0
43 86.0 40.0 0 0
44 161.0 40.0 0 0
45 248.0 48.0 0 0
46 177.0 40.0 0 0
47 231.0 40.0 0 0
48 177.0 42.0 0 0
49 196.0 43.0 0 0
50 281.0 57.0 0 0
51 176.0 46.0 0 0
52 248.0 52.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_33 1 153.0 42.0 0 0
2 123.0 32.0 0 0
3 203.0 38.0 0 0
4 117.0 36.0 0 0
5 135.0 39.0 0 0
6 130.0 36.0 0 0
7 117.0 37.0 0 0
8 187.0 44.0 0 0
9 149.0 45.0 0 0
10 208.0 44.0 0 0
11 141.0 33.0 0 0
12 167.0 40.0 0 0
13 148.0 42.0 0 0
14 211.0 51.0 0 0
15 156.0 44.0 0 0
16 106.0 37.0 0 0
17 150.0 48.0 0 0
18 197.0 40.0 0 0
19 236.0 48.0 0 0
20 174.0 39.0 0 0
21 190.0 39.0 0 0
22 231.0 56.0 0 0
23 196.0 38.0 0 0
24 206.0 39.0 0 0
25 238.0 41.0 0 0
26 241.0 40.0 0 0
27 130.0 57.0 0 0
28 252.0 48.0 0 0
29 224.0 43.0 0 0
30 142.0 59.0 0 0
31 251.0 49.0 0 0
32 124.0 47.0 0 0
33 240.0 42.0 0 0
34 120.0 37.0 0 0
35 172.0 53.0 0 0
36 199.0 49.0 0 0
37 174.0 53.0 0 0
38 136.0 41.0 0 0
39 196.0 53.0 0 0
40 133.0 50.0 0 0
41 137.0 44.0 0 0
42 193.0 57.0 0 0
43 140.0 59.0 0 0
44 132.0 49.0 0 0
45 327.0 59.0 0 0
46 207.0 65.0 0 0
47 244.0 56.0 0 0
48 146.0 48.0 0 0
49 258.0 53.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_36 1 115.0 36.0 0 0
2 220.0 44.0 0 0
3 208.0 37.0 0 0
4 194.0 37.0 0 0
5 209.0 37.0 0 0
6 220.0 40.0 0 0
7 106.0 39.0 0 0
8 214.0 48.0 0 0
9 112.0 40.0 0 0
10 120.0 40.0 0 0
11 176.0 36.0 0 0
12 204.0 40.0 0 0
13 145.0 37.0 0 0
14 157.0 58.0 0 0
15 115.0 41.0 0 0
16 193.0 38.0 0 0
17 256.0 41.0 0 0
18 196.0 52.0 0 0
19 227.0 44.0 0 0
20 171.0 51.0 0 0
21 219.0 32.0 0 0
22 88.0 39.0 0 0
23 201.0 40.0 0 0
24 187.0 49.0 0 0
25 236.0 49.0 0 0
26 183.0 52.0 0 0
27 134.0 43.0 0 0
28 216.0 58.0 0 0
29 124.0 52.0 0 0
30 94.0 59.0 0 0
31 129.0 45.0 0 0
32 323.0 42.0 0 0
33 293.0 64.0 0 0
34 164.0 44.0 0 0
35 223.0 44.0 0 0
36 171.0 45.0 0 0
37 317.0 45.0 0 0
38 136.0 40.0 0 0
39 258.0 55.0 0 0
40 139.0 45.0 0 0
41 258.0 53.0 0 0
42 212.0 44.0 0 0
43 235.0 53.0 0 0
44 247.0 58.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_37 1 141.0 51.0 0 0
2 169.0 45.0 0 0
3 138.0 37.0 0 0
4 172.0 35.0 0 0
5 174.0 40.0 0 0
6 124.0 40.0 0 0
7 188.0 45.0 0 0
8 151.0 40.0 0 0
9 200.0 38.0 0 0
10 179.0 38.0 0 0
11 161.0 44.0 0 0
12 154.0 52.0 0 0
13 195.0 48.0 0 0
14 146.0 39.0 0 0
15 144.0 55.0 0 0
16 114.0 36.0 0 0
17 120.0 47.0 0 0
18 177.0 43.0 0 0
19 182.0 38.0 0 0
20 196.0 41.0 0 0
21 167.0 51.0 0 0
22 201.0 41.0 0 0
23 221.0 49.0 0 0
24 222.0 42.0 0 0
25 178.0 39.0 0 0
26 133.0 42.0 0 0
27 224.0 55.0 0 0
28 96.0 51.0 0 0
29 260.0 49.0 0 0
30 192.0 38.0 0 0
31 175.0 49.0 0 0
32 218.0 39.0 0 0
33 266.0 46.0 0 0
34 154.0 42.0 0 0
35 286.0 69.0 0 0
36 229.0 69.0 0 0
37 198.0 46.0 0 0
38 112.0 66.0 0 0
39 121.0 46.0 0 0
40 220.0 56.0 0 0
41 120.0 42.0 0 0
42 232.0 40.0 0 0
43 134.0 52.0 0 0
44 198.0 43.0 0 0
45 276.0 43.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_39 1 232.0 36.0 0 0
2 194.0 36.0 0 0
3 99.0 43.0 0 0
4 214.0 51.0 0 0
5 246.0 45.0 0 0
6 227.0 55.0 0 0
7 114.0 47.0 0 0
8 123.0 48.0 0 0
9 240.0 47.0 0 0
10 196.0 50.0 0 0
11 202.0 42.0 0 0
12 153.0 38.0 0 0
13 230.0 58.0 0 0
14 121.0 48.0 0 0
15 164.0 40.0 0 0
16 223.0 47.0 0 0
17 180.0 49.0 0 0
18 145.0 46.0 0 0
19 238.0 49.0 0 0
20 258.0 40.0 0 0
21 228.0 49.0 0 0
22 187.0 51.0 0 0
23 81.0 40.0 0 0
24 117.0 33.0 0 0
25 246.0 44.0 0 0
26 175.0 39.0 0 0
27 230.0 55.0 0 0
28 243.0 42.0 0 0
29 208.0 48.0 0 0
30 120.0 45.0 0 0
31 269.0 45.0 0 0
32 281.0 63.0 0 0
33 241.0 62.0 0 0
34 125.0 37.0 0 0
35 257.0 53.0 0 0
36 188.0 54.0 0 0
37 140.0 56.0 0 0
38 142.0 55.0 0 0
39 247.0 46.0 0 0
40 276.0 56.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_41 1 159.0 51.0 0 0
2 156.0 40.0 0 0
3 189.0 39.0 0 0
4 172.0 35.0 0 0
5 140.0 39.0 0 0
6 134.0 47.0 0 0
7 226.0 43.0 0 0
8 110.0 44.0 0 0
9 157.0 46.0 0 0
10 141.0 43.0 0 0
11 197.0 45.0 0 0
12 250.0 41.0 0 0
13 110.0 40.0 0 0
14 198.0 51.0 0 0
15 132.0 43.0 0 0
16 113.0 30.0 0 0
17 136.0 37.0 0 0
18 229.0 44.0 0 0
19 181.0 37.0 0 0
20 128.0 52.0 0 0
21 98.0 38.0 0 0
22 215.0 38.0 0 0
23 228.0 48.0 0 0
24 165.0 41.0 0 0
25 225.0 40.0 0 0
26 167.0 42.0 0 0
27 141.0 32.0 0 0
28 147.0 40.0 0 0
29 163.0 41.0 0 0
30 173.0 40.0 0 0
31 270.0 54.0 0 0
32 168.0 40.0 0 0
33 275.0 48.0 0 0
34 171.0 67.0 0 0
35 136.0 49.0 0 0
36 171.0 47.0 0 0
37 88.0 51.0 0 0
38 249.0 44.0 0 0
39 213.0 55.0 0 0
40 148.0 49.0 0 0
41 143.0 41.0 0 0
42 270.0 52.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_43 1 144.0 42.0 0 0
2 111.0 45.0 0 0
3 155.0 30.0 0 0
4 170.0 44.0 0 0
5 141.0 37.0 0 0
6 157.0 46.0 0 0
7 143.0 46.0 0 0
8 132.0 36.0 0 0
9 171.0 45.0 0 0
10 116.0 37.0 0 0
11 211.0 38.0 0 0
12 184.0 33.0 0 0
13 189.0 41.0 0 0
14 107.0 38.0 0 0
15 203.0 48.0 0 0
16 262.0 44.0 0 0
17 149.0 48.0 0 0
18 235.0 48.0 0 0
19 243.0 50.0 0 0
20 231.0 42.0 0 0
21 233.0 50.0 0 0
22 218.0 43.0 0 0
23 179.0 36.0 0 0
24 229.0 54.0 0 0
25 162.0 57.0 0 0
26 170.0 56.0 0 0
27 245.0 45.0 0 0
28 267.0 45.0 0 0
29 231.0 49.0 0 0
30 216.0 58.0 0 0
31 256.0 38.0 0 0
32 145.0 48.0 0 0
33 229.0 45.0 0 0
34 157.0 56.0 0 0
35 233.0 52.0 0 0
36 120.0 48.0 0 0
37 234.0 46.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_45 1 201.0 41.0 0 0
2 168.0 37.0 0 0
3 169.0 30.0 0 0
4 144.0 38.0 0 0
5 192.0 37.0 0 0
6 120.0 43.0 0 0
7 125.0 43.0 0 0
8 178.0 51.0 0 0
9 204.0 38.0 0 0
10 219.0 47.0 0 0
11 211.0 48.0 0 0
12 145.0 55.0 0 0
13 158.0 38.0 0 0
14 180.0 51.0 0 0
15 253.0 50.0 0 0
16 165.0 52.0 0 0
17 156.0 42.0 0 0
18 235.0 52.0 0 0
19 231.0 51.0 0 0
20 142.0 57.0 0 0
21 160.0 46.0 0 0
22 282.0 53.0 0 0
23 212.0 42.0 0 0
24 208.0 45.0 0 0
25 296.0 60.0 0 0
26 192.0 44.0 0 0
27 203.0 51.0 0 0
28 232.0 41.0 0 0
29 203.0 41.0 0 0
30 233.0 69.0 0 0
31 178.0 43.0 0 0
32 240.0 50.0 0 0
33 175.0 51.0 0 0
34 228.0 41.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_47 1 116.0 45.0 0 0
2 184.0 44.0 0 0
3 201.0 45.0 0 0
4 158.0 45.0 0 0
5 171.0 38.0 0 0
6 163.0 37.0 0 0
7 239.0 39.0 0 0
8 75.0 40.0 0 0
9 221.0 41.0 0 0
10 190.0 50.0 0 0
11 139.0 40.0 0 0
12 147.0 44.0 0 0
13 262.0 45.0 0 0
14 173.0 49.0 0 0
15 229.0 47.0 0 0
16 219.0 43.0 0 0
17 136.0 51.0 0 0
18 202.0 52.0 0 0
19 177.0 52.0 0 0
20 177.0 43.0 0 0
21 220.0 46.0 0 0
22 255.0 47.0 0 0
23 163.0 43.0 0 0
24 301.0 52.0 0 0
25 287.0 45.0 0 0
26 294.0 42.0 0 0
27 137.0 45.0 0 0
28 260.0 66.0 0 0
29 176.0 53.0 0 0
30 178.0 42.0 0 0
31 269.0 44.0 0 0
32 207.0 57.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_49 1 129.0 54.0 0 0
2 165.0 40.0 0 0
3 258.0 53.0 0 0
4 143.0 49.0 0 0
5 209.0 47.0 0 0
6 140.0 49.0 0 0
7 189.0 43.0 0 0
8 237.0 50.0 0 0
9 152.0 49.0 0 0
10 232.0 56.0 0 0
11 148.0 40.0 0 0
12 225.0 49.0 0 0
13 206.0 50.0 0 0
14 168.0 35.0 0 0
15 161.0 42.0 0 0
16 196.0 45.0 0 0
17 197.0 55.0 0 0
18 189.0 40.0 0 0
19 150.0 49.0 0 0
20 166.0 48.0 0 0
21 222.0 38.0 0 0
22 266.0 46.0 0 0
23 139.0 33.0 0 0
24 170.0 57.0 0 0
25 166.0 40.0 0 0
26 161.0 47.0 0 0
27 118.0 46.0 0 0
28 201.0 56.0 0 0
29 220.0 56.0 0 0
30 209.0 51.0 0 0
31 317.0 43.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_51 1 228.0 52.0 0 0
2 140.0 37.0 0 0
3 213.0 47.0 0 0
4 156.0 48.0 0 0
5 213.0 55.0 0 0
6 247.0 40.0 0 0
7 130.0 39.0 0 0
8 106.0 41.0 0 0
9 128.0 44.0 0 0
10 135.0 55.0 0 0
11 256.0 43.0 0 0
12 251.0 53.0 0 0
13 192.0 41.0 0 0
14 156.0 40.0 0 0
15 194.0 62.0 0 0
16 182.0 62.0 0 0
17 161.0 42.0 0 0
18 209.0 45.0 0 0
19 228.0 45.0 0 0
20 227.0 55.0 0 0
21 228.0 52.0 0 0
22 234.0 40.0 0 0
23 271.0 44.0 0 0
24 231.0 42.0 0 0
25 119.0 43.0 0 0
26 271.0 40.0 0 0
27 238.0 40.0 0 0
28 233.0 42.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_53 1 145.0 61.0 0 0
2 225.0 50.0 0 0
3 195.0 40.0 0 0
4 212.0 40.0 0 0
5 172.0 40.0 0 0
6 250.0 40.0 0 0
7 199.0 51.0 0 0
8 131.0 42.0 0 0
9 143.0 48.0 0 0
10 224.0 43.0 0 0
11 210.0 48.0 0 0
12 296.0 48.0 0 0
13 197.0 56.0 0 0
14 239.0 46.0 0 0
15 219.0 54.0 0 0
16 283.0 40.0 0 0
17 247.0 54.0 0 0
18 241.0 44.0 0 0
19 138.0 44.0 0 0
20 167.0 44.0 0 0
21 207.0 45.0 0 0
22 84.0 42.0 0 0
23 127.0 30.0 0 0
24 244.0 57.0 0 0
25 117.0 52.0 0 0
26 156.0 47.0 0 0
27 137.0 59.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_55 1 117.0 47.0 0 0
2 171.0 50.0 0 0
3 151.0 53.0 0 0
4 147.0 42.0 0 0
5 221.0 50.0 0 0
6 206.0 48.0 0 0
7 211.0 40.0 0 0
8 126.0 40.0 0 0
9 244.0 42.0 0 0
10 146.0 53.0 0 0
11 165.0 42.0 0 0
12 263.0 50.0 0 0
13 196.0 39.0 0 0
14 86.0 42.0 0 0
15 121.0 45.0 0 0
16 234.0 42.0 0 0
17 280.0 67.0 0 0
18 243.0 53.0 0 0
19 306.0 47.0 0 0
20 176.0 47.0 0 0
21 214.0 47.0 0 0
22 201.0 66.0 0 0
23 241.0 45.0 0 0
24 262.0 45.0 0 0
25 123.0 51.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_57 1 181.0 47.0 0 0
2 126.0 58.0 0 0
3 259.0 41.0 0 0
4 225.0 44.0 0 0
5 274.0 60.0 0 0
6 300.0 44.0 0 0
7 144.0 55.0 0 0
8 142.0 50.0 0 0
9 163.0 53.0 0 0
10 288.0 42.0 0 0
11 132.0 40.0 0 0
12 234.0 64.0 0 0
13 213.0 45.0 0 0
14 104.0 41.0 0 0
15 90.0 50.0 0 0
16 228.0 39.0 0 0
17 234.0 50.0 0 0
18 247.0 37.0 0 0
19 246.0 40.0 0 0
20 227.0 39.0 0 0
21 221.0 59.0 0 0
22 252.0 44.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_59 1 250.0 45.0 0 0
2 118.0 44.0 0 0
3 238.0 42.0 0 0
4 272.0 39.0 0 0
5 201.0 54.0 0 0
6 241.0 41.0 0 0
7 102.0 46.0 0 0
8 182.0 51.0 0 0
9 132.0 49.0 0 0
10 235.0 55.0 0 0
11 168.0 55.0 0 0
12 208.0 49.0 0 0
13 161.0 45.0 0 0
14 180.0 54.0 0 0
15 178.0 43.0 0 0
16 175.0 53.0 0 0
17 191.0 40.0 0 0
18 209.0 49.0 0 0
19 229.0 44.0 0 0
20 232.0 44.0 0 0
21 240.0 45.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_61 1 172.0 52.0 0 0
2 264.0 41.0 0 0
3 204.0 42.0 0 0
4 136.0 43.0 0 0
5 160.0 41.0 0 0
6 129.0 41.0 0 0
7 211.0 42.0 0 0
8 203.0 53.0 0 0
9 176.0 54.0 0 0
10 241.0 47.0 0 0
11 203.0 52.0 0 0
12 166.0 44.0 0 0
13 303.0 57.0 0 0
14 258.0 42.0 0 0
15 166.0 53.0 0 0
16 213.0 35.0 0 0
17 153.0 57.0 0 0
18 191.0 46.0 0 0
19 236.0 49.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_63 1 146.0 77.0 0 0
2 290.0 42.0 0 0
3 159.0 41.0 0 0
4 186.0 43.0 0 0
5 207.0 43.0 0 0
6 247.0 53.0 0 0
7 171.0 40.0 0 0
8 271.0 44.0 0 0
9 88.0 39.0 0 0
10 157.0 40.0 0 0
11 253.0 44.0 0 0
12 126.0 56.0 0 0
13 248.0 57.0 0 0
14 175.0 56.0 0 0
15 213.0 53.0 0 0
16 150.0 51.0 0 0
17 226.0 51.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_65 1 260.0 50.0 0 0
2 220.0 59.0 0 0
3 142.0 46.0 0 0
4 163.0 42.0 0 0
5 201.0 45.0 0 0
6 220.0 47.0 0 0
7 238.0 42.0 0 0
8 136.0 52.0 0 0
9 170.0 55.0 0 0
10 176.0 52.0 0 0
11 136.0 54.0 0 0
12 189.0 60.0 0 0
13 170.0 49.0 0 0
14 126.0 47.0 0 0
15 222.0 59.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_67 1 215.0 43.0 0 0
2 200.0 37.0 0 0
3 272.0 38.0 0 0
4 275.0 44.0 0 0
5 130.0 44.0 0 0
6 191.0 53.0 0 0
7 171.0 63.0 0 0
8 239.0 43.0 0 0
9 241.0 43.0 0 0
10 209.0 43.0 0 0
11 274.0 48.0 0 0
12 210.0 51.0 0 0
13 134.0 44.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_69 1 223.0 53.0 0 0
2 128.0 52.0 0 0
3 145.0 45.0 0 0
4 262.0 47.0 0 0
5 134.0 47.0 0 0
6 294.0 53.0 0 0
7 163.0 50.0 0 0
8 197.0 42.0 0 0
9 230.0 50.0 0 0
10 137.0 44.0 0 0
11 160.0 43.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_71 1 307.0 71.0 0 0
2 149.0 53.0 0 0
3 133.0 48.0 0 0
4 198.0 41.0 0 0
5 286.0 40.0 0 0
6 250.0 63.0 0 0
7 254.0 43.0 0 0
8 227.0 47.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_73 1 291.0 59.0 0 0
2 125.0 42.0 0 0
3 234.0 55.0 0 0
4 225.0 57.0 0 0
5 192.0 46.0 0 0
6 199.0 53.0 0 0
7 222.0 52.0 0 0
8 201.0 41.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_75 1 123.0 50.0 0 0
2 167.0 46.0 0 0
3 203.0 50.0 0 0
4 170.0 52.0 0 0
5 184.0 48.0 0 0
6 156.0 43.0 0 0
7 174.0 56.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_77 1 185.0 60.0 0 0
2 111.0 56.0 0 0
3 209.0 45.0 0 0
4 241.0 42.0 0 0
5 125.0 49.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_79 1 274.0 43.0 0 0
2 313.0 46.0 0 0
3 164.0 47.0 0 0
AI name Run id Completion Times (s) Data gen time (s) AI Run Errors NTP Error
ai_81 1 157.0 53.0 0 0

Glossary of terms (sorted alphabetically)

Application Instance

A group of instances created to run a single workload together. An application instance comprises a workload driver instance and set of instances, which are stressed by the workload driver. SPEC Cloud IaaS 2016 benchmark uses multiple application instances during specific phases to determine elasticity and scaling.

Baseline phase

In baseline, peak performance for each workload is determined in five separate test runs. During each workload run, instances are provisioned, data set is generated, load generator is started, results are accumulated, and the instances are destroyed. The workloads are run in a sequential fashion. Data from the baseline phase is used to establish parameters for the Elasticity + Scalability phase.

Benchmark phases

Benchmark has two phases, namely baseline and elasticity + scalability.

Blackbox Cloud

A cloud-provider provides a general specification of the SUT, usually in terms of how the cloud consumer may be billed. The exact hardware details corresponding to these compute units may not be known. This will typically be the case if the entity benchmarking the cloud is different from a cloud provider.

Cloud (system or service)

Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Cloud Consumer

A person or organization that is a customer of a cloud; note that a cloud customer may itself be a cloud and that clouds may offer services to one another.

Cloud Provider

An organization that provides cloud services to customers who pay for only the computing time and services actually used.

Elasticity

Elasticity measures whether the work performed by application instances scales linearly in a cloud. That is, for statistically similar work, the performance of N application instances in a cloud must be the same as the performance of application instances during baseline phase when no other load is introduced by the tester. Elasticity is expressed as a percentage (out of 100).

Elasticity + scalability phase

In the Elasticity + Scalability phase, new application instances are created, and they run the workloads concurrently to determine the elasticity and scalability metrics. The benchmark reports are generated at the end of elasticity + scalability phase.

Infrastructure as a Service (IaaS)

The Service Provider gives the Cloud-Consumer the capability to the provision processing, storage, network, and basic computing resources. They can also deploy and run arbitrary operating systems. The End-Consumer does not manage or control the underlying physical cloud infrastructure, but has control over the operating system, assigned storage, deployed applications, and limited control of select networking components (e.g., host firewalls).

Instance

An instance is an abstracted execution environment which presents an operating system (either discrete or virtualized). The abstracted execution environment presents the appearance of a dedicated computer with CPU, memory and I/O resources available to the operating system. In SPEC Cloud, an instance consists of a single OS and the application software stack that supports a single SPEC Cloud component workload. There are several methods of implementing an instance, including physical machines, virtual machines, or containers. An instance is created or destroyed using an API provided by an IaaS cloud.

Instance image

An image on the disk from which an instance is provisioned. Common formats for instance image include QCOW2 (Qemu copy on write 2), RAW, or AMI (Amazon machine image)

Physical machine

A set of connected components consisting on one or more general purpose processors (CPUs), memory, network connectivity, and mass storage either local (disk) or remote (network attached, block storage). The physical machine can have its own physical packaging or be installed as a blade in a blade chassis. An example would be a multi-core server with 4 GB of memory, 250 GB disk, and 1 Gb/s network adapter.

Provisioning Time

The measured time needed to bring up a new instance, or add more resources (like CPU or storage) to an existing instance. [Reference: CloudWhitePaper]

Within this document, this will be divided into two measurements:

1) Instance: The time from request to create a new instance until that instance responds to a netcat probe on port 22.

2) Application instance: The time from request to create a new instance until the associated cluster reports readiness to accept client requests. YCSB (Yahoo! Cloud Serving Benchmark) is ready when all nodes in the underlying database are part of the Cassandra cluster. K-Means is ready when the all nodes in HDFS are ready and part of the HDFS cluster.

Private Cloud

The cloud infrastructure is provisioned for exclusive use by a single organization comprising single or multiple consumers (e.g., business units). It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises.

Public Cloud

The cloud infrastructure is provisioned for open use by the general public. It may be owned, managed, and operated by a business, academic, or government organization, or some combination of them. It exists on the premises of the cloud provider.

Quality of Service (QoS)

The minimum percent (e.g., 95%) of collected values that complete within a predefined threshold in each measurement category. The Run and Reporting Rules document contains specific QoS limitations for each workload and benchmark.

Response Time

The time between when the work item request is issued until the corresponding completion condition. This definition is identical to the YCSB Latency metric.

Scalability

Scalability measures the total amount of work performed by application instances running in a cloud. The aggregate work performed by one or more application instances should linearly scale in an ideal cloud. It answers the question: How much more gets done if N instead of one (1) application instances? Ideally, each additional application instance will contribute 100% of its individual capability, but this may not be the case as the cloud gets loaded with multiple application instances or other workloads (for public clouds). Scalability is reported as a unit-less number @ the number of valid application instances. The number is an aggregate of workloads metrics across all application instances running in a cloud normalized by workload metrics in a reference cloud platform. The reference platform metrics are an average of workload metrics taken across multiple cloud platforms during the benchmark development.

SUT

One or more cloud services under test. This includes all hardware, network, base software and management systems used for the cloud service.

Variability

The difference between different runs of the benchmark. Typically, variability arises due to factors such as multi-tenancy and time of day execution, and may be more pronounced on public clouds.

Whitebox Cloud

The SUT's exact engineering specifications including all hardware and software are known and under the control of the tester. This will typically be the case for private clouds.