SPEC Cloud IaaS 2018 Benchmark
Copyright © 2018 Standard Performance Evaluation Corporation
Cloud Vendor: Inspur
Cloud Type / SUT Type: private/whitebox
Hardware Platform: x86
Hypervisor: KVM
Cloud Infrastructure: InCloudOpenStack V5		 Replicated Application Instances: 18 copies
        Performance Score: 20.1
        Relative Scalability: 85.5%
        Mean Instance Provisioning Time: 36s
Tested by: Inspur SPEC Licence Number : 3358 Test Date : Mar 14, 2020
Performance Sections
Performance Summary
Performance Details
Validation, Errors, and Issues
Glossary of Terms		 SUT Configuration Sections
Instance Configuration
Cloud Configuration
Network Configuration
Storage Configuration		 Scale-out Phase Date/Time and Test Region
Scale-out Start Time: 2020-03-16_12:40:18_UTC
Scale-out End Time: 2020-03-16_13:50:58_UTC
Test Region: CN Central Standard Time
Cloud Informational Metrics
AI Provisioning Success: 100.00%
AI Run Success: 100.00
Total Instances: 117
Reference ID: allrun-13

Performance Summary

Baseline Summary Results for YCSB
Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) AI Provisioning Time (s)
Average 44829.5 1.441 1.857 294.0

Baseline Summary Results for KMeans
Completion Time (s) AI Provisioning Time (s)
Average 108.28 280.0

Scale-out Summary Results for YCSB for 9 Valid AIs
Av. Throughput (ops/s) Av. Insert Latency 99% (ms) Av. Read Latency 99% (ms) Av. Provisioning Time (s) Performance Score Relative Scalability (%)
32480.4 2.390 3.113 291.8 8.74 74.66

Scale-out Summary Results for KMeans for 9 Valid AIs
Av. Completion Time (s) Av. Provisioning Time (s) Performance Score Relative Scalability (%)
112.8 284.0 11.33 96.39

YCSB Summary Results by AI
AI name Run count Run codes Av. Throughput (ops/s) Av. Insert Latency 99% (ms) Av. Read Latency 99% (ms) AI. Provisioning time (s) AI. Prov. Initiated from Scale-out Start (s)
ai_1 19 0 34255.6 2.230 2.698 291.0 1.0
ai_4 17 0 35363.8 2.085 2.411 289.0 515.0
ai_6 16 0 37582.0 1.928 2.276 287.0 956.0
ai_8 10 0 28765.2 2.833 3.419 288.0 1527.0
ai_10 9 0 34695.7 2.245 2.696 287.0 2068.0
ai_12 6 0 30373.3 2.496 3.185 295.0 2500.0
ai_14 5 0 31024.4 2.492 2.972 292.0 2846.0
ai_15 3 0 31378.4 2.378 3.212 298.0 3244.0
ai_17 1 0 28885.0 2.821 5.147 299.0 3705.0
ai_19 0 4 0.0 0.000 0.000 0.0 4024.0

KMeans Summary Results by AI
AI name Run count Run code Av. Completion Time (s) Av. Provisioning time (s) AI. Prov. Initiated from Scale-out Start (s)
ai_2 24 0 122.04 321.0 7.0
ai_3 24 0 113.25 274.0 329.0
ai_5 21 0 107.33 273.0 770.0
ai_7 19 0 108.42 274.0 1178.0
ai_9 15 0 105.93 278.0 1717.0
ai_11 12 0 109.00 279.0 2220.0
ai_13 8 0 115.00 282.0 2726.0
ai_16 4 0 136.00 290.0 3262.0
ai_18 1 0 112.00 285.0 3753.0
ai_20 0 4 0 0.0 4231.0

Validation, Errors, and Issues

Glossary of "Run code":

Instance Configuration

Instance Configuration Hadoop Master
Parameter Value
instance_type_name medium
cpu 4
memory 4GB
disk_size 40GB
disk_backed block storage
image_name hadoop_kmeans
Notes None

Instance Configuration Hadoop Slave
Parameter Value
instance_type_name medium
cpu 4
memory 4GB
disk_size 40GB
disk_backed block storage
image_name hadoop_kmeans
Notes None

Instance Configuration YCSB
Parameter Value
instance_type_name medium
cpu 4
memory 4GB
disk_size 40GB
disk_backed block storage
image_name cassandra_ycsb
Notes None

Instance Configuration Cassandra
Parameter Value
instance_type_name medium
cpu 4
memory 4GB
disk_size 40GB
disk_backed block storage
image_name cassandra_ycsb
Notes None

Cloud Configuration

Cloud High Level Information and Availability
Parameter Value
Name IncloudOpenStack
Type private
Sut_Type whitebox
Virtualized KVM
Manager_Protocol InCloudOpenStack
Manager_Version V5
Hardware_Availability Apr-2019
Software/Service_Availability Dec-2019
Geographic_Distribution 1
Notes Cloud management deputy nodes reside at all data centers SUT exists in 3 different buildings within same campus Campus network backbone is infinet10000.

Cloud Under Test Software Levels
Parameter Value
Python_version 2.7.9
JVM_version 1.8.0_45
Hadoop_version 2.5.7
Cassandra_version 2.1.20
Notes All software compiled locally from git source archives Compiler options - debug, 128bits, optimized Internal database set to emulate Cassandra API.

Benchmark Harness Node
Parameter Value
Vendor Inspur
Platform NF5280M5
CPU_Description Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU_Frequency_MHz 2300
RAM_Memory 768 GB, ECC, DDR4, 2666MHz RDIMM
Local_Storage 5x1 TB HDD, RAID 5, 7200 RPM, 1 500G SSD
Quantity 1
Availability_Date Apr-2019
OS_Version Ubuntu 16.04.2 LTS
OS_Availability_Date Nov-2019
Network_Adapter_1 Intel Corporation I350 Gigabit Network Connection
Network_Adapter_2 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
Network_port_type_1 1000BASE-T
Network_port_type_2 10GBASE-SR
Network_speed_port_1 1 Gb/s
Network_speed_port_2 10 Gb/s
Notes the control machine is KVM virtual machine

Cloud Manager Node
Parameter Value
Hw_Vendor Inspur
Hw_Platform NF5280M5
CPU_Type Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU_Count 2
CPU_Cores_Per 16
CPU_Frequency_MHz 2300
RAM_Memory_GB 768
Primary_Cache_KB 32 KB I + 32 KB D per core, on chip
Secondary_Cache_KB 1 MB I+D per core, on chip
RAM_Memory_Type ECC DDR4 2666 MHz
Hw_Availability_Date May-2018
Local_Storage 5x1 TB HDD, RAID 5, 7200 RPM, 1 500G SSD
Quantity 1
Virtualized KVM
Network_Adapter_1 Intel Corporation I350 Gigabit Network Connection
Network_Adapter_2 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
Network_port_type_1 1000BASE-T
Network_port_type_2 10GBASE-SR
Network_speed_port_1 1 Gb/s
Network_speed_port_2 10 Gb/s
OS_Version InCloudOpenStack V5
OS_Availability_Date Dec-2019
Notes None

Cloud Computes Nodes
Parameter Value
type 1_Compute_nodes 9
type 1_Hw_Vendor Inspur
type 1_Hw_Platform Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
type 1_Quantity 9
type 1_CPU_Type x86
type 1_CPU_Count 2
type 1_CPU_Cores_Per 16
type 1_CPU_Frequency_MHz 2300
type 1_RAM_Memory_GB 768
type 1_Primary_Cache_KB 32 KB I + 32 KB D per core, on chip
type 1_Secondary_Cache_KB 1 MB I+D per core, on chip
type 1_RAM_Memory_Type ECC DDR4 2666 MHz
type 1_Hw_Availability_Date Apr-2019
type 1_Local_Storage 5x 1 TB HDD, SATA3, RAID 5, 7200 RPM, 1 500G SSD
type 1_Virtualized KVM
type 1_Network_Adapter_1 Intel Corporation I350 Gigabit Network Connection
type 1_Network_Adapter_2 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
type 1_Network_port_type_1 1000BASE-T
type 1_Network_port_type_2 10GBASE-SR
type 1_Network_speed_port_1 1 Gb/s
type 1_Network_speed_port_2 10 Gb/s
type 1_OS_Version InCloudOpenStack V5
type 1_OS_Availability_Date Dec-2019
type 1_Notes None

Cloud Network Information

Cloud Network Information
Parameter Value
network 1_Technology ethernet
network 1_Protocol TCP/IP
network 1_Protocol_version IPV4
network 1_Speed_Mbps 10000
network 1_Function data
network 1_Notes None
network 2_Technology ethernet
network 2_Protocol TCP/IP
network 2_Protocol_version IPV4
network 2_Speed_Mbps 1000
network 2_Function management
network 2_Notes None

Cloud Schematic

Storage Configuration

Cloud Storage
Parameter Value
Storage 1_Attach_Type local
Storage 1_Capacity 36 TB
Storage 1_Technology HDD
Storage 1_Notes Each node has 5 HDD RAID 5 for storage Each node has 1 SSD for OS

Performance Details

YCSB Detailed Results for Application Instance Runs
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_1 1 37963.3 1.805 2.527 57.00 0 0
2 39988.8 1.819 2.289 51.00 0 0
3 39494.5 1.838 2.185 49.00 0 0
4 38989.4 1.829 2.163 51.00 0 0
5 41516.2 1.745 2.077 50.00 0 0
6 38638.4 1.818 2.203 52.00 0 0
7 40142.9 1.777 2.143 50.00 0 0
8 35482.4 2.053 2.289 50.00 0 0
9 34308.3 2.141 2.397 53.00 0 0
10 33855.8 2.189 2.511 62.00 0 0
11 34464.3 2.257 2.619 64.00 0 0
12 33920.4 2.189 2.583 58.00 0 0
13 31929.5 2.389 2.775 56.00 0 0
14 29740.4 2.615 3.029 59.00 0 0
15 29365.3 2.669 3.279 67.00 0 0
16 27868.2 2.689 3.391 63.00 0 0
17 29129.3 2.695 3.281 67.00 0 0
18 27750.2 2.821 3.487 65.00 0 0
19 26308.2 3.033 4.037 70.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_4 1 40947.9 1.570 2.391 50.00 0 0
2 42457.9 1.470 2.032 54.00 0 0
3 39555.8 1.680 2.040 50.00 0 0
4 38549.4 1.763 2.001 53.00 0 0
5 37635.0 1.754 2.121 48.00 0 0
6 37790.8 1.767 2.020 49.00 0 0
7 39751.2 1.766 2.057 52.00 0 0
8 36804.6 1.716 2.016 53.00 0 0
9 35138.3 2.187 2.297 51.00 0 0
10 33664.9 2.313 2.461 55.00 0 0
11 33925.3 2.209 2.505 55.00 0 0
12 32569.3 2.321 2.495 58.00 0 0
13 33548.6 2.395 2.577 53.00 0 0
14 30320.7 2.539 2.723 57.00 0 0
15 31270.0 2.481 2.767 62.00 0 0
16 29601.6 2.653 3.093 70.00 0 0
17 27653.1 2.861 3.383 76.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_6 1 42633.5 1.501 2.245 52.00 0 0
2 41476.6 1.647 2.099 52.00 0 0
3 44177.4 1.460 1.889 51.00 0 0
4 41198.9 1.641 1.987 53.00 0 0
5 41192.9 1.710 1.972 49.00 0 0
6 40871.8 1.647 1.955 54.00 0 0
7 39768.2 1.743 2.063 53.00 0 0
8 39381.3 1.670 1.934 55.00 0 0
9 37200.3 1.772 2.107 51.00 0 0
10 35143.8 2.031 2.303 57.00 0 0
11 36743.4 1.923 2.287 59.00 0 0
12 34300.6 2.141 2.499 63.00 0 0
13 32995.7 2.417 2.619 60.00 0 0
14 32751.4 2.407 2.721 60.00 0 0
15 30886.1 2.495 2.759 60.00 0 0
16 30590.6 2.645 2.971 59.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_8 1 31774.8 2.457 3.081 59.00 0 0
2 32461.2 2.521 2.953 57.00 0 0
3 31970.6 2.467 2.797 55.00 0 0
4 29643.2 2.715 3.035 60.00 0 0
5 27895.6 2.859 3.355 57.00 0 0
6 27151.0 2.981 3.777 65.00 0 0
7 27226.3 2.885 3.553 67.00 0 0
8 27560.7 2.945 3.569 67.00 0 0
9 26930.9 3.077 3.741 69.00 0 0
10 25037.6 3.421 4.327 66.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_10 1 34518.5 2.223 2.861 60.00 0 0
2 38263.6 2.009 2.473 57.00 0 0
3 35130.6 2.065 2.485 57.00 0 0
4 37196.8 1.966 2.325 55.00 0 0
5 35943.1 2.147 2.517 55.00 0 0
6 36109.6 2.089 2.477 59.00 0 0
7 31692.0 2.625 3.107 60.00 0 0
8 32396.3 2.481 2.921 65.00 0 0
9 31010.9 2.603 3.099 62.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_12 1 30352.2 2.565 3.425 66.00 0 0
2 30609.8 2.419 3.039 63.00 0 0
3 31421.6 2.405 2.861 69.00 0 0
4 31254.4 2.405 2.891 69.00 0 0
5 29959.2 2.513 3.005 73.00 0 0
6 28642.4 2.669 3.889 74.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_14 1 32452.8 2.363 3.051 61.00 0 0
2 30566.3 2.847 3.125 58.00 0 0
3 30310.8 2.491 2.853 57.00 0 0
4 31737.2 2.231 2.743 60.00 0 0
5 30054.9 2.529 3.089 64.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_15 1 31309.4 2.371 3.601 66.00 0 0
2 32631.0 2.261 2.785 64.00 0 0
3 30194.8 2.503 3.249 62.00 0 0
AI name Run id Throughput (ops/s) Insert Latency 99% (ms) Read Latency 99% (ms) Data gen time (s) AI Run Errors NTP Error
ai_17 1 28885.0 2.821 5.147 71.00 0 0

KMeans Detailed Results for Application Instance Runs
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_2 1 149 5 31.00 0 0
2 72 2 26.00 0 0
3 132 5 27.00 0 0
4 121 4 26.00 0 0
5 99 3 28.00 0 0
6 134 5 30.00 0 0
7 141 5 32.00 0 0
8 72 2 27.00 0 0
9 86 2 30.00 0 0
10 133 5 26.00 0 0
11 142 5 29.00 0 0
12 117 4 32.00 0 0
13 125 4 33.00 0 0
14 117 4 29.00 0 0
15 146 5 27.00 0 0
16 74 2 30.00 0 0
17 143 5 29.00 0 0
18 117 4 29.00 0 0
19 120 4 28.00 0 0
20 159 5 32.00 0 0
21 123 4 32.00 0 0
22 126 4 28.00 0 0
23 128 4 29.00 0 0
24 153 5 31.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_3 1 113 4 34.00 0 0
2 145 5 27.00 0 0
3 77 2 32.00 0 0
4 142 5 28.00 0 0
5 138 5 28.00 0 0
6 71 2 27.00 0 0
7 140 5 26.00 0 0
8 140 5 26.00 0 0
9 89 3 28.00 0 0
10 72 2 26.00 0 0
11 72 2 26.00 0 0
12 114 4 27.00 0 0
13 111 4 31.00 0 0
14 113 4 26.00 0 0
15 102 3 26.00 0 0
16 135 5 27.00 0 0
17 134 5 27.00 0 0
18 97 3 28.00 0 0
19 72 2 29.00 0 0
20 139 5 27.00 0 0
21 99 3 31.00 0 0
22 146 5 31.00 0 0
23 153 5 29.00 0 0
24 104 3 35.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_5 1 77 2 31.00 0 0
2 77 2 27.00 0 0
3 117 4 26.00 0 0
4 142 5 27.00 0 0
5 93 3 33.00 0 0
6 74 2 26.00 0 0
7 156 5 27.00 0 0
8 97 3 27.00 0 0
9 73 2 27.00 0 0
10 98 3 28.00 0 0
11 74 2 28.00 0 0
12 121 4 28.00 0 0
13 146 5 30.00 0 0
14 162 5 35.00 0 0
15 74 2 29.00 0 0
16 154 5 29.00 0 0
17 112 3 29.00 0 0
18 79 2 31.00 0 0
19 100 3 31.00 0 0
20 77 2 29.00 0 0
21 151 5 30.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_7 1 98 3 39.00 0 0
2 77 2 27.00 0 0
3 75 2 34.00 0 0
4 114 4 30.00 0 0
5 97 3 27.00 0 0
6 121 4 28.00 0 0
7 70 2 28.00 0 0
8 117 4 28.00 0 0
9 150 5 28.00 0 0
10 117 4 30.00 0 0
11 96 3 28.00 0 0
12 121 4 28.00 0 0
13 122 4 29.00 0 0
14 99 3 31.00 0 0
15 145 5 29.00 0 0
16 86 2 35.00 0 0
17 123 3 33.00 0 0
18 151 5 30.00 0 0
19 81 2 32.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_9 1 101 3 40.00 0 0
2 98 3 26.00 0 0
3 123 4 29.00 0 0
4 73 2 27.00 0 0
5 135 5 29.00 0 0
6 112 4 29.00 0 0
7 71 2 26.00 0 0
8 113 4 28.00 0 0
9 141 5 30.00 0 0
10 70 2 27.00 0 0
11 112 4 28.00 0 0
12 142 5 28.00 0 0
13 119 4 29.00 0 0
14 100 3 32.00 0 0
15 79 2 28.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_11 1 103 3 33.00 0 0
2 123 4 27.00 0 0
3 101 3 31.00 0 0
4 52 1 26.00 0 0
5 76 2 27.00 0 0
6 120 4 28.00 0 0
7 76 2 29.00 0 0
8 76 2 30.00 0 0
9 129 4 31.00 0 0
10 153 5 32.00 0 0
11 137 4 29.00 0 0
12 162 5 30.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_13 1 128 4 39.00 0 0
2 77 2 33.00 0 0
3 122 4 28.00 0 0
4 152 5 35.00 0 0
5 100 3 30.00 0 0
6 104 3 30.00 0 0
7 84 2 31.00 0 0
8 153 5 37.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_16 1 105 3 35.00 0 0
2 118 4 28.00 0 0
3 172 5 31.00 0 0
4 149 5 30.00 0 0
AI name Run id Completion Times (s) Hadoop Iterations Data gen time (s) AI Run Errors NTP Error
ai_18 1 112 3 33.00 0 0

Glossary

Glossary of Terms

SPEC and SPEC Cloud are registered trademarks of the Standard Performance Evaluation Corporation.

All other brand and product names appearing in this result are trademarks or registered trademarks of their respective holders.

For questions about this result, please contact the tester. For other inquiries, please contact info@spec.org