Running Benchmark with a Simulated Cloud

Running the benchmark for a cloud can be involved. This includes setting of images, instances, and testing connectivity between benchmark harness and cloud.

The benchmark harness provides a simulated mode which allows a tester to get a flavor of the benchmark within couple of minutes while avoiding the vagaries of a cloud.

In simulated, no real instances are created nor the real workloads are run. Rather, CBTOOL ‘fakes’ the creation of instances and workload metrics using probability distributions.

To get started, open two terminals into the benchmark harness machine.

In the first terminal:

cd ~/osgcloud/cbtool
./cb --hard_reset

In the second terminal:

cd ~/osgcloud/driver
./all_run.sh RUN2

Where RUN2 is the id of the SPEC Cloud IaaS 2016 Benchmark run.

The results are stored in:

cd ~/results/RUN2/perf

The result directory is configurable from:

cd ~/osgcloud/driver
vi osgcloud_rules.yaml

results_dir: HOMEDIR/results

The following figure shows the overall data flow across different phases of the benchmark. These files are stored in:

cd ~/results/RUN2/perf_dir

Observe that the run id (‘RUN2’ in the above example) will be different if a different id is used in all_run.sh script.

CAUTION: instance_support_evidence flag must not be set to true for testing with a simulated cloud.

More details on the simulated cloud can be found on CBTOOL external documentation

Data flow

The following picture shows how different files and directories are generated as part of the run.