SPEC Virtualization Committee

The SPEC Virtualization Committee is developing a new industry standard benchmark for evaluating virtualization performance for data center servers.

Over the last several years the interest in using virtualization techniques to optimize data center efficiency has grown, particularly in the area of server consolidation. Virtualizing a number of servers onto a single platform can help reduce costs, save energy, and ease IT infrastructure management. IT professionals and decision makers considering virtualization for server consolidation will benefit from the availability of a standard benchmark from SPEC to compare servers and different virtualization options.

The committee plans to make use of the comprehensive system-level application-based benchmarks that SPEC offers. The workloads represented by these benchmarks are also representative of commonly virtualized server applications. SPEC's expertise in system level benchmarks that support a wide range of hardware architectures and operating systems will greatly benefit the committee's efforts to develop a standard methodology to evaluate the performance of servers using virtualization for server consolidation.

Benchmark Goals

The goals for SPEC's first virtualization benchmark include:

Delivering a benchmark that will model server consolidation of commonly virtualized systems such as mail servers, database servers, application servers, web servers, and file servers.
Providing a means to fairly compare server performance while running a number of virtual machines (VMs).
Producing a benchmark designed to scale across a wide range of systems.
Supporting hardware virtualization, operating system virtualization, and hardware partitioning schemes for server consolidation scenarios.

Current Status

We are currently evaluating load levels for individual workloads, researching the effects of periodic burstiness in some workloads and have started building VM prototypes for various workloads. The group is continuing to build consensus and has agreed on an overall benchmark design. A specific release date has not been set at this early stage, as unanticipated issues may arise during the development and additional time may be needed to addresses these issues.

Future Efforts

Once SPEC has released its first Virtualization benchmark, the committee plans to look at applying the SPECpower methodology for energy efficiency to the benchmark. The committee may also look at other data center scenarios where virtualization can be applied.

Committee Membership

Current SPEC member companies committed to developing an industry standard virtualization benchmark include AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.

SPEC welcomes organizations to join and participate in our work, please contact info@spec.org for additional information.

Standard Performance Evaluation Corporation