Skip navigation

Standard Performance Evaluation Corporation

 
 

SPEC's Benchmarks

CPU

  • SPEC CPU2006
    [benchmark info] [published results] [support] [order benchmark]
    Designed to provide performance measurements that can be used to compare compute-intensive workloads on different computer systems, SPEC CPU2006 contains two benchmark suites: CINT2006 for measuring and comparing compute-intensive integer performance, and CFP2006 for measuring and comparing compute-intensive floating point performance.

  • SPEC CPUv6
    [info]
    The CPU Search Program seeks to to encourage those outside of SPEC to assist us in locating applications that could be used in the next CPU-intensive benchmark suite, currently designated as SPEC CPUv6.

  • SPEC CPU2000
    [Retired]

  • SPEC CPU95
    [Retired]

  • SPEC CPU92
    [Retired]

Graphics and Workstation Performance

Handheld

  • Handheld
    [info]
    SPEC has formed a committee chartered for the development of, and support for, a compute intensive benchmark suite for handheld devices.

High Performance Computing, OpenMP, MPI, OpenACC, OpenCL

  • SPEC ACCEL
    [benchmark info] [published results] [support] [order benchmark]
    SPEC ACCEL tests performance with a suite of computationally intensive parallel applications running under the OpenCL and OpenACC APIs. The suite exercises the performance of the accelerator, host CPU, memory transfer between host and accelerator, support libraries and drivers, and compilers.

  • SPEC MPI2007
    [benchmark info] [published results] [support] [order benchmark]
    MPI2007 is SPEC's benchmark suite for evaluating MPI-parallel, floating point, compute intensive performance across a wide range of cluster and SMP hardware. The suite consists of the intial MPIM2007 suite and MPIL2007, which contains larger working sets and longer run times than MPIM2007.

  • SPEC OMP2012
    [benchmark info] [published results] [support] [order benchmark]
    The successor to the OMP2001, designed for measuring performance using applications based on the OpenMP 3.1 standard for shared-memory parallel processing. OMP2012 also includes an optional metric for measuring energy consumption.

  • SPEC OMP2001
    [Retired]

  • SPEC HPC2002
    [Retired]

  • SPEC HPC96
    [Retired]

Java Client/Server

  • SPECjAppServer2004
    [Retired]

  • SPECjAppServer2002
    [Retired]

  • SPECjAppServer2001
    [Retired]

  • SPECjbb2013
    [Retired]

  • SPECjbb2005
    [Retired]

  • SPECjbb2000
    [Retired]

  • SPECjEnterprise2010
    [benchmark info] [published results] [support] [order benchmark]
    SPECjEnterprise2010 measures full-system performance for Java Enterprise Edition (Java EE) 5 or later application servers, databases and supporting infrastructure and expands the scope of the SPECjAppServer2004 benchmark.

  • SPECjms2007
    [benchmark info] [published results] [support] [order benchmark]
    SPECjms2007 is the first industry-standard benchmark for evaluating the performance of enterprise message-oriented middleware servers based on JMS (Java Message Service). It provides a standard workload and performance metrics for competitive product comparisons, as well as a framework for indepth performance analysis of enterprise messaging platforms.

  • SPECjvm2008
    [benchmark info] [published results] [support] [download benchmark]
    SPECjvm2008 is a benchmark suite for measuring the performance of a Java Runtime Environment (JRE), containing several real life applications and benchmarks focusing on core java functionality. The SPECjvm2008 workload mimics a variety of common general purpose application computations.

  • SPEC JVM98
    [Retired]

Mail Servers

Solution File Server (SFS)

Power

  • SPECpower_ssj2008
    [benchmark info] [published results] [support] [order benchmark]
    SPECpower_ssj2008 is the first industry-standard SPEC benchmark that evaluates the power and performance characteristics of volume server class computers. The initial benchmark addresses the performance of server-side Java, and additional workloads are planned.

  • Server Efficiency Rating Tool (SERT)
    [benchmark info] [support] [order benchmark]
    The Server Efficiency Rating Tool (SERT) was created by Standard Performance Evaluation Corporation (SPEC) at the request of the US Environmental Protection Agency. It is intended to measure server energy efficiency, initially as part of the second generation of the US Environmental Protection Agency (EPA) ENERGY STAR for Computer Servers program. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT uses a set of synthetic worklets to test discrete system components such as memory and storage, providing detailed power consumption data at different load levels.

SIP

  • SPECsip_infrastructure2011
    [benchmark info] [published results] [support] [order benchmark]
    SPEC's benchmark designed to evaluate a system's ability to act as a SIP server supporting a particular SIP application. The application modeled is a VoIP deployment for an enterprise, telco, or service provider, where the SIP server performs proxying and registration.

SOA

  • SOA
    [info]
    SPEC has formed a new subcommittee to develop standard methods of measuring performance for typical middleware, database and hardware deployments of applications based on the Service Oriented Architecture (SOA).

Virtualization

  • SPECvirt_sc2013
    [benchmark info] [published results] [support] [order benchmark]
    SPEC's updated benchmark addressing performance evaluation of datacenter servers used in virtualized server consolidation. SPECvirt_sc2013 measures the end-to-end performance of all system components including the hardware, virtualization platform, and the virtualized guest operating system and application software. The benchmark supports hardware virtualization, operating system virtualization, and hardware partitioning schemes.

  • SPECvirt_sc2010
    [Retired]

Web Servers

SPEC Tools

  • Server Efficiency Rating Tool (SERT)
    [info] [support] [order software]
    Intended to measure server energy efficiency, initially as part of the second generation of the US Environmental Protection Agency (EPA) ENERGY STAR for Computer Servers program. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT uses a set of synthetic worklets to test discrete system components such as memory and storage, providing detailed power consumption data at different load levels.

  • Chauffeur Worklet Development Kit (WDK)
    [kit info] [forum] [order software]
    Chauffeur was designed to simplify the development of workloads for measuring both performance and energy efficiency. Because Chauffeur contains functions that are common to most workloads, developers of new workloads can focus on the actual business logic of the application, and take advantage of Chauffeur's capabilities for configuration, run-time, data collection, validation, and reporting.

    Chauffeur was initially designed to meet the requirements of the SERT. However, SPEC recognized that the framework would also be useful for research and development purposes. The Chauffeur framework is now being made available as the Chauffeur Worklet Development Kit (WDK). This kit can be used to develop new workloads (or "worklets" in Chauffeur terminology). Researchers can also use the WDK to configure worklets to run in different ways, in order to mimic the behavior of different types of applications. These features can be used in the development and assessment of new technologies such as power management capabilities.

  • PTDaemon
    [info]
    The SPEC PTDaemon software is used to control power analyzers in benchmarks which contain a power measurement component.