Defects Identified in SPECjbb®2013
December 9, 2014 - SPEC has identified a defect in its SPECjbb®2013 benchmark
suite. SPEC has suspended sales of the benchmark software and is no longer accepting new submissions of
SPECjbb®2013 results for publication on SPEC's website. Current
SPECjbb®2013 licensees will receive a free copy of the new version of
the benchmark when it becomes available.
SPEC is advising SPECjbb®2013 licensees and users of the
SPECjbb®2013 metrics that the recently discovered defect impacts
the comparability of results. This defect can significantly impact the amount of work done during the
measurement period, resulting in an inflated SPECjbb®2013 metric. SPEC
recommends that users not utilize these results for system comparisons without a full understanding of the
impact of these defects on each benchmark result.
Additional information is available here.
The SPECjbb®2013 benchmark has been developed
from the ground up to measure performance based on the latest Java application features. It is
relevant to all audiences who are interested in Java server performance, including JVM vendors,
hardware developers, Java application developers, researchers and members of the academic
New features include:
- A usage model based on a world-wide supermarket company with an IT infrastructure that handles a mix of point-of-sale requests, online purchases and data-mining operations.
- Both a pure throughput metric and a metric that measures critical throughput under service level agreements (SLAs) specifying response times ranging from 10ms to 500ms.
- Support for multiple run configurations, enabling users to analyze and overcome bottlenecks at multiple layers of the system stack, including hardware, OS, JVM and application layers.
- Exercising new Java 7 features and other important performance elements, including the latest data formats (XML), communication using compression, and messaging with security.
- Support for virtualization and cloud environments.
- Submitted Results
- Includes all of the
results submitted to SPEC from licensees of the benchmark.
Search across all SPECjbb®2013 benchmark results.
- SPECjbb®2013 Benchmark Press Release
- Benchmark announcement day press release materials:
- SPECjbb®2013 Benchmark Documentation
- Issues Building/Running the SPECjbb®2013 Benchmark
- Installation, build, and runtime issues raised by users of the benchmark. (Last updated: Apr. 16, 2014)