Credits for SPECjms2007

Creating a benchmark to provide a consistent, comparative measure of JMS-based messaging server performance is not simple. The challenges are numerous, for example:

choosing a realistic application scenario and defining a meaningful and representative set of workloads;
implementing the workloads in a platform-independent manner;
defining rules to govern how the workloads are compiled, run and reported;
developing a toolset to help enforce these rules while still making things easy to use; and
defining a process by which decisions can be made amongst the group working on the project.

To be useful and reliable, a JMS messaging benchmark must fulfill several fundamental requirements:

it must be designed to stress platforms in a manner representative of real-world messaging applications;
it must exercise all critical services provided by platforms and must provide a level playing field for performance comparisons;
it must be focused on the middle-tier;
it must generate reproducible results; and
it must not have any inherent scalability limitations.

And in the end, you hope you have something that is fair, relevant, and understandable, that will address technology for a period of years.

Such is the case with SPECjms2007, which is the first industry-standard benchmark designed specifically for enterprise messaging servers based on JMS. It provides a consistent workload and performance metrics for competitive product comparisons, as well as a framework for in-depth performance analysis of enterprise JMS messaging.

All of this is due to the efforts of a team of people from around the world, that SPEC would like to thank (with apologies and thanks to those who may not be listed).

SPECjms2007 Development Team

SPECjms2007 was developed by the SPECjms working group which is part of SPEC’s Java subcommittee with the participation of Technische Universität Darmstadt, IBM, Sun, Oracle, BEA, Sybase and Apache.

Thanks to all members of the SPECjms working group, in particular:

Samuel Kounev from TU Darmstadt for serving as project manager, benchmark architect and working group chair,

Kai Sachs from TU Darmstadt for serving as benchmark architect, designer and lead developer,

Marc Carter from IBM for serving as responsible developer for the design and implementation of the driver, automation, and reporting framework,

George Tharakan from Sun Microsystems for his invaluable contributions to the specification and design of the workload as well as to the definition of the run and reporting rules,

Martin Ross from IBM, Binu John, Eileen Loh, Ken Dyer and Sagar Shirguppi from Sun Microsystems, Russell Raymundo and Tom Barnes from BEA, Anoop Gupta from Oracle, Sebastian Frischbier from TU Darmstadt and Evan Ireland from Sybase for their hard work on testing, profiling and debugging the benchmark,

Adrian Co from Apache for his excellent work on implementing the reporter,

Tim Dunn from IBM, Saraswathy Narayan from Oracle and Tom Barnes from BEA for their contributions to the specification of the benchmark run and reporting rules.

Many thanks also to

Alejandro Buchmann from TU Darmstadt, Lawrence Cullen, Robert Berry, Alan Adamson, John Stecher and Matt Hogstrom from IBM, Steve Realmuto from BEA and Ricardo Morin from Intel for their continued support of the SPECjms2007 project.

The German Research Foundation (Deutsche Forschungsgemeinschaft) for funding Samuel Kounev's work as part of grant No. KO 3445/2-1.

SPEC

We would also like to thank:

SPEC management, who provided us with resources, and kept the SPEC Java Subcommittee on their toes:
- Walter Bays - President, SPEC
- Kaivalya Dixit - Long-time President, and Guiding Light of SPEC
- Larry Gray - Treasurer, SPEC
- Alan Adamson, Chair, SPEC Open Systems Steering Committee
The SPEC office, who were the ones who really provided us with the resources:
- Kathy Powers - Administration, including event coordination: SPEC face-to-face meetings
- Dianne Rice - Administration
- Cathy Sandifer, IT/Webmaster/Support
- Jason Glick - IT/Engineering/Support

Behind the scenes

Finally, we thank all of the people behind the scenes in architecture groups, product development and performance groups who supported their work for SPEC at their respective companies.

Samuel Kounev, Release Manager, SPEC Java Subcommittee