SPECopc Overview

SPECopc Project Rules

Q. What is SPECglperf™?

A. SPECglperf is an executable benchmark that measures the performance of OpenGL 2D and 3D graphics operations. These operations are low-level primitives (points, lines, triangles, pixels, etc.) rather than entire models such as those used in the Viewperf benchmark.

Q. What is the purpose of SPECglperf?

A. The purpose of SPECglperf is two-fold: to standardize the method by which vendors report OpenGL primitive performance on their systems, and to use interactive graphs and other data analysis tools to present a much fuller performance picture than that provided by isolated primitive numbers.

Q. Who has developed SPECglperf and who will maintain it?

A. The benchmark was developed and will be maintained by the OpenGL Performance Characterization (SPECopc^SM) project group, a graphics benchmarking organization that operates under the auspices of the Graphics Performance Characterization (GPC) Group.

Q. What is a SPECglperf script?

A. A SPECglperf script describes the graphics primitives that will be included in performance tests. One script tests performance, for example, based on a 50-pixel triangle strip. A script allows many disparate primitives to be tested during one SPECglperf run and allows easy changes and comparisons between different tests.

Q. What SPECglperf scripts are used for reporting results?

A. The SPECopc project group has approved a set of 13 SPECglperf scripts for reporting results within its Web publication, the GPC News. These are split into 10 RGB scripts and three color index scripts. The scripts are further divided by functionality: the OPClist scripts (RGB and color index) contain a number of tests for a variety of graphics primitives and other operations (such as window-clears). These tests are probably the closest parallel to primitive-level results available from most vendors today. Other scripts feature specific graphics operations, such as CopyPixl.rgb, DrawPixl.rgb, ReadPixl.rgb, TexImage. rgb measure glCopyPixels, glDrawPixels, glReadPixels and glTexImage2D RGB operations. DrawPixl.ndx and ReadPixl.ndx are the color index analogs to DrawPixl.rgb and ReadPixl.rgb.

Remaining scripts address underlying graphics concepts that affect OpenGL performance -- BgnEnd.rgb measures performance as it varies with the number of primitives batched together (in a glBegin/glEnd pair), FillRate.rgb measures how fast rasterization operations are performed (how many pixels are drawn per second), Light.rgb measures the effect of the number of enabled light sources on drawing a particular primitive, and LineFill.rgb and TriFill.rgb measure the effect of increasing primitive size on the drawing rates of lines and triangles, respectively.

Q. Can vendors choose to omit results from some of the SPECglperf scripts or tests?

A. No, except for those vendors whose windowing systems or OpenGL implementations don't support the operations being tested. An example of this would be a vendor whose windowing system does not support color index windows, only RGB. This vendor could report all the RGB results, but omit the color index results.

Q. Why are there so many numbers, instead of just one?

A. One number is certainly preferable for marketing purposes. It does not serve justice, however, to boil down meaningful results into one meaningless number. Take the analogy of buying a car: one must compare many aspects of each car to arrive at the best buying decision. Most car shoppers do not take totally unrelated metrics and composite them for comparison purposes. They don’t, for example, take an average of the acceleration, top speed, trunk space, fuel efficiency, leg room, gas tank capacity and inverted braking distance. So why would one insist on doing so for something as complex as 3D graphics performance? Rather, one should choose the metrics most related to one's needs and compare using those figures. A CAD user, for example, might not care particularly how fast a system can transfer an image from memory to the screen, but this is vitally important to others, such as those in the film and video industry.

Q. What is the difference between SPECviewperf™ and SPECglperf?

A. Both SPECviewperf and SPECglperf measure the graphics performance of a system through the OpenGL API. They were designed, however, with different goals in mind. While Viewperf draws an entire model with differing sizes of primitives (as you would see in an actual application), SPECglperf artificially assigns a specific size to every primitive drawn within a test. While Viewperf attempts to emulate what an application would do graphically and measure it, SPECglperf makes no such attempt. Instead, SPECglperf provides a more controlled environment within which to extract and measure the highest performance or "upper bound" of a particular system.

Another difference is that Viewperf reports results in frames drawn per second, whereas SPECglperf measures its results in primitives drawn per second, whether the primitive is pixels, points, lines, triangles or some other object. To give an analogy to the automotive world, SPECglperf would be the equivalent of a speedometer measuring top speed, while Viewperf would be a stopwatch measuring the average speed through a slalom course.

Q. How do I look at all these numbers and make sense of them?

A. A substantial amount of work went into making the visualization and analysis of SPECglperf data as simple as possible. This goal is realized in the form of Java applets that read the SPECglperf performance results from the server's database, and allow tabular and graphical comparisons on the client's web browser. At the top level, the SPECglperf visualization tools are broken into two branches: the OPClist and the other scripts. Going down the OPClist branch, one can construct a performance comparison by selecting the desired primitives from one list, and the desired systems on another list. The other branch allows users to choose from the different SPECglperf scripts to display the data that most interests them. Each choice enables a table to be displayed with graphs on one axis and system configurations on the other. By selecting different cells, different sets of data will be displayed. Also, the axes can be linear or a log scale, allowing the most flexible plotting of data.

Q. What systems are capable of running SPECglperf?

A. Those systems supporting OpenGL drivers on OS/2, Windows 95/98, Windows NT and UNIX derivatives.
 
 

This document was prepared by John Spitzer, john.spitzer@raycer.com, chair of the SPECopc  project group.

If your browser refuses to load the following links, use [Shift] [left click] to save the file to disk.

• InstallShield for Windows, Intel Processor

By downloading the following, you acknowledge you have read, understand, and agree to abide by the terms of the License Agreement.

• Tar.Z for UNIX (compile it yourself). Also requires GLscrp06.tar.Z