The OpenGL Performance Characterization (OPC) organization began in 1993 as an ad-hoc project group aimed at establishing graphics performance benchmarks for systems running under the OpenGL application programming interface (API). The group joined the GPC committee in the summer of 1994.

The non-profit OPC project is providing unambiguous methods for comparing the performance of OpenGL implementations across vendor platforms, operating systems, and windowing environments. Operating systems covered by the OPC group include, but are not limited to, OS/2, UNIX and Windows NT. Windowing environments include, but are not limited to, Presentation Manager, Windows and X. The intention of the OPC group is to characterize graphics performance for computer systems running applications, not overall graphics performance.

Current Viewperf Reports

Defining OpenGL Performance

The OPC group has established the following goals to define OpenGL performance:

• To permit standardized OpenGL performance measurement and evaluation by creating unambiguous, vendor-neutral measures for product evaluation and comparison.
• To provide mechanisms that enable vendors, customers and others to perform their own performance measurements and evaluations.
• To provide formal beta and final software releases to the public in a timely fashion.
• To develop a list of all those directly and materially affected by the benchmark specifications, and to offer the specifications for public review by all such parties with the possibility that, if there is consensus, the specifications might be offered for consideration by ANSI and other standards bodies.
• To contribute to the coherence of the field of OpenGL performance measurement and evaluation so that vendors will be better able to present well-defined measures, and customers will be better able to compare and evaluate vendors' products and environments.

Viewperf Paves the Way

The first benchmark released by the OpenGL Performance Characterization (OPC) group is Viewperf, which measures the 3D rendering performance of systems running under OpenGL. The OPC project group has worked with independent software vendors (ISVs) to obtain tests, data sets and weights that constitute what is called a viewset. Each viewset represents the graphics rendering portion of an actual application. Numbers are reported in this publication for five viewsets: Parametric Technology's CDRS, IBM's Data Explorer, Intergraph's DesignReview, Alias/Wavefront's Advanced Visualizer, and Lightscape Technologies' Lightscape Visualization System. The ISVs that develop OPC viewsets have provided percentage weights for each test for which a performance number is reported. ISVs have defined these percentages to indicate the relative importance of a test within the overall application.

Viewperf offers the following characteristics:

• It provides a single-source code for apples-to-apples comparison and performance tuning across different hardware platforms.
• It runs on multiple operating systems, including OS/2, UNIX and Windows NT.
• It runs across different processors, including Alpha, Intel, MIPS and PowerPC.
• It runs on multiple windowing environments, including Presentation Manager, X and Windows.
• It encompasses a wide variety of OpenGL features and rendering techniques.
• It is easily accessible through the OPC project subcommittee, ftp and through OpenGL sample disk distribution.

Several factors make Viewperf unique from other benchmarks:

• It uses datasets that are designed for and used by real applications.
• It uses rendering parameters and models selected by independent software vendors (ISVs) and graphics users.
• It produces numbers based on frames per second, a measurement with which users can readily identify.
• It provides one number for each rendering path using one data set.

What Viewperf Measures

Viewperf measures performance for the following entities:

• 3D primitives, including points, lines, line_strip, line_loop, triangles, triangle_strip, triangle_fan, quads and polygons;
• attributes per vertex, per primitive and per frame;
• lighting;
• texture mapping;
• alpha blending;
• fogging;
• anti-aliasing; and
• depth buffering.

The Five-Step Program

Viewperf is not a single-number benchmark. In order to use it to its fullest advantage, ISVs and users need to relate the benchmark to their actual applications. Here are the five steps recommended for using Viewperf effectively:

1. Identify software code paths that are important to the application.
2. Identify the primitives used within the application.
3. Select datasets that are most appropriate to the application. The datasets should reflect the level of geometry and rasterization found in the application.
4. Identify attributes and the level at which they are applied (per vertex, per primitive or per frame).
5. Assign a weight to each path based on the percentage of time in each path and the importance of the path to the application.

What Viewperf Can't Do

Although Viewperf is a good tool for measuring OpenGL performance as it relates to applications, like all benchmarks it has limitations. Most important of these is that it cannot be used to compare performance across different application programming interfaces (APIs). Also, it does not run itself; users must participate in the benchmarking process. When testing and reporting results, Viewperf does not account for the following key factors:

• effects caused by switching primitives,
• input effects on the event loop,
• user interface rendering and management,
• complex motion of multiple models,
• effects of CPU load on the graphics subsystem,
• color index visual performance, and
• multi-context, multi-window effects.

Continued Improvements

Development of Viewperf within the OPC group is an ongoing process, with future enhancements designed to address key graphics applications issues not covered by the current benchmark. Viewperf is available via anonymous ftp.

The OPC group is also working on a performance measurement tool called GLperf, which is expected to be ready for reporting in April 1997. GLperf is designed to measure optimal performance of 2D and 3D graphics primitives across vendor platforms. Like Viewperf, GLperf is designed as a single-source code to run across all platforms and windowing systems supporting OpenGL. Both benchmarks will be maintained by the OPC project subcommittee.

Structure and Membership

The OPC project subcommittee is a non-profit group whose general costs are paid via membership fees or dues. Any company, individual, institution or other group can become a sustaining member at any point in the development phase by remitting these dues. The project subcommittee will put in place and maintain an appeals process through which any interested or affected party may register and have heard a complaint concerning committee action or inaction. The principles of due process and antitrust compliance are adhered to by the subcommittee at all times.

OPC project subcommittee membership includes the following benefits:

• participation in all subcommittee business and technical activities,
• direct input in developing all releases of OPC benchmark software,
• access to pre-release versions of OPC benchmark software,
• publication of performance reports in OPC publications, and
• active participation in the planning and development of future versions of OPC benchmarks.

Future Plans

Performance measurement for OpenGL under Viewperf and GLperf is just the beginning for the OPC group. Future plans call for the group to develop benchmarks that encompass a wide range of primitives and attributes, including 2D geometry, 3D geometry, and image data. Planned enhancements for Viewperf would address other graphics applications issues such as selecting objects, CPU load, and additional OpenGL features. The ultimate challenge, at least for now, is to develop tools to consistently measure performance for OpenGL applications.

For more information regarding OPC project membership, contact John Spitzer, the group's chair, at 415-933-4760, or e-mail the OPC alias at opc@sgi.sgi.com.