|
|
|
SPECviewperf™ is a portable OpenGL performance benchmark program written in C. It was developed by IBM. Later updates and significant contributions were made by SGI, Digital and other SPECopcSM project group members. SPECviewperf provides a vast amount of flexibility in benchmarking OpenGL performance. Currently, the program runs on most implementations of UNIX, Windows NT, Windows 95/98 and Linux.
The OpenGL Performance Characterization (SPECopc) project group has endorsed SPECviewperf as its first OpenGL benchmark. Performance numbers based on SPECviewperf were first published in the Q4 1994 issue of The GPC Quarterly.
SPECopc group member companies have ported the SPECviewperf code to their operating systems and window environments. The SPECopc project group maintains a single source code version of the SPECviewperf code that is available to the public.
SPECviewperf reports performance in frames per second. Other information about the system under test -- all the rendering states, the time to build display lists (if applicable), and the data set used -- are also output in a standardized report.
A "benchmark" using SPECviewperf is really a single invocation of SPECviewperf with command-line options telling the SPECviewperf program which data set to read in, which texture file to use, what OpenGL primitive to use to render the data set, which attributes to apply and how frequently, whether or not to use display lists, and so on. One quickly realizes that there are an infinite number of SPECviewperf "benchmarks" (an infinite number of data sets multiplied by an almost infinite number of command-line states).
Viewsets are generally not developed by the SPECopc project group; they come from the ISVs themselves. Members of the SPECopc project group often "sponsor" the ISV. Sponsorship entails helping the ISV in several areas, including how to obtain the SPECviewperf code, how to convert data sets to a SPECviewperf format, how to use SPECviewperf, how to create SPECviewperf tests to characterize the application, how to determine weights for each of the individual SPECviewperf tests based on application usage, and finally to help offer the viewset to the SPECopc project group for consideration as a standard SPECopc viewset. Any ISV wishing to develop a viewset should contact an SPECopc representative listed on the GPC Organization page.
Currently, there are five standard SPECopc application viewsets and one 'generic' viewset: ProCDRS viewset is intended to model the graphics performance of Parametric Technology Corporation's CDRS industrial design software. IBM's Data Explorer (DX), which has 10 different tests, is a visualization application. Intergraph's DesignReview (DRV), which now has 5 different tests, is a 3D computer model review package. Alias/Wavefront's Advanced Visualizer, now with 11 tests, is an animation application. And Discreet Logic's Lightscape Visualization System, with four tests, is a radiosity visualization application. These type of applications typically render large data sets. They almost always include lighting, smooth shading, blending, line antialiasing, z-buffering, and some texture mapping. Finally, the newest viewset is medMCAD. This is a 'generic' viewset that models the graphics performance of a range of immediate mode, MCAD applications suitable for medium sized models. It contains twelve tests. All six viewsets represent relatively high-end applications.
The input to SPECviewperf is the command line. Here are the current command line options for SPECviewperf 6.1.2:
Program options:
SPECviewperf v6.1.2 Program options: -polygon -pg <file> : Viewpoint object to be used in the tests -triangle -tr <file> : Viewpoint object to be used in the tests -quad -qd <file> : Viewpoint object to be used in the tests -mesh -mh <file> : Mesh object to be used in the tests -rendermode -rm <mode> : POINT, VECTOR, LINE, POLYGON, TMESH, TFAN, TRIANGLE, or QUAD - default LINE -vcriteria -vcrit : AUX Visual selection criteria - EXACT, MIN- default MIN -vid <id> : Ask AUX for visual with ID = <id> -vaccum -vac : Ask AUX for an accumulation buffer visual -valpha -val : Ask AUX for an alpha buffer visual -vdepthbuffer -vz : Ask AUX for a depth buffer visual -vstencil -vst : Ask AUX for a stencil buffer visual -indirectrender -ir : Render indirect - default direct -nodither -ndi : Disable dithering -ortho -or : Parallel/Orthographic projection - default Perspective -displaylist -dl : Render with display list mode -vertexarray -va : Render with vertex array mode (OpenGL1.1) -interleaved -il : Packing vertex data interleaved in memory -colorper -cp <mode> : FRAME = Color per Frame, PRIMITIVE = Color per Primitive, VERTEX = Color per Vertex - default FRAME -texture -tx <file> : Image for texturing -texgen -txg <file> <mode> : <file> is image for enviroment mapping <mode> is SPHERE_MAP, OBJECT_LINEAR, EYE_LINEAR - default EYE_LINEAR -texobj -txo <file> : Texture object mode <file> is list of images -magfilter -magf <flt> : NEAREST, LINEAR - default NEAREST -minfilter -minf <flt> : NEAREST, LINEAR, NEAREST_MIPMAP_NEAREST, LINEAR_MIPMAP_NEAREST, NEAREST_MIPMAP_LINEAR, LINEAR_MIPMAP_LINEAR - default NEAREST -texenv -te <env> : Texture enviroment, MODULATE, DECAL, BLEND - default DECAL -texcomp -tc <num> : Texture components where <num> is 1,2,3, or 4 - default 3 -blend -bl : Enable Blending -srcblendfunc -sbf : ZERO, ONE, DST_COLOR, ONE_MINUS_DST_COLOR, SRC_ALPHA, ONE_MINUS_SRC_ALPHA, DST_ALPHA, ONE_MINUS_DST_ALPHA, SRC_ALPHA_SATURATE - default SRC_ALPHA -dstblendfunc -dbf : ZERO, ONE, SRC_COLOR, ONE_MINUS_SRC_COLOR, SRC_ALPHA, ONE_MINUS_SRC_ALPHA, DST_ALPHA, ONE_MINUS_DST_ALPHA, - default ONE_MINUS_SRC_ALPHA -linewidth -lw <width> : Linewidth for wire/vector tests - default 1.0 -xwinsize -xws <side> : Size of test windows X dimension - default 700 -ywinsize -yws <side> : Size of test windows Y dimension - default 700 -numframes -nf <num> : Number of frames to be rendered during measurement. Takes priority over -mp -numilights -nil <num> : Turns on <num> infinite lights - default 0 -numllights -nll <num> : Turns on <num> local lights - default 0 -colormaterial -cm <side> <mode> : <side> is FRONT, BACK, FRONT_AND_BACK - default FRONT <mode> is AMBIENT, DIFFUSE, EMISSION, SPECULAR, AMBIENT_AND_DIFFUSE - default AMBIENT_AND_DIFFUSE -backface -bf : Cull Backfacing primitives - default off -frontface -ff : Cull Frontfacing primitives - default off -singlebuffer -sb : Single buffer mode -fog -fg : Enable fog -linesmooth -ls : Enable line antialiasing -polysmooth -ps : Enable polygon antialiasing -facetnormal -fn : Use facet normals when lighting -linestipple -lp : Enable line stipple -polystipple -pp : Enable polygon stipple -batch -bt <num> : Batch <num> primitives together per glBegin/glEnd. Valid with POINT, VECTOR, TRIANGLE, and QUADS -polymodefront -pmf : POINT, LINE, or FILL - default FILL -polymodeback -pmb : POINT, LINE, or FILL - default FILL -flat -f : Set shademodel to FLAT - default GOURAUD -zbuffer -zb : Enable zbuffer for tests - default off -clip -c : Align object on 3D clip boundary -lighttwoside -l2s : Light both sides of model -localview -lv : Define local viewer for lit tests -normalize -nm : Enable normal normalization -minperiod -mp <num> : Set minimum testing period in seconds -mblur <num> : Use motion blur with num being amount of decay -aa_multi <x> <r> : Full scene antialiasing rendered x times at an offset of r. r should be tuned to the viewset -walkthru -wt : Walkthru mode -grab <script> : Screen capture -grabonly -go : Don't run benchmark, just do screen capture -swap <true/false> : Force byte swapping -notitle -nt : Disable title screen -startframe -sf <num> : Set starting frame of a walkthru -framepause -fp : Pause after each frame waiting for enter key -newview -nv : Enable new view frustum calculation -fov <angle> : Set field of view in degrees -spin x y z dx dy dz : Set rotation initial values and speeds (degrees) -clipplane : Enables a user clip plane -nomaterial : Disables material changes if present -threads -th <num> : Sets number of threads (no arg means 1 per processor)The -threads option is applicable only if SPECviewperf has been compiled with the C Preprocessor variable MP defined. SPECviewperf should only be compiled this way if the vendor supplied makefile has defined the MP flag (-DMP).
Here is an example of the command line from test #3 of the CDRS-03 viewset:
viewperf -mh mower-ts -dl -rm TMESH -vz -bl -zb -nil 1 -cp PRIMITIVE
-mp 10 -nf 10
This command line is providing the following directions:
Upon entering the command line, the system under test will run the benchmark for at least 10 seconds or 10 frames. This is to insure a valid test result. During the test, the SPECviewperf code renders the data set, mower-ts.msh, which is half of the shell of a lawnmower, rotating it about the center of the window with the above specified characteristics.
What follows is the output of SPECviewperf 6.1 running the first test from ProCDRS-01 on an SGI OCTANE/MXE:
Viewperf Version 6.1 Viewperf Arguments -mh newtruck-wire -dl -linesmooth -srcblendfunc SRC_ALPHA -dstblendfunc ONE_MINUS_SRC_ALPHA -f -xws 1260 -yws 980 -newview -mp 20 -grab grab1.scr Month 11 Day 6 Year 1998 Host pptemp5 Operating System IRIX64 Operating System Version 6.5 07271714 Host Vendor SGI Host Model IP30 Host CPU 250 MHz R10000 Host CPU Count 2 Host Memory Size (MB) 128 Host Primary Cache Size (KB) 32/32 (D/I) Host Secondary Cache Size (KB) 1024 (D+I) Window System X11R5 Driver Version N/A OpenGL Vendor SGI OpenGL Version 1.1 Irix 6.5 OpenGL Extensions GL_EXT_abgr GL_EXT_blend_color GL_EXT_blend_logic_op GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_convolution GL_EXT_copy_texture GL_EXT_histogram GL_EXT_packed_pixels GL_EXT_polygon_offset GL_EXT_subtexture GL_EXT_texture GL_EXT_texture3D GL_EXT_texture_object GL_EXT_vertex_array GL_SGI_color_matrix GL_SGI_color_table GL_SGI_texture_color_table GL_SGIS_texture_filter4 GL_SGIS_detail_texture GL_SGIS_texture_border_clamp GL_SGIS_texture_select GL_SGIS_texture_lod GL_SGIX_pixel_texture GL_SGIX_texture_multi_buffer OpenGL Renderer IMPACT/2/2/4 OpenGL Client Vendor SGI OpenGL Client Version 1.2 Irix 6.5 OpenGL Client Extensions GLX_EXT_import_context GLX_EXT_visual_info GLX_EXT_visual_rating GLX_SGI_make_current_read GLX_SGI_swap_control GLX_SGI_video_sync GLX_SGIS_multisample GLX_SGIX_fbconfig GLX_SGIX_pbuffer GLX_SGIX_swap_barrier GLX_SGIX_swap_group GLX_SGIX_video_resize GLX_SGIX_video_source GLU Version 1.2 Irix 6.5 GLU Extensions GLU_EXT_abgr GLU_EXT_nurbs_tessellator GLU_EXT_object_space_tess GLU_EXT_packed_pixels GLU_EXT_texture GLU_SGI_filter4_parameters Direct Rendering True Double Buffer True Stereo False RGBA True Color Index Size -1 Red Size 12 Green Size 12 Blue Size 12 Alpha Size 0 Accum Red Size 16 Accum Green Size 16 Accum Blue Size 16 Accum Alpha Size 16 Depth Size 0 Stencil Size 0 Auxiliary Buffer Count 0 Frame BufferLevel 0 Visual ID 0X38 Visual Class TrueColor Window Width (pixels) 1260 Window Height (pixels) 980 Screen Width (pixels) 1280 Screen Height (pixels) 1024 Display :0.0 OpenGL Server Vendor SGI OpenGL Server Version 1.2 Irix 6.5 OpenGL Server Extensions 1.2 Irix 6.5 GLX Server Version 1.1 GLX Server Extensions GLX_EXT_import_context GLX_EXT_visual_info GLX_EXT_visual_rating GLX_SGI_make_current_read GLX_SGI_swap_control GLX_SGI_video_sync GLX_SGIX_fbconfig GLX_SGIX_pbuffer GLX_SGIX_swap_group Screen Number 0 Shared Memory Connection False Visual Selection Criteria MINIMUM Number of Execution Threads 1 Geometry File newtruck-wire Input Mode -mh Minimum Test Period 20.000000 Number of Frames 0 Number of Primitives 18691 Number of Vertices per Frame 202481 Number of Vertices per Primitive 10.833075 Toggle Mode NONE Batching Count 0 Render Mode MIXED Color per COLOR_PER_FRAME Orthographic Projection FALSE Display List TRUE Vertex Array (OpenGL1.1) FALSE Interleaved Data FALSE Clip Geometry FALSE Walkthrough Mode FALSE Back Face Cull FALSE Front Face Cull FALSE Front Polygon Mode FILL Back Polygon Mode FILL Polygon Stipple Enable FALSE Polygon Antialiasing Enable FALSE Line Width 1.000000 Line Stipple Enable FALSE Line Antialiasing Enable TRUE Number of Infinite Lights 0 Number of Local Lights 0 Color Material Enable FALSE Color Material Face FRONT Color Material Mode AMBIENT_AND_DIFFUSE Facet Normals FALSE Two Sided Lighting Enable FALSE Local Viewer Enable FALSE Flat Shading TRUE Fog Enable FALSE Normal Normalization FALSE Texture Enable FALSE TexObj Enable FALSE Texture Generation Mode NO_TEXTURE_GENERATION Texture File NONE Texture Minification Filter NEAREST Texture Magnification Filter NEAREST Texture Environment Mode DECAL Texture Components 3 Depth Test Enable FALSE Blend Enable FALSE Source Blend Function SRC_ALPHA Destination Blend Function ONE_MINUS_SRC_ALPHA Dithering Enable TRUE Motion Blur Amount 0.000000 Full Scene Antialiasing Redraws 0 Full Scene Antialiasing Jitter Amount 0.000000
0.0900 sec (DL Build) -- -mh newtruck-wire -rm MIXED -mp 20 -cp FRAME -dl -f -ls -xws 1260 -yws 980 -nv Number of frames run: 406, Test period: 20.120001 (sec) 20.2 frames/sec -- -mh newtruck-wire -rm MIXED -mp 20 -cp FRAME -dl -f -ls -xws 1260 -yws 980 -nv =============================================================The output provides all the SPECviewperf state information, as well as the configuration parameters for the system under test. The display list build time was 0.09 seconds, the number of frames was 406, and the actual test time was 20.12 seconds.
SPECviewperf does a test to see how many frames need to be rendered to meet the minimum time period. In this case, the calibration calculates the number of frames necessary to meet the minimum, bumps up the number of frames if the minimum is not met, then reruns the test.
The last line is the number of frames per second, which is the fundamental metric for each test. This is the result that is printed in the summary and individual report pages within the GPC News SPECopc section. Also printed in the summary and individual report pages is a single composite number for each viewset. This composite number is derived using a weighted geometric mean methodology (essentially the product of the results of each test raised to the power of the test weightings).
SPECviewperf 6.1.2
The latest version of SPECviewperf includes a number of new features intended to allow it to more closely model the performance of OpenGL graphics applications. The new features include support for:
Rob Putney / Simon Green / Chris Watts