SPEC CPU2017 Benchmark Description

Benchmark Name


Benchmark Author

Submitted to SPEC by:

Additional Authors

Benchmark Program General Category

Video compression

Benchmark Description

x264 is a free software library and application for encoding video streams into the H.264/MPEG-4 AVC format. x264 is distributed and maintained by the VideoLan project

The 625.x264_s source code, which is part of the SPEC CPU suite, was ported from the source obtain from the May 15, 2009 daily snapshot downloaded from VideoLan.

Input Description

625.x264_s uses the Blender Open Movie Project's "Big Buck Bunny", Copyright 2008, Blender Foundation / www.bigbuckbunny.org. Each workload uses a portion of the movie.

To save space on the SPEC CPU media, the movie is first decoded to YUV format in a (non-timed) setup phase, using the decoder 'ldecod' from the H.264/AVC reference software implementation. (The H.264/AVC encoder was used in SPEC CPU2006 benchmark 464.h264ref.)

The reference workload encodes back to MPEG-4. It is run two separate ways on different parts of the movie. First a two pass method uses the options:

Followed by a single pass method:

Output Description

625.x264_s creates individual frames from the movie as it is being encoded. The option --dumpyuv 200 writes every 200th frame to a .yuv file. Therefore,

The 2-pass method
creates output files


The single-pass method
creates output files


Frames 0 and 500 are simply dumps of the starting points, and are not validated. The others from the lists just above are validated using the SPEC utility imagevalidate, which calculates the Structural SIMilarity (SSIM) index over 8x8 pixel areas between the benchmark-produced image and a SPEC-provided expected image. SSIM indexes range from -1 (maximally different) to 1 (identical) for each of those regions.

You can see the validation commands if you go to a run directory and use specinvoke -n compare.cmd. The commands can be entered by hand if you wish. A brief (one line) report is sent to the .out file, and a longer report that you might wish to examine is sent to the .err. For example:

$ ./imagevalidate_r_base.aug23b -avg -dumpfile -threshold 0.5 \
  -maxthreshold 20 frame_999.yuv frame_999.org.tga \
  > imageValidate_frame_999.out 2> imageValidate_frame_999.err
$ cat imageValidate_frame_999.out
frame_999.yuv: AVG SSIM = 0.952274876
$ cat imageValidate_frame_999.err
frame_999.yuv: INFO:
  Number below threshold: 1  (out of 14400) 
                AVG SSIM: 0.952274876
                MIN SSIM: 0.485218018
       SSIM distribution: 
          0.800 to 1.000: 14146
          0.600 to 0.800: 249
          0.400 to 0.600: 5
          0.200 to 0.400: 0
          0.000 to 0.200: 0
          -0.200 to 0.000: 0
          -0.400 to -0.200: 0
          -0.600 to -0.400: 0
          -0.800 to -0.600: 0
          -1.000 to -0.800: 0

The option -dumpfile for imagevalidate can be used to output the generated file in TGA format, which can then be viewed by a suitable image viewer.

In all cases, the expected output from imagevalidate is:

 <filename>: AVG SSIM = 1.000000000

which is then checked by specdiff to verify that the overall average of SSIM indexes is at least 0.85.

Programming Language


Known portability issues

Sources and Licensing

As noted above, the encoder is from the VideoLan x264 project (GPL); Big Buck Bunny is from the Blender Foundation (Creative Commons); and the (non-timed) decoder is from the H.264/AVC reference implementation.

Please see details in the document SPEC CPU2017 Licenses.


Last updated: $Date: 2017-05-01 13:34:29 -0400 (Mon, 01 May 2017) $