525.x264_r
SPEC CPU®2017 Benchmark Description

Benchmark Name

525.x264_r

Benchmark Author

Submitted to SPEC® by:

Additional Authors

Benchmark Program General Category

Video compression

Benchmark Description

x264 is a free software library and application for encoding video streams into the H.264/MPEG-4 AVC format. x264 is distributed and maintained by the VideoLan project

The 525.x264_r source code, which is part of the SPEC CPU® suite, was ported from the source obtain from the May 15, 2009 daily snapshot downloaded from VideoLan.

Input Description

525.x264_r uses the Blender Open Movie Project's "Big Buck Bunny", Copyright 2008, Blender Foundation / www.bigbuckbunny.org. Each workload uses a portion of the movie.

To save space on the SPEC CPU® media, the movie is first decoded to YUV format in a (non-timed) setup phase, using the decoder 'ldecod' from the H.264/AVC reference software implementation. (The H.264/AVC encoder was used in SPEC CPU®2006 benchmark 464.h264ref.)

The reference workload encodes back to MPEG-4. It is run two separate ways on different parts of the movie. First a two pass method uses the options:

Followed by a single pass method:

Output Description

525.x264_r creates individual frames from the movie as it is being encoded. The option --dumpyuv 200 writes every 200th frame to a .yuv file. Therefore,

The 2-pass method
creates output files

 frame_0.yuv
 frame_200.yuv
 frame_400.yuv
 frame_600.yuv
 frame_800.yuv
 frame_999.yuv 

The single-pass method
creates output files

 frame_500.yuv
 frame_700.yuv
 frame_900.yuv
 frame_1100.yuv
 frame_1249.yuv 

Frames 0 and 500 are simply dumps of the starting points, and are not validated. The others from the lists just above are validated using the SPEC utility imagevalidate, which calculates the Structural SIMilarity (SSIM) index over 8x8 pixel areas between the benchmark-produced image and a SPEC-provided expected image. SSIM indexes range from -1 (maximally different) to 1 (identical) for each of those regions.

You can see the validation commands if you go to a run directory and use specinvoke -n compare.cmd. The commands can be entered by hand if you wish. A brief (one line) report is sent to the .out file, and a longer report that you might wish to examine is sent to the .err. For example:

$ ./imagevalidate_r_base.aug23b -avg -dumpfile -threshold 0.5 \
  -maxthreshold 20 frame_999.yuv frame_999.org.tga \
  > imageValidate_frame_999.out 2> imageValidate_frame_999.err
$ cat imageValidate_frame_999.out
frame_999.yuv: AVG SSIM = 0.952274876
$ cat imageValidate_frame_999.err
frame_999.yuv: INFO:
  Number below threshold: 1  (out of 14400) 
                AVG SSIM: 0.952274876
                MIN SSIM: 0.485218018
       SSIM distribution: 
          0.800 to 1.000: 14146
          0.600 to 0.800: 249
          0.400 to 0.600: 5
          0.200 to 0.400: 0
          0.000 to 0.200: 0
          -0.200 to 0.000: 0
          -0.400 to -0.200: 0
          -0.600 to -0.400: 0
          -0.800 to -0.600: 0
          -1.000 to -0.800: 0
$ 

The option -dumpfile for imagevalidate can be used to output the generated file in TGA format, which can then be viewed by a suitable image viewer.

In all cases, the expected output from imagevalidate is:

 <filename>: AVG SSIM = 1.000000000

which is then checked by specdiff to verify that the overall average of SSIM indexes is at least 0.85.

Programming Language

C

Known portability issues

  1. C99: The benchmark uses some C99 features so you may need a compiler flag such as GCC's "-std=c99".

  2. alignment: On most platforms, the benchmark source code assumes that your compiler supports the non-standard extension __attribute__((aligned(n))).

  3. char: The original benchmark source code assumed that the "char" data type is signed, which is not true on all platforms. SPEC has attempted to edit the code to mark the affected declarations with the 'signed' keyword; however, it is possible that some instances may have been missed, in which case platforms where "char" is by default unsigned may need a compiler portability flag such as GCC's "-fsigned-char".

  4. GCC 10 multiple definitions: If you compile using GCC 10, you will likely encounter errors similar to:

    ldecod_src/decoder_test.o (symbol from plugin): In function `cfgparams':
    (.text+0x0): multiple definition of `cfgparams'
    collect2: error: ld returned 1 exit status
    specmake: *** [...benchspec/Makefile.defaults:337: ldecod_r] Error 1 

    The solution is to include the flag -fcommon, as discussed at https://gcc.gnu.org/gcc-10/porting_to.html.

    Note that -fcommon would not qualify as a PORTABILITY flag, per www.spec.org/cpu2017/Docs/runrules.html#portability. Therefore it must be used in accordance with the same-for-all rule www.spec.org/cpu2017/Docs/runrules.html#BaseFlags. The Example GCC config files as updated for SPEC CPU 2017 v1.1.5 demonstrate how to set it correctly.

Sources and Licensing

As noted above, the encoder is from the VideoLan x264 project (GPL); Big Buck Bunny is from the Blender Foundation (Creative Commons); and the (non-timed) decoder is from the H.264/AVC reference implementation.

Please see details in the document SPEC CPU®2017 Licenses.

References

Last updated: $Date: 2020-09-23 10:06:01 -0400 (Wed, 23 Sep 2020) $

Copyright © 2017-2020 Standard Performance Evaluation Corporation (SPEC®)