SPECaccel® 2023 Config Files

For the benchmark specifier, the precedence is:

  highest   named benchmark(s)
            suite name
  lowest    default

The flagsurl line picks up definitions that are provided with SPECaccel 2023.

404.lbm is an accel benchmark, but it does not use the setting on lines 12-14. Instead, it uses the higher precedence lines 10-11 (named benchmark).

456.spF is an accel benchmark. The highest precedence section for it is on lines 15-17.

450.md is an accel benchmark. It gets the OPTIMIZE setting from lines 12-14.

$ cat -n precedence_example1.cfg
     1  flagsurl             = $[top]/config/flags/gcc.xml
     2  iterations           = 1
     3  output_format        = text
     4  output_root          = /tmp/ptest
     5  runlist              = 404.lbm,450.md,456.spF
     6  size                 = test
     7  default:
     8     CC_VERSION_OPTION = --version
     9     CC                = gcc
    10  404.lbm:
    11     OPTIMIZE          = -O3
    12  450.md:
    13     pmodel            = ACC
    14     OPTIMIZE          = -O2 -fopenacc
    15  456.spF:
    16     pmodel            = TGT
    17     OPTIMIZE          = -O1 -fopenmp

For the tuning specifier, base or peak has higher precedence than default.

The first few lines of the config file use similar features as the previous example.

The tuning from line 12 is used for base, and line 14 for peak.

$ cat -n precedence_example2.cfg 
     1  flagsurl             = $[top]/config/flags/gcc.xml
     2  iterations           = 1
     3  output_format        = text
     4  output_root          = /tmp/ptest2
     5  runlist              = 404.lbm
     6  size                 = test
     7  default=default:
     8     CC_VERSION_OPTION = --version
     9     CC                = gcc
    10     OPTIMIZE          = -fopenacc
    10  default=base:
    11     OPTIMIZE          += -O1
    12  default=peak:
    13     OPTIMIZE          += -O3
$ cat precedence_example2.sh 
runaccel --config=precedence_example2 --tune=base | grep txt
runaccel --config=precedence_example2 --tune=peak | grep txt
cd /tmp/ptest2/result
grep 'O[0-9]' *txt
$ ./precedence_example2.sh 
    format: Text -> /tmp/ptest2/result/accel2023.001.acc.test.txt
    format: Text -> /tmp/ptest2/result/accel2023.002.acc.test.txt
accel2023.001.acc.test.txt: 404.lbm: -fopenacc -O1
accel2023.002.acc.test.txt: 404.lbm: -fopenacc -O3
$

For the label specifier, any named label has higher precedence than the default.

This config file is simpler than the previous examples, because we don't even bother to run it; instead, --fake is used.

The runaccel command uses --label=OhTwo. Therefore, the default setting for OPTIMIZE on line 6 is over-ridden by the settings on lines 7-8.

$ cat -n precedence_example3.cfg 
 1  runlist              = 404.lbm
 2  size                 = test
 3  accel=base=default:
 4     CC_VERSION_OPTION = --version
 5     CC                = gcc
 6     OPTIMIZE          = -O0 -fopenacc
 7     pmodel            = ACC
 8  accel=base=OhTwo:
 9     OPTIMIZE          = -O2 -fopenacc
$ cat precedence_example3.sh 
runaccel --config=precedence_example3 --fake --label=OhTwo | grep lbm.c
$ ./precedence_example3.sh 
gcc -c -o lbm.o -DSPEC  -DSPEC_OPENACC -DNDEBUG -O2 -fopenacc lbm.c
$  

Combine sections that apply to a benchmark, if there is no conflict among them.

Note that line 1 sets the label, and line 3 sets the tuning.

All sections -- including lines 6, 8, 10, and 12 -- contribute to the compile command, which has been wrapped for readability.

$ cat -n precedence_example4.cfg 
  1  label                = wall
  2  runlist              = 404.lbm
  3  tune                 = peak
  4  default:
  5     CC_VERSION_OPTION = --version
  6     CC                = gcc
  7  accel:
  8     OPTIMIZE          = -O1 -fopenmp
  9  default=peak:
 10     COPTIMIZE         = -ftree-vectorize
 11  default=default=wall:
 12     EXTRA_COPTIMIZE   = -Wall
$ cat precedence_example4.sh  
runaccel --config=precedence_example4 --fake | grep lbm.c
$ ./precedence_example4.sh 
mpicc -c -o lbm.o -DSPEC -DSPEC_OPENMP -DNDEBUG -O1 -fopenmp -ftree-vectorize -Wall  lbm.c
$  

If sections conflict with each other, the order of precedence is:

highest     benchmark
            tuning
lowest      label 

$ diff --side-by-side precedence_example6a.cfg precedence_example6b.cfg
iterations           = 1                            iterations           = 1
output_format        = text                         output_format        = text
output_root          = /tmp/ptest                   output_root          = /tmp/ptest
runlist              = 404.lbm                      runlist              = 404.lbm
size                 = test                         size                 = test
default:                                            default:
   CC_VERSION_OPTION = --version                       CC_VERSION_OPTION = --version
   CC                = gcc                             CC                = gcc
404.lbm:                                          <
   OPTIMIZE          = -O3                        <
accel:                                            <
   OPTIMIZE          = -O1                        <
default:                                            default:
   OPTIMIZE          = -O0                             OPTIMIZE          = -O0
                                                  > accel:
                                                  >    OPTIMIZE          = -O1
                                                  > 404.lbm:
                                                  >    OPTIMIZE          = -O3
$ cat precedence_example6.sh
runaccel --fake --config=precedence_example6a | grep lbm.c
runaccel --fake --config=precedence_example6b | grep lbm.c
$ chmod +x precedence_example6.sh
$ ./precedence_example6.sh 
mpicc -c -o lbm.o -DSPEC -DNDEBUG -O3  lbm.c
mpicc -c -o lbm.o -DSPEC -DNDEBUG -O3  lbm.c
$  

 7  accel:
12  accel=default:
17  accel=default=default:

$ cat -n precedence_example7.cfg
     1  label                = wall
     2  runlist              = 404.lbm
     3  tune                 = base
     4  default:
     5       CC_VERSION_OPTION = --version
     6       CC                = gcc
     7  accel:
     8       OPTIMIZE          = -O1 -fopenmp
     9       EXTRA_CFLAGS      = -finline-functions
    10  accel=default:
    11       pmodel            = ACC
    12       OPTIMIZE          = -O2 -fopenacc
    13       COPTIMIZE         = -ftree-vectorize
    14  accel=peak:
    15       OPTIMIZE          = -O4 -fopenacc
    16  accel=default=default:
    17       pmodel            = LOP
    18       OPTIMIZE          = -O3 -fopenmp
    19       EXTRA_COPTIMIZE   = -Wall
$ cat precedence_example7.sh 
runaccel --config=precedence_example7 --fake | grep lbm.c 
$ ./precedence_example7.sh 
gcc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  -O3 -fopenmp -ftree-vectorize -finline-functions -DSPEC_OPENMP -DSPEC_OPENMP_LOOP   -Wall     lbm.c
$  

I.F. Variables and Variable Substitution

I.F.1. Defining variables

You can create your own runaccel variables using a line of the form

name = value

The name may contain only letters, digits, and underscores (a hyphen is NOT allowed).
Start with a letter.
You may indent your definitions if you wish (see: whitespace)

Exception: preprocessor macros are different on all of the above. Hyphens are allowed, use

%define name value

and the % must appear in column 1. You can indent after the % if you wish.

Conventions: Although not required, certain conventions are usually followed:

1. lower case is typically used for macros. When combining words, CamelCase may be useful.
2. lower case is also typically used for runaccel variables, combining words with underscores
3. CAPITALS are typically used when creating make variables
4. CAPITALS are also typically used for shell or environment variables.

Examples of the above (in the same order)

%ifndef processorNumaControl
%   define processorNumaControl firstTouch # macro
%endif
default:
    my_submit_cmd       = numactl -C $BIND # runaccel variable
    MYTUNE              = -O2 --math=SIMD  # make variable
    ENV_LD_LIBRARY_PATH = /opt/lib         # environment variable

The remainder of this section I.F is about runaccel variables -- the $[square] and ${curly} rows from the table at the top.

I.F.2. $[square] substitution at startup

Immediately after preprocessing, variables that are delimited by $[square brackets] are substituted.
Any value set in the config file can be substituted, provided that is visible in the current scope.
You can access the value of additional variables that you may have created.
See the list of useful variables below.
Perhaps the most useful is $[top] for the top of the SPECaccel 2023 tree, often found in contexts such as:

    flagsurl01          = $[top]/config/compiler.xml
    flagsurl02          = $[top]/config/platform.xml
    EXTRA_LIBS          = $[top]/mylibs
    preENV_LIBRARY_PATH = $[top]/lib64:$[top]/lib

Traps for the unwary: In some cases it may be obvious to the human which value to use, but the tools aren't as smart as you.

1. Timing: You cannot substitute variables that do not exist, or whose values aren't known when the config file is first read. For example, the label is not sorted out until later. Values that pertain to individual benchmarks (such as the current iteration number or the name of the benchmark being run) are not set until much later, when benchmarks are actually being run.
2. Command line: If a value is mentioned on both the runaccel command line and in the config file, the substitution might not do what you expect.

3. output_root:
   You cannot set an output_root that depends on a runaccel variable.
   You can set one that uses a macro:

   output_root=$[top]/my/path      # wrong
   output_root=${top}/my/path      # wrong
   output_root=%{ENV_SPEC}/my/path # right

Square substitution is done early. That comes in handy if you need a variable right away, for example, in order to use it with preENV.

$ cat EarlySub.cfg 
SW_DIR                 = /opt/path/to/compilers
preENV                 = 1
preENV_LD_LIBRARY_PATH = $[SW_DIR]/lib

$ cat EarlySub.sh
runaccel --config=EarlySub --fake 404.lbm | grep '^LD' | uniq
$ ./EarlySub.sh 
LD_LIBRARY_PATH=/opt/path/to/compilers/lib
$  

The example below uses variables defined in several named sections. The sections are delimited by section markers default: (line 8), default=base: (line 11), and default=peak: (line 15).

$ cat -n square.cfg
     1  expand_notes         = 1
     2  iterations           = 1
     3  output_format        = text
     4  output_root          = /tmp/square
     5  runlist              = 404.lbm
     6  size                 = test
     7  tune                 = base,peak
     8  default:
     9     CC                = gcc
    10     CC_VERSION_OPTION = --version
    11  default=base:
    12     the_system        = STAR
    13     OPTIMIZE          = -O1 -fopenacc
    14     notes_base_100    = base tuning uses '$[CC]' '$[OPTIMIZE]' on system '$[the_system]'
    15  default=peak:       
    16     OPTIMIZE          = -O2 -fopenacc
    17     notes_peak_100    = peak tuning uses '$[CC]' '$[OPTIMIZE]' on system '$[the_system]'
    18  
$ cat square.sh 
runaccel --config=square | grep txt
grep tuning /tmp/square/result/accel2023.001.omp.test.txt
$ ./square.sh 
    format: Text -> /tmp/square/result/accel2023.001.small.test.txt
     base tuning uses 'gcc' '-O1 -fopenacc' on system 'STAR'
     peak tuning uses 'gcc' '-O2 -fopenacc' on system ''
$  

Note that line 14 finds all three variables that it is looking for, but line 17 does not. If it is not clear why this happens, please see the descriptions of named sections and precedence above.

I.F.3. Useful $[square] variables

Useful $[variables] include:

You can access the initial value of most options that you can enter into a config file, including:

I.F.4. ${curly} substitution (interpolation) during a run
+ $unbracketed substitution

runaccel uses perl interpolation.
Only scalars (not: perl arrays and hashes) can be interpolated.

Example: on the notes100 line, you could optionally add say either ${lognum} or $lognum, but don't try taking the curly brackets away from ${size}.

$ cat just.cfg 
expand_notes      = 1
notes100          = Just ${size}ing, in run $lognum 
output_root       = /tmp/just
runlist           = 404.lbm
size              = test
CC                = gcc
CC_VERSION_OPTION = --version
$ 
$ cat just.sh
runaccel -c just | grep txt
grep Just /tmp/just/result/accel2023.001.acc.test.txt
$   
$ ./just.sh 
    format: Text -> /tmp/just/result/accel2023.001.acc.test.txt
     Just testing, in run 001
$ 

Traps for the unwary

1. Try not to confuse ${curly} interpolation vs. %{curly} (macros)

2. Timing: Some variables are only defined at certain times, and a line that uses it might be interpolated at a different time. Therefore interpolation won't always do what you might wish. In particular, notes are not expanded in the context of a particular benchmark run, and therefore variables such as ${tune} are not useful within them.

3. output_root:
   You cannot set an output_root that depends on a runaccel variable.
   You can set one that uses a macro:

   output_root=$[top]/my/path      # wrong
   output_root=${top}/my/path      # wrong
   output_root=%{ENV_SPEC}/my/path # right

I.F.5. Useful ${curly} variables

These variables may be of interest:

For a complete list of the available variables relative to the current config file, set

expand_notes = 1
verbose      = 35 # or higher

Then, do a run that causes a command substitution to happen.
In the log, you will find many lines of the form:

 - Variables available for interpolation that have changed since the last list:
    (From config) $runmode = "speed"
    (From config) $size = "test"
 - Variables available for interpolation that have changed since the last list:
    (From config) $size = "ref"

I.F.6. Unsetting a variable with "%undef%"

It is sometimes useful to undo the setting of a variable that would otherwise be included from another section. This can be accomplished using the special value %undef%. In the example, line 14 undefines COPTIMIZE when compiling peak:

 $ cat -n dave.cfg 
 1  flagsurl             = $[top]/config/flags/gcc.xml
 2  iterations           = 1
 3  output_format        = text
 4  output_root          = /tmp/undef
 5  runlist              = 404.lbm
 6  size                 = test
 7  tune                 = base,peak
 8  default:
 9     CC_VERSION_OPTION = --version
10     CC                = gcc
11     OPTIMIZE          = -O2 -fopenacc
12     COPTIMIZE         = -fno-tree-pre
13  404.lbm=peak:
14     COPTIMIZE         = %undef%
15  
$ runaccel --config=dave | grep txt
    format: Text -> /tmp/undef/result/accel2023.001.small.test.txt
$ cd /tmp/undef/benchspec/ACCEL/404.lbm/build
$ grep OPTIMIZE build_base_none.0000/Makefile.spec 
COPTIMIZE        = -fno-tree-pre
OPTIMIZE         = -O2 -fopenacc
$ grep OPTIMIZE build_peak_none.0000/Makefile.spec 
COPTIMIZE        = 
OPTIMIZE         = -O2 -fopenacc
$ 

Notice that in the build directory, COPTIMIZE is present for base and absent for peak.

I.F.7. Debug tips for runaccel variables

When debugging a config file that uses runaccel variables, try:

iterations       = 1
minimize_rundirs = 0
reportable       = 0
runlist          = (one or two benchmarks)
size             = test 
verbose          = 40

Using --fake will probably be informative. Look inside the log for the (case-sensitive) word 'From'.

II. Config file options for runaccel

This section documents options that control the operation of runaccel itself.

II.A. Precedence: config file vs. runaccel command line

In the list that follows, some items are linked to the document Using SPECaccel 2023 - the 'runaccel' Command because they can be specified either in a config file, or on the runaccel command line.

New with SPECaccel 2023, If an option is specified in both places, the command line wins.

II.B. Options

In the table that follows, if an option is documented as accepting the values "no" and "yes", these may also be specified as "false" and "true", or as "0" and "1".

The "Use In" column indicates where the option can be used:

submit = send_job job="${command}" rank_id=$rank

         send_job job="exe >out" rank_id=n

         send_job job="exe" rank_id=n >out

 size        = test
 tune        = peak
 label       = srini
+env_vars    = 1
 #
 404.lbm=peak:
+   ENV_LD_LIBRARY_PATH = /tmp/compiler/lib64:%{ENV_LD_LIBRARY_PATH}
    CC                  = alt_gcc
    CC_VERSION_OPTION   = -v
    OPTIMIZE            = -O1 -fopenacc
 

404.lbm=default:
#> I am posting this config file for use by others in the
#> company, but am forcing it to fail here because
#> I want to force users to review this section.
#>
#> Once you find your way here, you should test whether
#> bug report 234567 has been fixed, by using the first
#> line below.  If it has not been fixed, then use the
#> second.  In either case, you'll need to remove the
#> fail_build.
#>
#>   - Pney Guvaxre
#>     Boomtime, the 66th day of Confusion in the YOLD 3172

OPTIMIZE = -Osuperduper
# OPTIMIZE = -Omiddling
fail_build = yes

     flagsurl1 = mycompiler.xml
     flagsurl2 = myplatform.xml 

http_proxy = http://webcache.tom.spokewrenchdad.com:8080 

label=jun12.old.CC 
label=jun12.new.CC 
label=jun14-flagday 
label=jun15-robert.wants.yet.another.test

runaccel --label=sandra [...]
ERROR: The label 'sandra' defines no settings in the config file!

output_format        = text
runlist              = 404.lbm
default:
   CC_VERSION_OPTION = -v
   CC                = gcc
default=base=OhZero:
   OPTIMIZE          = -O0
default=base=OhThree:
   OPTIMIZE          = -O3 

$ find . -type d -name '*OhZ*'
./benchspec/ACCEL/404.lbm/build/build_base_OhZero.0000
./benchspec/ACCEL/404.lbm/run/run_base_test_OhZero.0000
./benchspec/ACCEL/404.lbm/run/run_base_test_OhZero.0001
./benchspec/ACCEL/404.lbm/run/run_base_test_OhZero.0002
$ cd benchspec/ACCEL/404.lbm/build
$ grep OPTIMIZE */Makefile.spec
build_base_OhThree.0000/Makefile.spec:OPTIMIZE   = -O3
build_base_OhZero.0000/Makefile.spec:OPTIMIZE   = -O0
$ 

   mailto=fast.guy@welovebenchmarks.org
   output_format=text,mail

   mailto=fast.guy@welovebenchmarks.org
   output_format=text,mail
   mail_reports=text

runaccel --output_root=/local/home/junjie --config=mine 404.lbm 

default:
accel:

preENV_LD_LIBRARY_PATH = %{ENV_SPEC}/libraries:%{ENV_LD_LIBRARY_PATH}

$ cd $SPEC   
$ specxz -dc nnn.benchmark.FixMumble.tar.xz | spectar -xvf -
$ cat README.nnn.benchmark.src.alt.FixMumble.txt

$ cat testme.cfg
action               = buildsetup
runlist              = nnn.benchmark
tune                 = base
default:
   CC                = gcc
   CC_VERSION_OPTION = --version
nnn.benchmark=base=without:
   OPTIMIZE          = -O2
nnn.benchmark=base=with:
   OPTIMIZE          = -O2
   srcalt            = FixMumble

runaccel --label=without --config=testme
runaccel --label=with    --config=testme

III. Config file options for specmake

For SPECaccel you do not write Makefiles. Instead, you set Make variables in the config file, which are sent to a SPEC-supplied copy of GNU Make, known as specmake. Variables with a dollar sign and parentheses, aka "round brackets", are substituted by specmake. For example:

COMPILER_DIR = /usr/local/bin/
CC           = $(COMPILER_DIR)cc
CXX          = $(COMPILER_DIR)c++
FC           = $(COMPILER_DIR)f90

See below for more information on syntax of variables that you create and reference.

III.A. Commonly used Make variables

• Basic specmake variables are described in this section.
• Many more are described in the Make Variables document.
• The specmake utility can be run directly from the command line, as described in Utilities.

The following Make variables are frequently useful. When selecting where to put a flag, please bear in mind that the run rules require that portability flags must use PORTABILITY variables.

III.B. Parallel Model Selection

SPECaccel contains one suite for OpenACC and OpenMP. Previously, accel 1.4 contained three separate suites, one for each model. To select OpenACC choose "ACC". For OpenMP, you may select between three different methods of the directives, "loop", "distribute/parallel", or "distribute/parallel" with "simd" applied to inner loops.

The choice of directives will depend on the target device and compiler support. Not all compiler support all forms, and the applied parallelism may differ depending on the target. For example, using "loop" may be better when targeting offload devices such as GPUs, but "simd" better when targeting multi-core CPUs. The different forms are provide to allow users the best choice for their target and mitigate bias when favoring one form over another.

Note the same model must be used for all benchmarks in base. For peak, you may select the model on a per benchmark basis.

To enable, use the appropriate 'pmodel' setting. The tools then set the appropriate preprocessor define flag which makes the directives visible in the source. Note that not all compilers support all models and the compiler flag that enables the model must be set in your OPTIMIZE variable in the config file. Please consult your compiler's documentation for the supported models and options.

Note that using the define flags directly in you OPTIMIZE or PORTABILITY flags will cause the tools to error. This is to help reduce the possibiltiy of setting the pmodel to one model but then accidently enabling a different model.

The config file fragment below demonstrates how to conditionally include a parallel model selected via runaccel --define option:

% cat -n config/pmodel.cfg
     1  # If model is not set use OpenACC
     2  %ifndef %{model}
     3  %   define model acc
     4  %endif
     5
     6  # Compiler flags applied to all models
     7  default:
     8    CC             = nvc
     9    OPTIMIZE       = -O3
    10    CXXPORTABILITY = --c++14
    11    submit = $command
    12
    13  # OpenACC flags
    14  %if %{model} eq 'acc'
    15      pmodel = ACC
    16      OPTIMIZE += -acc
    17  %endif
    18
    19  # OpenMP Loop flags
    20  %if %{model} eq 'loop'
    21      pmodel = LOP
    22      OPTIMIZE += -mp=gpu
    23  %endif
    24
    25  # OpenMP Distribute flags
    26  %if %{model} eq 'dist'
    27      pmodel=TGT
    28      OPTIMIZE += -mp=gpu
    29  %endif
    30
    31  # OpenMP Distribute w/ Inner SIMD flags
    32  %if %{model} eq 'simd'
    33      pmodel=SMD
    34      OPTIMIZE += -mp=gpu
    35  %endif
    36
    37  # This will error since the model define flags
    38  # should only be set by the tools
    39  404.lbm=peak:
    40      OPTIMIZE = -mp -fast -DSPEC_OPENMP_TARGET
    41
% runaccel -c pmodel --fake 404.lbm | grep lbm.c
nvc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  -O3 -acc           -DSPEC_OPENACC       lbm.c
% runaccel -c pmodel --fake --define model=acc 404.lbm | grep lbm.c
nvc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  -O3 -acc           -DSPEC_OPENACC       lbm.c
% runaccel -c pmodel --fake --define model=loop 404.lbm | grep lbm.c
nvc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  -O3 -mp=gpu           -DSPEC_OPENMP -DSPEC_OPENMP_LOOP        lbm.c
% runaccel -c pmodel --fake --define model=dist 404.lbm | grep lbm.c
nvc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  -O3 -mp=gpu           -DSPEC_OPENMP -DSPEC_OPENMP_TARGET        lbm.c
% runaccel -c pmodel --fake --define model=simd 404.lbm | grep lbm.c
nvc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  -O3 -mp=gpu           -DSPEC_OPENMP -DSPEC_OPENMP_INNER_SIMD        lbm.c
% runaccel -c pmodel --fake --define model=simd -T peak 404.lbm | grep ERROR
ERROR: pmodel define flag "-DSPEC_OPENMP_TARGET" may only be set by the tools.

III.C. Creating your own Make variables

Variables with a dollar sign and parentheses, aka "round brackets", are substituted by specmake.

Deprecated feature alert: Although it is also possible to pass information to specmake using curly brackets: ${COMPILER_DIR}, this is not recommended. Instead, you should consistently use curly brackets to address runaccel and round brackets to address specmake. It is possible that a future version of runaccel may insist on interpolating curly brackets itself, rather than allowing specmake to do so.

Example:

$ cat makevar.cfg 
action            = build
runlist           = 404.lbm
default:
   DEBUG_SYMBOLS  = --debug:symbols=expanded_info_level:42
   EXTRA_FFLAGS   = $(GEE)
small=base:
   GEE            = -g $(DEBUG_SYMBOLS)
small=peak:
   GEE            = -g 

$ cat makevar.sh
runaccel --config=makevar --fake --tune=base | grep COMP:
runaccel --config=makevar --fake --tune=peak | grep COMP:
$ ./makevar.sh 
COMP: "cc -c -o lbm.o -g --debug:symbols=expanded_info_level:42 <source>"
COMP: "cc -c -o lbm.o -g <source>"
$  

The config file above creates two variables options (DEBUG_SYMBOLS and GEE). Both are passed to specmake, which interprets them. The results are shown above using the runaccel --fake option.

For an extensive example of variable substitution handled by specmake, see the SPEC CPU®2000 example at www.spec.org/cpu2000/docs/example-advanced.cfg. Search that file for LIBS, and note the long comment which provides a walk-through of a complex substitution handled by specmake.

III.D. The operator "+=" is available (but should be used with caution)

The operator "+=" adds to specmake variables. It may be convenient; it also may cause hard-to-diagnose bugs. Example:

$ cat tmp.cfg  
action  = build
runlist = 404.lbm
tune    = peak
default:
   OPTIMIZE   = -O1
fprate=default:
   OPTIMIZE  += --unroll
default=peak:
   OPTIMIZE  += --inner_unroll
404.lbm=peak:
   OPTIMIZE  += --outer_unroll
default=default=breakfast:
   OPTIMIZE   = --jelly_roll

$ runaccel --fake --config=tmp | grep lbm.c
cc -c -o lbm.o -DSPEC -DSPEC_CPU -DNDEBUG -O1 --inner_unroll --unroll --outer_unroll lbm.c
$   

Note that the options accumulate.

Caution: although the += operator adds flexibility, it may introduce hard-to-predict behavior, depending on precedence of section specifiers, the order of your config file, and other features, such as include files. Instead of using '+=' try picking different make variables for different purposes. For an example of hard-to-predict behavior, what will happen if you add --label=breakfast to the above runaccel command? (Try it.)

Recommendations:
Avoid += to prevent surprises.
Keep it simple.
Pick different makevars for different purposes.
Create conventions for your config files and write them down, in config file comments.
If you must use += review its effects carefully (--fake is your friend).

III.E Using action=buildsetup to create a sandbox

When debugging a set of build options, it is often useful to create a "sandbox" - that is, a directory where you can play with the benchmark and its options. This example creates a build sandbox with action buildsetup.

$ cat sandbox.cfg 
action        = buildsetup
label         = fast
output_root   = /tmp/demo_buildsetup
runlist       = 404.lbm
tune          = peak
default=peak:
   OPTIMIZE   = -fast
$ cat sandbox.sh 
runaccel --config=sandbox | grep log
grep Makefile.spec /tmp/demo_buildsetup/result/accel2023.001.log
$ ./sandbox.sh 
The log for this run is in /tmp/demo_buildsetup/result/accel2023.001.log
Wrote to makefile '/tmp/demo_buildsetup/benchspec/ACCEL/404.lbm/build/build_peak_fast.0000/Makefile.spec':

The action causes a directory to be created
There, the Makefile can be examined, used in a dry run, or modified as part of a testing effort.

$ cd /tmp/demo_buildsetup/benchspec/ACCEL/404.lbm/build/build_peak_fast.0000/
$ grep OPTIMIZE Makefile.spec 
OPTIMIZE         = -fast
$ specmake --dry-run
cc -c -o lbm.o -DSPEC -DNDEBUG   -fast                  lbm.c
cc -c -o main.o -DSPEC -DNDEBUG   -fast                  main.c
cc -c -o specrand/specrand.o -DSPEC -DNDEBUG   -fast                  specrand/specrand.c
cc      -fast          lbm.o main.o specrand/specrand.o             -lm         -o lbm

See also the chapter on specmake in SPECaccel 2023 Utilities and the sandbox examples in Avoiding runaccel.

III.F. About Automatic Rebuilds

The SPECaccel tools try to keep config files and binaries synchronized with each other.(*)
Edits to a config file may cause binaries to be rebuilt, sometimes to the surprise(**) of testers.

Testing option sensitivity: The first thing that happens in a rebuild is to delete the old binary.
If that is a potential problem (perhaps it takes a long time to build), you can test whether a config file change will cause a rebuild:

1. Copy the config file to a test version (say, mycopy.cfg)
2. Edit the copy.
3. Remove any lines that change defaults for ignore_errors or verify_binaries
4. Use: runaccel --config=mycopy --nobuild --size=test --iterations=1 ...

The command:

• completes quickly (because --size=test and iterations=1)
• exits if binaries are not up to date, because --nobuild

No binaries are harmed.

IV. Config file options for the shell

Some options in your config file cause commands to be executed by your shell (/bin/sh).

IV.A. \$SHELLVAR variable substitution

Substitution by the shell uses backslash dollar sign.
The backslash protects the variables from interpretation by runaccel.

$ cat tmp.cfg
expand_notes         = 1
iterations           = 1
runlist              = 404.lbm
size                 = test
tune                 = base,peak
default:
   CC                = gcc
   CC_VERSION_OPTION = --version
default=base:
   submit    = echo home=$HOME, spec=$SPEC   > /tmp/chan; ${command}
default=peak:
   submit    = echo home=\$HOME, spec=\$SPEC > /tmp/nui;  ${command}

IV.B. Shell Options

Warning: SPECaccel config files can execute arbitrary shell commands.
Read a config file before using it.
Don't be root. Don't run as Administrator. Turn privileges off.

These options cause commands to be executed:

IV.C. Using submit

The config file feature submit allows you specify the command needed to launch your command as well as include and process to core bindings. This section provides examples to demonstrate how submit works with several other config file features. You might also want to search published results at www.spec.org/accel2023 for systems that are similar to your system.

IV.C.1. Basic usage of submit

You can use submit to specify scripts to perform binding before the benchmark command.

The example below runs user supplied "bind.sh" which binds CPU threads to cores, or which GPU to use.

$ cat submit.cfg 
runlist  = 404.lbm
submit   = sh $[top]/bind.sh ${command}

IV.C.2. Reporting of submit usage

If you use the submit feature, a notes section will automatically be created to indicate that you have done so.

                                 Submit Notes
                                 ------------
    The config file option 'submit' was used.

You can add notes to that section, or customize it as you wish, by creating lines with notes_submit_NNN. The phrase The config file option 'submit' was used must appear somewhere in your customized notes. You can vary the capitalization of the phrase, you can even break it across multiple lines; it just needs to be present. If it is not, it will automatically be added. If you used scripts in your submit command, the full text of the script should be listed in the submit notes.

V. Config file options for readers

Whether or not you send your result to SPEC, you should fully disclose how you achieved the result. If it requires the installation of the GoFastLinker, you should say so. By setting the appropriate fields in the config file, you can cause information about the GoFastLinker to appear in the reports that are intended for humans.

V.A. Descriptive fields

Here are the fields that you can set to describe your testbed to readers:

      test_sponsor = Genius Compilers
      tester       = Pawtuckaway State College
      hw_vendor    = TurboBlaster

V.B. Fields can vary by scope.

Fields can appear and disappear based upon scope. For example, you can define the compiler optimizations you use for all benchmarks while adding those for peak only in the peak section. You can also have notes which apply to all runs and add additional notes for another section.

Most (*) fields can be continued to another line by appending a numeral, for example sw_os1, sw_os2, sw_os3 if you need 3 lines to fully identify your operating system.

Here is an example that uses both of these features:

   default:
   OPTIMIZE    = -fast -xipo=2
   notes_000   =  System used mega-threading scheduler.

   default=peak:
   COPTIMIZE    = -xhyper -lsmash
   FOPTIMIZE    = -xblaster -lpile
   notes_001    = Smash library was used to allow potater tool optimization.
   notes_002    = Pile library allowed dynamic adjustment of spin delay, set with BLASTER=17.

In the above example, the information about the mega-threading scheduler will be printed for both base and peak runs. The information about the smash and pile libraries will only be printed for peak runs.

Note that the numerals above will be converted to a more formal style of three-digit numbers with leading zeros in the raw file; as discussed in utility.html, if you edit a rawfile you must use the more formal style of 000, 001, etc.

(*) The fields which cannot be continued are the ones that are expecting a simple integer, such as: hw_nchip, hw_ncores, hw_ncoresperchip, hw_nthreadspercore, license_num; and the ones that are expecting a date: hw_avail, and sw_avail.

V.C. Additional notes for the reader

In addition to the pre-defined fields, you can write as many notes as you wish. These notes are printed in the report, using a fixed-width font. For example, you can use notes to describe software or hardware information with more detail beyond the predefined fields:

   notes_os_001  = The operating system used service pack 2 plus patches 
   notes_os_002  = 31415, 92653, and 58979.  At installation time, the 
   notes_os_003  = optional "Numa Performance Package" was selected.

V.C.1. Notes sections

There are various notes sections. If there are no notes in a particular section, it is not output, so you don't need to worry about making sure you have something in each section.

The sections, in order of appearance, are as follows:

1. notes_comp_NNN -- Notes about compiler invocation.
2. notes_port_NNN -- Notes about portability options.
3. notes_base_NNN -- Notes about base optimization options.
4. notes_peak_NNN -- Notes about peak optimization options.
5. notes_submit_NNN -- Notes about use of the submit option.
6. notes_os_NNN -- Notes about operating system tuning and changes.
7. notes_plat_NNN -- Notes about platform tuning and changes.
8. notes_part_NNN -- Notes about component parts (for kit-built systems).
9. notes_NNN -- General notes.

Notes about the submit command are described above, with the description of the submit option.

V.C.2. Note numbering

Start your notes with the name of the notes section where you want the note to appear, and then add numbers to define the order of the lines. Within a section, notes are sorted by line number. The NNN above is not intended to indicate that you are restricted to 3 digits; you can use a smaller or larger number of digits as you wish, and you can skip around as you like: for example, ex-BASIC programmers might naturally use line numbers 100, 110, 120... But note that if you say notes_plat782348320742972403 you just might encounter the dreaded (and highly unusual) "out of memory" error, so don't do that.

You can optionally include an underscore just before the number, but beware: if you say both notes_plat_105 and notes_plat105, both are considered to be the same line. The last one mentioned will replace the first, and it will be the only one output.

V.C.3. Additional tags

For all sections you can add an optional additional tag of your choosing before the numbers. Notes will be organized within the tags.

The intent of the feature is that it may allow you to organize your system information in a manner that better suits your own categories for describing it.

For example:

$ cat notes_tags.cfg 
iterations      = 1
output_format   = text
output_root     = /tmp/notes_tags
runlist         = 404.lbm
size            = test
small=base:
   CC                       = mpicc
   CC_VERSION_OPTION        = -V
   notes_part_greeting_011  = ++ how
   notes_part_greeting_20   = ++ you?
   notes_part_greeting_012  = ++ are
   notes_part_aname_1       = ++ Veronica,
   notes_part_080           = ++ hi

$ cat notes_tags.sh  
runaccel --config=notes_tags --fakereport 404.lbm | grep txt
grep '++' /tmp/notes_tags/result/accel2023.001.test.txt
$ ./notes_tags.sh
    format: Text -> /tmp/notes_tags/result/accel2023.001.test.txt
     ++ hi
     ++ Veronica,
     ++ how
     ++ are
     ++ you?
$ 

V.C.4. Links in notes sections

You can mention URLs in your notes section, and html reports will correctly render them as hyperlinks. For example:

notes_plat_001 = Additional detail may be found at
notes_plat_002 = http://www.turboblaster.com/servers/big/green/

If you like, you can use descriptive text for the link by preceding it by the word LINK and adding the descriptive text in square brackets:

LINK url AS [descriptive text]

The brackets may be omitted if your descriptive text is a single word, without blanks.

For example:

notes_plat_001 = Additional detail may be found at 
notes_plat_002 = LINK http://www.turboblaster.com/servers/big/green/ AS [TurboBlaster Servers]

When the above appears in an html report, it is rendered as:

 Additional detail may be found at 
 TurboBlaster Servers
      

And in a text report, it appears as:

                                Platform Notes
                                --------------
     Additional detail may found at 
     TurboBlaster Servers (http://www.turboblaster.com/servers/big/green/)

Since the text report is not a context in which the reader can click on a link, it is spelled out instead. Note that because the text report spells the link out, the text line is wider than in HTML, PS, and PDF reports. When deciding where to break your notes lines, you'll have to pick whether to plan line widths for text (which may result in thin-looking lines elsewhere) or plan your line widths for HTML/PS/PDF (which may result in lines that fall of the right edge with text). The feature notes_wrap_columns won't help you here, since it is applied before the link is spelled out.

VI. The config file preprocessor

VI.A. Introduction

The SPECaccel tools include a configuration file macro preprocessor. This section introduces preprocessor examples, and then discusses a few basics of syntax.

VI.A.1. Preprocessor Example: Picking a Model

In this example, the user is testing several configurations, and wants to pick an appropriate pmodel and set the appropriate optimization and portability flags..

$ cat -n preprocessor_example1.cfg
     1  %if %{model} eq 'acc'
     2  pmodel=ACC
     3  OPTIMIZE     = -fast -acc
     4  %endif
     5
     6  %if %{model} eq 'lop'
     7  pmodel=LOP
     8  OPTIMIZE     = -fast -mp=gpu
     9
    10  403.stencil:
    11  PORTABILITY += -DSPEC_NO_NOTHING
    12
    13  %endif

Lines that start with percent (%) are preprocessor directives. The percent (%) character must be in column 1.

The config file preprocessor is called a macro processor because it allows you to define macros, which are brief abbreviations for longer constructs. In the example above, two macros are used: (%{model}, and %{usegpu}). Macro values are tested, and hunks of config file are included or excluded, using the conditionals %if (lines 2 and 5), and %elif (line 4). One of the conditionals includes an %else clause (line 7).

The preprocessor is automatically run whenever you use runaccel, and macros may be set on the runaccel command line. Or, you can run it separately, as configpp:

% specperl bin/harness/configpp -c preprocessor_example1.cfg --define model=acc | grep -e pmodel
pmodel=ACC
% specperl bin/harness/configpp -c preprocessor_example1.cfg --define model=lop | grep -e pmodel -e stencil -e PORT
pmodel=LOP
403.stencil:
PORTABILITY += -DSPEC_NO_NOTHING

Notice above that preprocessor variables can be set on the command line, using either --define or '-S'.

VI.A.2. Preprocessor Syntax Basics

Emphasis: Column 1. Always punch column 1.

%define foo
%    define bar
%		undef hello!

 %define foo           Space in the first column
	%define foo    Tab in the first column
#define foo            <--wrong!

One line per directive

VI.B. Defining macros

Macro names may only be composed of alphanumeric characters, underscores, and hyphens, and they ARE case-sensitive.

Macros can be defined in a config file or on the command line. Both ways are equivalent.

Macros set on the command-line are defined first; therefore, it is not possible to use the command line to override a macro definition that occurs in the config file itself. If you want the command line to "win", see the notes below about redefinition.

A macro that has not been defined will not be substituted.

A macro that has been defined, but not given a value,

• behaves as if it were 1 in a numeric context
• behaves as if it were true in a logical context (because for Perl, 1 is true)
• behaves as an empty string in a string context

See the example in the next section.

Having no value is not the same as having an empty string as the value. The following are NOT equivalent:

See the table about Truth and definition, below.

VI.B.1. In a config file

To define a macro in a config file, use the %define preprocessor directive. Note that no quoting is necessary when specifying the names of macros or their values.

# Define a simple macro
%define foo bar
# Now the macro called 'foo' has the value 'bar'

# It's not necessary for a macro to have a value to be useful
%define baz
# Now the macro called 'baz' is defined, but it has no value.

Above, the macro baz has been defined, but has not been assigned a value.
Nevertheless, if you use it in an expression, it will behave as if it has the value 1.

$ cat macroDefinedButNoValue.cfg 
%define baz
%ifdef %{baz}
%   info baz is defined. In text contexts, it looks like this: "%{baz}".
%endif
%if %{baz} 
%   info In a logical context, it behaves as if it were 1 (i.e. true)
%endif
%if %{baz} + 3  == 4
%   info In a numeric context, it behaves as if it were 1.
%endif
$ cat macroDefinedButNoValue.sh  
specperl bin/harness/configpp --config=macroDefinedButNoValue | grep INFO
$ ./macroDefinedButNoValue.sh 
  INFO: baz is defined. In text contexts, it looks like this: "".
  INFO: In a logical context, it behaves as if it were 1 (i.e. true)
  INFO: In a numeric context, it behaves as if it were 1.
$ 

VI.B.2. On the command line

Macros can also be defined on the command line, using the --define switch. You may find it convenient to abbreviate that switch, if you prefer. The following are entirely equivalent:

$ runaccel --define mymacro=something
$ runaccel -S mymacro=something

VI.B.3. Predefined macros endian, hostname... %{ENV_variable_name}

Some useful macros are predefined, as shown in the table below. New with SPECaccel 2023: The config file preprocessor can do environment variable substitution. Environment variables to be substituted use a percent sign and curly brackets, and the name is prefixed with the string "ENV_", as %{ENV_variable_name}.

Automating output_root: You can automatically bring an output_root into your config file using %{ENV_GO} and navigate around it, as shown in the example for the ogo utility.

VI.B.4. Example: Adjusting the Environment

Notice that %{ENV_variable_name} brings an environment variable into your config file; what if you want to push it back out to affect other things? In that case, you will also need the feature preenv, which causes runaccel to re-invoke itself with your requested variable in the environment.

For example, you could set up paths for your compiler using something like this:

$ cat -n test.cfg
     1  action       = build
     2  output_root  = /tmp/me
     3  rebuild      = 1
     4  runlist      = 404.lbm
     5  verbose      = 99
     6
     7  %define tentop /SW/compilers/GCC/Linux/x86_64/gcc-10.1.0
     8  %define ninetop  /SW/compilers/GCC/Linux/x86_64/gcc-9.2.0
     9
    10  %ifdef %{wantGcc9}
    11     preENV_PATH            = %{ninetop}/bin:%{ENV_PATH}
    12     preENV_LD_LIBRARY_PATH = %{ninetop}/lib64:%{ENV_LD_LIBRARY_PATH}
    13  % else
    14     preENV_PATH            = %{tentop}/bin:%{ENV_PATH}
    15     preENV_LD_LIBRARY_PATH = %{tentop}/lib64:%{ENV_LD_LIBRARY_PATH}
    16  %endif
    17
    18  default:
    19     CC                = gcc
    20     CC_VERSION_OPTION = --version
    21
$ 

The config file above builds a single benchmark, 404.lbm, using the local paths for GCC 9.2 or 10.1. Notice that in both cases, the desired path is added to the existing path: for example, on line 12, the right side accesses the existing setting via %{ENV_LD_LIBRARY_PATH} and the left side applies it to the build, via preENV_LD_LIBRARY_PATH. On line 5, the config file sets verbose=99, so that runaccel will print all possible detail and we will be able to confirm the adjusted variables.

VI.B.5. Redefinition

You will receive a warning if a previously defined macro is re-defined. In the example below, the first config file just sets a macro; the second one tests before acting.

$ cat preproc.symbol.just_define.cfg 
%define build_ncpus 20
makeflags   = -j%{build_ncpus}
$ 
$ cat preproc.symbol.test_first.cfg 
%ifndef %{build_ncpus}
%   define build_ncpus 20
%endif
makeflags   = -j%{build_ncpus}
$ 

A run script uses both config files with and without a definition on the runaccel command line:

$ cat preproc.symbol.sh 
JUST_DEFINE="configpp --config=preproc.symbol.just_define.cfg"
 TEST_FIRST="configpp --config=preproc.symbol.test_first.cfg"
    GREP_IT='grep -e WARNING -e makeflags'
echo original:
$JUST_DEFINE --define build_ncpus=8 | $GREP_IT
$JUST_DEFINE                        | $GREP_IT
echo
echo test1st: 
$TEST_FIRST  --define build_ncpus=8 | $GREP_IT
$TEST_FIRST                         | $GREP_IT
$

Result: testing first with ifndef avoids a warning and does a better job of respecting the user's wishes.

$ ./preproc.symbol.sh 
original:
WARNING: Redefinition of preprocessor macro 'build_ncpus' on line 1
makeflags   = -j20
makeflags   = -j20

test1st:
makeflags   = -j8
makeflags   = -j20
$  

VI.C. Un-doing macro definition

Sometimes you want to make the preprocessor forget about a macro that you taught it. This is easily accomplished.

Macros can be undefined in two ways:

1. On the command line, using the --undef switch, or
2. In the configuration file, using the '%undef' preprocessor directive.

   %define foo bar
   # Now the macro called 'foo' has the value 'bar'
   
   %undef foo
   # Now it doesn't
   

   Note that no quoting is necessary when specifying the names of macros.

Like macro definition, the undefinition requests can't affect macros set in the config file because those definitions effectively happen after the un-definition. For this reason, command-line undefinition is basically useless; it can only undo macros also set on the command line.
So why was such a useless ability added to the tools? The writer likes orthogonality.

VI.D. Using macros

By now you're probably over that initial euphoric rush that comes from wantonly defining and undefining macros, and you're looking for something more. This is it!

When you want to use the value of a macro, you refer to it by name. Unfortunately, the syntax for this is not as simple as you might hope. It's not too complicated, though; to have the preprocessor expand the macro 'foo', just write

  %{foo}

in the place where you'd like it to appear. Given the following config file snippet:

%define foo Hello_
%define bar baz
%define foobar Huh?
%define foobaz What?
%define Hello_baz Please don't do this

Here's a handy table to see the various ways you can reference these values:

Easy, right? The following is also possible:

Because macro values can only be one line long, it's not possible to use the preprocessor to macro-ize large chunks of your config file at once, as may be common practice for advanced users of CPP.

A macro that has not been defined will not be substituted. Note the mention of %{FOO} on the last line of this config file.

$ cat macroDef.cfg 
action          = build
output_format   = text
output_root     = /tmp/mDef
rebuild         = 1
runlist         = 404.lbm
size            = test
teeout          = 1
default:
   CC                 = gcc
   C_VERSION_OPTION   = --version 
   OPTIMIZE           = %{FOO}
$

The failure for the first test below is expected: the GNU C compiler does not know what to do with ${FOO}.

$ cat macroDef1.sh 
runaccel --config=macroDef | grep -e lbm.o -e ^Error -e uccess
$ ./macroDef1.sh 
rm -rf *.o  lbm.out
gcc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  %{FOO}           -DSPEC_OPENMP -DSPEC_OPENMP_LOOP        lbm.c
specmake: *** [/tmp/SPECACCEL/benchspec/Makefile.defaults:365: lbm.o] Error 1
Error with make 'specmake --output-sync build':
$ cat /tmp/mDef/benchspec/ACCEL/404.lbm/build/build*/make.out
gcc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC  %{FOO}           -DSPEC_OPENMP -DSPEC_OPENMP_LOOP        lbm.c
gcc: error: %{FOO}: No such file or directory
specmake: *** [/tmp/SPECACCEL/benchspec/Makefile.defaults:365: lbm.o] Error 1
$

If you want to substitute an empty string, then assign it one.

$ cat macroDef2.sh 
runaccel --config=macroDef --define FOO="" | grep -e lbm.o -e ^Error -e uccess
$ ./macroDef2.sh 
rm -rf *.o  lbm.out
gcc -c -o lbm.o -DSPEC -DNDEBUG -DSPEC             -DSPEC_OPENMP -DSPEC_OPENMP_LOOP        lbm.c
gcc                lbm.o main.o             -lm         -o lbm
$ 

VI.E. Conditionals

Defining, undefining, and expanding macros is quite an enjoyable activity in and of itself, and can even be useful on occasion. However, conditionals add an entirely new dimension to config file processing: the ability to include and exclude entire sections of text based on macros and their values.

VI.E.1. %ifdef .. %endif

The %ifdef conditional provides a way to determine whether or not a particular macro has been defined. If the named macro has been defined, the conditional is true, and the text to the matching %endif is included in the text of the config file as evaluated by runaccel. Note that the matching %endif may not necessarily be the next %endif; conditionals may be nested.

For example, given the following section of a config file:

%define foo
%ifdef %{foo}
This text will be included
%endif

%ifdef %{bar}
This text will not be included
%endif

The preprocessor would produce the following output:

This text will be included

Note especially the quoting used for the macro names in the conditional; the only time macro name quoting may be omitted is when defining or undefining it.

VI.E.2. %ifndef .. %endif

The %ifndef conditional is the converse of %ifdef; If the named macro has not been defined, the conditional is true, and the text to the matching %endif is included in the text of the config file as evaluated by runaccel. Note that the matching %endif may not necessarily be the next %endif; conditionals may be nested.

Given a slightly modified version of the example from earlier:

%define foo
%ifndef %{foo}
Now THIS text will not be included
%endif

%ifndef %{bar}
This text WILL be included
%endif

The preprocessor would produce the following output:

This text WILL be included

VI.E.3. %if .. %endif

Checking whether or not a macro is defined is quite useful, but it's just a subset of the more general conditional facility available. This general form is

%if expression
...
%endif

The expression is evaluated using a subset of the Perl interpreter, so the possibilities for testing values are fairly broad. For example,

%ifdef %{foo}
...
%endif

is exactly equivalent to

%if defined(%{foo})
...
%endif

Likewise,

%ifndef %{foo}
...
%endif

is exactly equivalent to

%if !defined(%{foo})
...
%endif

Using the general form, it's possible to string conditionals together:

%if defined(%{foo}) && !defined(%{bar}) || %{baz} == 0
...
%endif

To compare versus a string value, you must supply quotes:

%if %{foo} eq 'Hello, Dave.'
...
%endif

You may perform basic math on macro values:

%if %{foo} * 2004 > 3737
...
%endif

You can do basic regular expression matching, as shown in the example for %info.

More precisely, the Perl operations allowed are the :base_core and :base_math bundles, with the ability to dereference and modify variables disallowed. For more details, see the source code for config.pl (the eval_pp_conditional subroutine) and Perl's own Opcode documentation.

$ cat test_define.cfg
%ifdef %{a}
   OPTIMIZE = -a
%endif
%if defined(%{b})   
   LDOPTIMIZE = -b
%endif
%if %{c}
   COPTIMIZE = -c
%endif
$ specperl bin/harness/configpp --config=test_define -S a=1 -S b=1 -S c=1 | grep OPT  
   OPTIMIZE = -a
   LDOPTIMIZE = -b
   COPTIMIZE = -c
$ specperl bin/harness/configpp --config=test_define -S a=0 -S b=0 -S c=0 | grep OPT
   OPTIMIZE = -a
   LDOPTIMIZE = -b

VI.E.4. %else

It's possible to get by without the "else" part of the classic "if .. then .. else" trio, but it's not any fun. It works as you'd expect:

%define foo
%ifndef %{foo}
This text will not be included
%else
This text WILL be included (from the else clause)
%endif

The preprocessor would produce the following output:

This text WILL be included (from the else clause)

Only one %else per conditional is allowed.

VI.E.5. %elif

%elif is another convenience that's been added. For those not familiar with CPP, it's an "else if" construct. You may have as many of these as you'd like. Given:

%define foo Hello!

%if !defined(%{foo})
This text will not be included
%elif defined(%{bar})
This text won't be included either
%elif '%{foo}' eq 'Hello!'
This text WILL be included (from the second elif clause)
%else
Alas, the else here is left out as well.
%endif

The preprocessor would produce the following output:

This text WILL be included (from the second elif clause)

VI.F. Informational Directives

It's often helpful to be able to warn or exit on certain conditions. Perhaps there's a macro that must be set to a particular value, or maybe it's just very highly recommended.

VI.F.1. %warning

%warning does just what you'd expect; when the preprocessor encounters this directive, it prints the text following to stdout and the current log file, along with its location within the file being read, and continues on.

$ cat warning.cfg 
%if !defined(%{somewhat_important_macro})
%   warning You have not defined somewhat_important_macro!
%endif
$ 
$ cat warning.sh 
specperl bin/harness/configpp --config=warning | grep -C2 WARNING
$ 
$ ./warning.sh 

***
  WARNING: You have not defined somewhat_important_macro!
           (From line 2 of
           /accel2023/config/warning.cfg.)
$  

VI.F.2. %error

Like %warning, %error logs an error to stderr and the log file. Unlike %warning, though, it then stops the run.

Consider a slightly modified version of the previous example:

$ cat error.cfg 
%if !defined(%{REALLY_important_macro})
%   error You have not defined REALLY_important_macro!
%endif
$ 
$ cat error.sh 
specperl bin/harness/configpp --config=error > /tmp/out
echo runaccel exit code: $?
grep -C2 ERROR /tmp/out
$ ./error.sh 
runaccel exit code: 1

************************************************************************
  ERROR: You have not defined REALLY_important_macro!
         (From line 2 of
         /accel2023/config/error.cfg.)
$ 

Unlike a warning, the error will be close to the last thing output. As you can see from the output of echo $? above, runaccel exited with an error code 1.

VI.F.3. %info

The %info directive prints a message preceded by the word INFO:. For example,

$ cat -n info.cfg 
   1  %if !defined(%{chip}) || %{chip} !~ m/(sparc|x86)/
   2  %   error Please use --define chip=sparc or --define chip=x86
   3  %endif
   4  
   5  %if  %{chip} eq "sparc"
   6  %   define default_build_ncpus 64
   7  %elif %{chip} eq "x86"
   8  %   define default_build_ncpus 20
   9  %endif
  10  %ifndef   %{build_ncpus}
  11  %   define  build_ncpus   %{default_build_ncpus}
  12  %endif
  13  
  14  %info Preprocessor selections: 
  15  %info    .    build_ncpus      %{build_ncpus}
  16  %info    .    chip             %{chip}
  17  
  18  make = specmake --jobs=%{build_ncpus} --load-average=%{build_ncpus} 
$
$ specperl bin/harness/configpp -c info --define chip=sparc | grep -e make -e INFO
INFO: Preprocessor selections:
INFO: .    build_ncpus      64
INFO: .    chip             sparc
make = specmake --jobs=64 --load-average=64
$ 
 

Note in the example above:

• Line 1 demonstrates basic perl regular expression matching
• Lines 14-16 use %info
• After all preprocessing has been completed, line 18 tells the specmake utility that it can start up to 64 compile jobs, provided that the system load average is no higher than 64.

VI.F.4. %dumpmacros

The %dumpmacros prints the values of all macros currently defined at the point where it appears. Each macro value is preceded by the word 'DBG:'. For example,

$ cat -n dumpmacros.cfg 
     1	%if !defined(%{chip}) || %{chip} !~ m/(sparc|x86)/
     2	%   error Please use --define chip=sparc or --define chip=x86
     3	%endif
     4	
     5	%if  %{chip} eq "sparc"
     6	%   define default_build_ncpus 64
     7	%elif %{chip} eq "x86"
     8	%   define default_build_ncpus 20
     9	%endif
    10	%ifndef   %{build_ncpus}
    11	%   define  build_ncpus   %{default_build_ncpus}
    12	%endif
    13	
    14	%dumpmacros
$
$ specperl bin/harness/configpp -c dumpmacros --define chip=sparc | grep -e DBG
  DBG: build_ncpus: '%{default_build_ncpus}'
  DBG: chip: 'sparc'
  DBG: default_build_ncpus: '64'
  DBG: runaccel: 'configpp -c dumpmacros --define chip=sparc'
$
 

The %dumpmacros directive is new with SPECaccel 2023.

VII. Output files - and how they relate to your config file

This section describes how the location and contents of several kinds of output files are influenced by your config file.

VII.A. Help, I've got too many config files!

It was mentioned above that the HASH section of the config file is written automatically by the tools. Each time your config file is updated, a backup copy is made. Thus your config directory may soon come to look like this:

$ cd $SPEC/config
$ ls tune.cfg*
tune.cfg                        tune.cfg.2021-02-05T124831      tune.cfg.2021-02-05T125733
tune.cfg.2021-02-05T120242      tune.cfg.2021-02-05T125557      tune.cfg.2021-02-05T125738
tune.cfg.2021-02-05T122021      tune.cfg.2021-02-05T125603      tune.cfg.2021-02-05T125744
tune.cfg.2021-02-05T122026      tune.cfg.2021-02-05T125608      tune.cfg.2021-02-05T125749
tune.cfg.2021-02-05T124215      tune.cfg.2021-02-05T125614      tune.cfg.2021-02-05T125756
tune.cfg.2021-02-05T124222      tune.cfg.2021-02-05T125620      tune.cfg.2021-02-05T125802
tune.cfg.2021-02-05T124728      tune.cfg.2021-02-05T125626      tune.cfg.2021-02-05T125807
tune.cfg.2021-02-05T124739      tune.cfg.2021-02-05T125632      tune.cfg.2021-02-05T125812
tune.cfg.2021-02-05T124821      tune.cfg.2021-02-05T125727
$  

If this feels like too much clutter, you can disable the backup mechanism, as described under backup_config. Note that doing so may leave you with a risk of losing the config file in case of a filesystem overflow or system crash. A better idea may be to periodically remove just portions of the clutter, with selective removal of older version; or sweep them to a wastebasket every now and then:

$ cd $SPEC/config
$ mkdir wastebasket
$ mv *cfg.2021* wastebasket/
$

VII.B. The log file and verbosity levels

$SPEC/result (Unix) contains reports and log files. When you are doing a build, you will probably find that you want to pay close attention to the log files such as accel2023.001.log. Depending on the verbosity level that you have selected, it will contain detailed information about how your build went.

The SPECaccel 2023 tool suite provides for varying amounts of output about its actions during a run. These levels range from the bare minimum of output (level 0) to copious streams of information that are probably useful only to tools developers (level 99). Selecting one output level gives you the output from all lower levels, which may cause you to wade through more output than you might like.

VII.B.1. Useful Search Strings

When you are trying to find your way through a log file, you will probably find these (case-sensitive) search strings useful:

VII.B.2. About Temporary Debug Logs

There are also temporary debug logs, such as accel2023.001.log.debug. A debug log contains very detailed debugging output from the SPEC tools, as if --verbose 99 had been specified.

For a successful run, the debug log will be removed automatically, unless you specify "--keeptmp" on the command line, or "keeptmp=yes" in your config file.

For a failed run, the debug log is kept. The debug log may seem overwhelmingly wordy, repetitive, detailed, redundant, repetitive, and long-winded, and therefore useless. Suggestion: after a failure, try looking in the regular log first, which has a default verbosity level of 5. If your regular log doesn't have as much detail as you wish, then you can examine the additional detail in the debug log.

If you file a support request, you may be asked to send in the debug log.

VII.B.3. Verbosity levels

The 'level' referred to in the table below is selected either in the config file verbose option or in the runaccel command as in 'runaccel --verbose n'.

Levels higher than 99 are special; they are always output to your log file. You can also see them on the screen if you set verbosity to the specified level minus 100. For example, the default log level is 5. This means that on your screen you will get messages at levels 0 through 5, and 100 through 105. In your log file, you'll find the same messages, plus the messages at levels 106 through 199.

VII.D. Help, I've got too many log files!

If you do many builds and runs, you may find that your result directory gets too cluttered. Within a result directory, all output formats other than .rsf can be regenerated from your .rsf files. Therefore, you could reduce clutter by deleting HTML, PDF, and other reports. You can delete old .debug logs unless you plan to submit a support request.

Still feel cluttered? A simple solution is to move your result directory aside, giving it a new name. Don't worry about creating a new directory; runaccel will do so automatically. You should be careful to ensure no surprises for any currently-running users. If you move result directories, it is a good idea to also clean temporary directories at the same time.
Example:

cd $SPEC
mv result old-result
rm -Rf tmp/
cd output_root     # (If you use an output_root)
rm -Rf tmp/

VII.E. Finding the build directory

As described under "About Disk Usage" in runaccel.html, the SPECaccel 2023 tools do the actual builds and runs in newly created directories. The benchmark sources are never modified in the src directory.

The build directories for a benchmark are located underneath that benchmarks' top-level directory, typically $SPEC/benchspec/ACCEL/nnn.benchmark/build (Unix)

(If you are using the output_root feature, then the first part of that path will change to be your requested root instead of SPEC.)

The build directories have logical names, typically of the form build_<tune>_<label>.0000.

If the directory build_<tune>_<label>.0000 already exists when a new build is attempted for the same tuning and label, the directory will be re-used, unless:

• You have set the environment variable SPEC_NO_RUNDIR_DEL, or
• The directory has been left in a "locked" state by an aborted previous build.

In such cases, the 0000 will be incremented until a name is generated that is available. You can find locked directories by searching for lock=yes in the file $SPEC/benchspec/ACCEL/<nnn.benchmark>/run/list (Unix)

When more than one build directory has been created for a given tuning and label, you may need to trace the directory back to the specific build attempt that created it. You can do so by searching for the directory name in the log files:

$ grep build_peak_blue271.0000 *log | grep Building
accel2023.838.log:  Building 404.lbm peak blue271: (build_peak_blue271.0000) [2021-02-14 05:26:09]
$  

In the above example, the grep command locates log #838 as the log that corresponds to this run directory.

VII.F. Files in the build directory

A variety of files are output to the build directory. Here are some of the key files which can usefully be examined:

VII.G. For more information

For more information about how the run directories work, see the descriptions of specinvoke, specmake, and specdiff in utility.html.

Appendix A. Troubleshooting

When something goes wrong, here are some things to check:

1. Are there any obvious clues in the log file? Search for the word "Building". Keep searching until you hit the next benchmark AFTER the one that you are interested in. Now scroll backward one screen's worth of text.

2. Did your desired switches get applied? Go to the build directory, and look at options*out.

3. Did the tools or your compilers report any errors? Look in the build directory at *err.

4. What happens if you try the build by hand? See the section on specmake in utility.html.

5. If an actual run fails, what happens if you invoke the run by hand? See the information about "specinvoke -n" in utility.html

6. Do you understand what is in your path, and why? Sometimes confusion can be greatly reduced by ensuring that you have only what is needed, avoiding, in particular, experimental and non-standard versions of standard utilities.

   Note: on Windows systems, SPEC recommends that Windows/Unix compatibility products should be removed from the %PATH% prior to invoking runaccel, in order to reduce the probability of certain difficult-to-diagnose error messages.

7. Try asking the tools to leave more clues behind, with keeptmp.

Appendix B. Obsolete/removed items