SPEC CPU2017 System Requirements

$Id: system-requirements.html 5862 2017-09-25 11:56:04Z JohnHenning $ Latest: www.spec.org/cpu2017/Docs/

Contents

Overview

I. Introduction

II. Hardware

A. CPU

B. Memory

C. Disk

III. Software

A. Operating System

B. Compilers or binaries

IV. Install media

V. Don't be root. Don't be Administrator. Turn privileges off. (Usually.)

VI. Problems?

System Requirements Overview
CPU ARM, Power ISA, SPARC, or x86. Limitations apply. See: Supported Toolsets.
Physical Memory SPECspeed 16 GB May vary: memory detail.
SPECrate 1 to 2 GB per copy
Disk Space Recommended 250 GB. For SPECrate, add 1.2 GB per copy May vary: see disk detail.
Other options are available for smaller installations: see disk detail.
Operating System AIX, Linux, Mac OS X, Solaris, or Windows. Limitations apply. See: Supported Toolsets.
Compilers

C99, C++2003, Fortran-2003.

or

Pre-compiled benchmarks.

All 3 are required for reportable. See: Compilers and Precompiled binaries.
 

I. Introduction: Suites and Metrics

SPEC CPU2017 includes source code and data sets for 43 benchmarks, organized into 4 suites:

Short
Tag
Suite Contents Metrics How many copies?
What do Higher Scores Mean?
intspeed SPECspeed 2017 Integer 10 integer benchmarks SPECspeed2017_int_base
SPECspeed2017_int_peak
SPECspeed suites always run one copy of each benchmark.
Higher scores indicate that less time is needed.
fpspeed SPECspeed 2017 Floating Point 10 floating point benchmarks SPECspeed2017_fp_base
SPECspeed2017_fp_peak
intrate SPECrate 2017 Integer 10 integer benchmarks SPECrate2017_int_base
SPECrate2017_int_peak
SPECrate suites run multiple concurrent copies of each benchmark.
The tester selects how many.
Higher scores indicate more throughput (work per unit of time).
fprate SPECrate 2017 Floating Point 13 floating point benchmarks SPECrate2017_fp_base
SPECrate2017_fp_peak
The "Short Tag" is the canonical abbreviation for use with runcpu, where context is defined by the tools. In a published document, context may not be clear.
To avoid ambiguity in published documents, the Suite Name or the Metrics should be spelled as shown above.

You may choose which suite(s) you would like to run (there is no requirement to run all of them), and your choice affects hardware requirements: both memory and disk space differ between SPECspeed and SPECrate.

Having chosen a suite, if you will use your results in public, then you must run all the benchmarks in the suite (exceptions) and produce at least the base metric. The peak metric is optional. If producing both base and peak, you will need more disk space.

II. Hardware requirements

II.A. CPU

II.A.1 CPUs supported

SPEC supplies toolsets for ARM, Power ISA, SPARC, or x86.
Limitations apply:

Although SPEC CPU suites are intended to be useful with a wide range of chip architectures, in some cases it is possible that you may find that your chip is not compatible with the available toolsets (for example, if your chip is too old to run one of the supported OS+chip combinations). See the section on Supported Tools.

II.A.2 CPUs: Physical, Virtual, SMP, OpenMP

You can use SPEC CPU2017 to measure the compute performance of a system with physical or virtual CPUs/processors. You can choose to measure all of the processors on a system, or a subset.

Typical: full system, all its CPUs

Usually SPEC CPU has been used to measure entire systems, with all of the physical CPU chips.

Alternatives: VM, partition, zone,...

There is no prohibition against using SPEC CPU2017 to measure a subset of the CPU resources on a system, for example with a Virtual Machine (VM), partition, zone, domain, container, processor set, core, hardware thread, or other subset.

If you use a subset (such as a VM), it must have enough memory and disk. For public results, follow the usual rules. (Examples: use only methods that are documented, supported, and generally available to customers; fully disclose what you do, with sufficent detail so that the result can be reproduced; and if you enhance performance by doing something outside the subset -- such as temporarily silencing all the "noisy neighbor" VMs -- fully disclose that, too.)

How many CPUs? [link].

You can use SPEC CPU2017 to measure performance with arbitrarily large numbers of processors. Usually, if your operating system can handle it, the SPEC CPU toolset can also handle it; if not, contact SPEC. (Of course, you must meet the other requirements of this document, such as sufficient disk and memory for your intended test.)

SPECrate suites run multiple identical jobs at the same time.

Typically, your OS spreads them across CPUs using capabilities known as SMP (Symmetric MultiProcessing).

SPECspeed suites run a single job at a time.

Some of the jobs can use multiple CPUs via OpenMP or compiler auto-parallelization.

The above points are summarized in the table below

SPEC CPU2017 use of multiple CPUs
  SPECspeed SPECrate
Copies Only one copy of each benchmark is run at a time. Multiple copies of each benchmark run simultaneously.
Compiler Parallelism May be compiled using OpenMP and/or compiler automatic parallelism. OpenMP and compiler autopar are forbidden.
How To At build time, enable OpenMP.
Use runcpu --threads=N
or the corresponding config file feature.
Use runcpu --copies
or the corresponding config file feature.

II.B. Memory

The nominal main memory requirements are:

(In this section, 1 GB = 2^30 bytes.)

More detail:

The nominal memory requirements are based on pre-release testing by SPEC using Linux with GCC and

-m64          for speed 
-m32          for rate
ps -o vsz     to record memory usage 

which measured the SPECrate benchmarks at 0.89 GB and SPECspeed at 14.99 GB, thus leaving a small margin for overhead vs. the nominal 1 GB and 16 GB.

If you compile the SPECrate benchmarks in 64-bit mode, more memory will be needed. On one system that was tested prior to the release of SPEC CPU2017, the largest benchmark used 1450 MB when compiled with -m64. Your system might need more.

The SPECspeed benchmark 627.cam4_s needs large stacks, both for the main process (typically controlled via ulimit -s) and for OpenMP threads (typically controlled through your config file setting of environment variables). See the FAQ item on cam4.

Warnings:

II.C. Disk space

The disk space recommendations below are only estimates. Your environment may differ.

Installation Type Intended Usage Detail: observatons from one user's system. Yours may differ!
Development
250 GB
+
1.2 GB / copy
  • Install, compile, and run on a single system.
  • Do tuning runs. Test compiler options.
  • Keep 2 to 4 sets of builds and runs.
  • Eventually you will still need clean up.
2.5 GB Install SPEC CPU2017
26 GB Build base+peak for all 4 suites
34 GB Reportable directories, base+peak, 4 suites, 1 copy
0.6 GB Add another copy to intrate and fprate, base+peak
Reportable Basic
40 GB
+
0.8 GB / copy
  • Install binaries built elsewhere.
  • Use --reportable for 1 suite
    (intrate, fprate, intspeed, or fpspeed)
  • Requires clean up between suites.
  • Cannot select multiple suites with one runcpu command
2.5 GB Install SPEC CPU2017
5 GB Install benchmark binaries (base+peak) for all 4 suites
10 GB Reportable directories, base+peak, 1 suite, 1 copy
0.4 GB Add another copy, base+peak, 1 suite
Minimized
10 GB
+
0.2 GB / copy
  • Cannot do reportable runs.
  • Cannot build a full suite (intrate, fprate, intspeed, fpspeed)
  • Can build and run 1 benchmark (e.g. 500.perlbench_r)
  • Must use minimize_builddirs.
  • Must runcpu --action=clobber frequently
2.5 GB Install SPEC CPU2017
2.5 GB Build and run largest single benchmark, minimize_builddirs, base only
0.1 GB Add another copy, base only, largest single benchmark
The "Detail" column on the right shows actual disk space usage on one particular system.
Your usage may differ, due to hardware, operating system, disk type, file system type, compiler, and, especially, compiler tuning.
The far left column is a factor of 2x to 4x the detail column.
Once you know the space consumption pattern for your hw/sw, you might adjust the above calculations to be more accurate for your needs.

III. Software requirements

III.A. Operating System

The SPEC CPU2017 toolset relies on several open source components, including GNU Make, Perl, and others. SPEC supplies pre-built versions of these for particular combinations of hardware and operating system, as shown in the table below.

Supported Toolsets for SPEC CPU2017 V1.0
Toolset Intended Use
aix-ppc 64-bit PowerPC systems running AIX V7.1 or later.
linux-aarch64 64-bit AArch64 systems running Linux.
linux-armv7l Linux systems with ARM Cortex-A7-compatible CPUs.
linux-ppc 32-bit PowerPC systems running Linux.
linux-ppc64 64-bit big-endian PowerPC Linux systems
linux-ppc64le 64-bit little-endian PowerPC Linux systems
linux-sparc 64-bit Linux systems on SPARC hardware
linux-x86_64 x86_64 Linux systems
macosx-x86 MacOS X 10.6+ on Intel systems.
solaris-sparc 64-bit SPARC systems running Solaris 10 08/11 or later, including Solaris 11 and later.
solaris-x86 x86 systems running Solaris 10 08/11 or later, including Solaris 11 and later.
windows-x64 Windows systems running Windows 7 or later

Limitations apply: Although SPEC has tested the above, it is possible that you may enounter an OS+chip combination that is not compatible with its intended toolet. In such cases:

What about other systems? For systems that are not listed in the table of supported toolsets:

III.B. Compilers or binaries

SPEC supplies the benchmarks in source code form. Therefore, you will need either:

  1. A set of compilers. All three are needed:
    1. C99
    2. C++2003
    3. Fortran 2003

    --or--

  2. A pre-compiled set of benchmark binaries (executables), given to you by another user of the same revision of SPEC CPU2017.
    Binaries generated by a different revision of SPEC CPU2017 will not work.
    You might also need a set of run-time libraries for those executables.
    Note that even if someone else compiles the benchmarks, the tester is responsible for compliance with the rules: do not accept binaries from untrusted sources.

New with CPU2017, the integer suites include a Fortran benchmark.
Reportable runs for all suites require compilation of all three languages (C, C++, Fortran).

Config file

IV. Install Media

You should be familiar either with basic shell commands for Unix (ls, cp, mkdir ...) or basic commands for the Windows Command Prompt (dir, copy, mkdir...).

You will need access to the SPEC CPU2017 installation media, typically as an ISO image. The Installation Guides (Unix Windows) explain how to use it.

On some systems, the mount command may require privileges or may require additional software. In such cases, you might need to burn a physical DVD using some other system; or, you might need to use the procedure described in the appendix to the Unix installation guide to extract a tarball and use that instead.

Please note that the SPEC CPU2017 license agreement does not allow you to post the SPEC CPU2017 software on any public server. If your institution has a SPEC CPU2017 license, then it's fine to post it on an internal server that is accessible only to members of your institution.

V. Don't be root. Don't run as Administrator. Turn privileges off. (Usually.)

Usually, you do not need privileges. The one known exception is that during installation, on some systems it is possible that you might need a privileged user to mount the installation media or to allocate resources, for example disk space.

After installation is complete, you should not need privileges to run SPEC CPU benchmarks. In general, it is recommended that you use an ordinary user account for SPEC CPU; that way, if your config file accidentally tries to delete the wrong directory, you are much less likely to damage your system.

Warning: SPEC CPU config files can execute arbitrary shell commands.
Read a config file before using it.

SPEC recommends:

VI. Problems?

In case of difficulties, please check the document SPEC CPU2017 Frequently Asked Questions.

SPEC CPU®2017 System Requirements: Copyright © 2017 Standard Performance Evaluation Corporation (SPEC)