Getting Started with SPECweb2005 Release 1.00

User Guide: Getting Started with SPECweb2005

1.0 Requirements
2.0 Setting Up The Test Environment
3.0 Running SPECweb2005
Appendix A - README file from SPECweb2005 client kit
Appendix B - Examples of common changes to configurable parameters
Appendix C - Wafgen README file
Appendix D - BeSim Make_Readme file
Appendix E - Common Errors and Debugging Tips
Appendix F - FAQ

1.0 Requirements

These instructions assume that you are familiar with the following:

Web server Software configuration
Server and Network configuration
Installing and Using Open Source Software
Installing a JVM and running Java programs

The SPECweb2005 kit contains the following components:

Test Harness, which consists of:
- specwebclient.class - a load generator which runs on each client system
- specweb.class - a test manager which runs on the prime client
- reporter.class - a report generator
- configuration files for the test and for each workload (banking, ecommerce, and support)
- Java source code
Wafgen - a file set generator which is run on the Web server to create the static files referenced by the workload.
BeSim - a Backend Simulator which runs on an independent Web server system connected to both the prime client and the system under test (SUT). BeSim includes C code which can be compiled for FastCGI, ISAPI, or NSAPI.
Dynamic scripts to be run on the Web server that implement the dynamic pages for each workload:
- PHP scripts and Smarty (http://smarty.php.net/) templates
- JSP scripts and Java class files
Assorted "readme" and "install" documentation files to help explain how do setup, configure, and run specific components. Please review these files; most have been included as appendixes to the document for your convenience.

Additional Software Requirements:

Web server Software (such as IIS, Apache, Zeus, Sun One that can support PHP or JSP scripts)
J2SE 1.4.n JVM (for running the test harness)
C compiler (for compiling BeSim)
PHP 4.3.n from www.php.net (if running the supplied PHP dynamic scripts)
Application Server such as Tomcat 4.n (if running the supplied JSP dynamic scripts)

Hardware Requirements:

SUT: a system configured as a Web server
Client(s): one or more client systems to act as load drivers for SPECweb2005
BeSim system: a client class system configured as a Web server with BeSim installed as a FastCGI, NSAPI, or ISAPI
Network: Clients must be able to connect to the SUT; BeSim system and SUT must be able to connect to the BeSim system.

Note: For debugging purposes, you could run everything on 1 or 2 systems but these instructions assume you have at least 3.

2.0 Setting Up The Test Environment

Running the SPECweb2005 installer:

Place a copy of the provided setup.jar file on all systems that are part of your test setup. You will need to have a JVM installed; a JDK or JRE version 1.4.1 or later is recommended. Invoke the Java installer as follows:

To run via X/GUI: java -jar setup.jar
To run via console: java -jar setup.jar -console

Follow the menus and select the type of installation for a given system type from the menu shown below:

SPECweb2005  - InstallShield Wizard

Choose the installation type that best suits your needs.

[X] 1 - Client / Prime Client Installation
        The benchmark Java source code and class files for running the prime
        client and client harness. Requires a Java Virtual Machine (JVM).
        Benchmark documentation is included as well in this installation type.

[ ] 2 - Web server Installation
        The Web server php and jsp script implementations that run on the System
        Under Test (SUT), file set generator tool for script paddings, images,
        and static content (requires a Java Virtual Machine), and benchmark
        documentation are included in this installation type.

[ ] 3 - Backend Simulator (BeSim) Installation
        Backend Simulator (BeSim); a Web server API that simulates an application
        or database server. Must be installed on a separate machine than the SUT.
        Benchmark documentation is included as well in this installation type.

[ ] 4 - Full Installation
        The program will be installed with the suggested configuration.
        Recommended for most users.

[ ] 5 - Custom
        The program will be installed with the features you choose.
        Recommended for advanced users.

2.1 Web Server Setup

General Directions:

Install and configure your favorite Web server with SSL and PHP or JSP support configured in. If using JSP, a separate Java application server may be needed. Check your Web server software documentation for assistance.
Run the SPECweb2005 installer (i.e. setup.jar) and set your current working directory to the wafgen directory.
Review the Wafgen README file which contains the instructions for tailoring the resource files (.rc) to create the workload specific file sets for the Web server.
Edit the two .rc files for the workload that you wish build the file set for.
Run wafgen according to the directions provided.
Check that the files generated are correct and under your Web server�s document root.
If you plan to use the JSP dynamic scripts provided:
1. Check to see that you have either a Web server that supports JSP scripts or a Java application server (i.e. Tomcat) configured.
2. Follow the directions provided in the SPECweb2005 scripts/jsp/README.FIRST and the INSTALL file from each .war file (jar xvf <wkld>.war) to install the JSP scripts in their proper locations.
If you plan to use the PHP dynamic scripts provided:
1. Check to see that you have a version of PHP (such as PHP 4.3.10 from www.php.net) and that your Web server is configured to support PHP dynamic scripts.
2. Move the contents of the SPECweb2005 scripts/php/<workload> directory to the corresponding workload directory in your document root.

2.2 BeSim Setup

Configure the BeSim system with Web server software installed on Port 81 or another port (avoid port 80).
Run the SPECweb2005 installer (i.e. setup.jar) and set your current working directory to the BeSim directory.
Review the Make_Readme.
You may choose from FastCGI* for Zeus or Apache or ISAPI for Zeus or IIS or NSAPI for SUN One and edit the appropriate MakeIncl.<type>.<OS> file to reflect your compile and linker options and select one or more Defines. Then you can build the BeSim code for your selected API.
Install this code to your BeSim Web server.
Start your BeSim Web server.
Use the test_besim_<wkld>.pl Perl scripts included in the kit to check that BeSim is functioning.

*Note: You may need FastCGI support added to your BeSim system (see: http://www.fastcgi.com/)

2.3 Client Setup

Install a 1.4.n JVM (or later) on all client systems.
Run the SPECweb2005 installer (i.e. setup.jar) on each client and set your current directory to the SPECweb2005 directory created by the installation.
Select one of the client systems to serve as the "Prime Client" (can also be a load generating client).
Review the SPECweb_<workload>.config, Test.config, and Testbed.config to understand the parameters in these files, with attention to those in the beginning (configurable) section of the workload and Test configuration files which may need to be updated to match your test bed.
On the Prime Client, edit configuration files to match your Web server, BeSim configuration, Client, and network configuration. Change only those parameters listed above the section marked "# Fixed workload properties" or in Test.config above the line marked " # SPECweb2005 fixed test properties":
- Testbed.config - contains your test bed description, tuning disclosures, and notes.
- SPECweb_Banking.config - parameters specific to Banking workload.
- SPECweb_Ecommerce.config - parameters specific to Ecommerce workload.
- SPECweb_Support.config - parameters specific to Support workload.

2.4 Network Setup

Configure all the test bed systems so that they can communicate with one another; make sure to update your /etc/hosts file (or equivalent). The Prime must be able to communicate with each Client in the test environment. The Web server and the BeSim system must be able to communicate with each other. Finally, the Prime Client and BeSim system must be able to communicate with each other.

3.0 Running SPECweb2005

Once the Web server, clients, and BeSim system have been configured and networked, the next step is to try running a test strobe for one of the workloads. To start with a short test, edit Test.config again and put these lines at the end of the file to override the defaults:

KILL_CLIENT = 1
DEBUG_LEVEL = 2
RAMPUP_SECS = 60
WARMUP_SECONDS = 60
RUN_SECONDS = 300
SIMULTANEOUS_SESSIONS = 1

Also set TEST_TYPE to the desired workload (SPECweb_Banking, SPECweb_Ecommerce, or SPECweb_Support).

Next start the test using the commands below. Note, you may want to use 2 windows or redirect the output of the two Java commands to files or create a script run the harness and to save the output:

Start the specwebclient process on each client machine: java -Xms512m -Xmx512m specwebclient
Start the test harness manager process on the prime client (this starts the test): java -Xms512m -Xmx512m specweb

After test has completed check the redirected output and the contents of the results.

Note: you may wish to set other Java parameters besides the heap parameters to optimize the specwebclient processes as you increase the test load. Check the documentation for your JVM.

Did that test work?

Yes? Great, so did these instructions �work� or were there missing details or do you have additional questions that you are now writing up to send to web2005support@spec.org?

No? Well, there should be enough information in the debug output to start debugging. Please send any corrections or requests for clarification to these instructions to the email address above. Continue reading and reviewing the Web server, BeSim, and Client Checks below, along with the related README files for the various components in the attached appendixes, particularly Appendix E on common errors and Appendix F FAQ.

Now, that the basics are working, you can try tests using more Sessions and adding more clients�..

3.1 Web server Checks

Have you configured your Web server to support the HTTP or HTTPS based on the workload you are running? Banking is all HTTPS (SSL), Ecommerce requires both HTTP and HTTPS, and Support uses only HTTP. Note: that you can temporarily override the use of HTTPS in Banking and Ecommerce by setting USE_SSL = 0 in the .config file. Results will be noncompliant but this may be useful when debugging.
Have you configured you Web server to support PHP or JSP according to the directions provided with your Web server? If using JSP and a separate Java application server such as Tomcat also review its documentation and configure appropriately. Try running the PHP or JSP "Hello World" equivalent to make sure your Web server's script engine is working.
Have you configured the wafgen .rc files for the workload and run Wafgen to create the fixed and scaling file sets? Did you set the number of SIMULTANEOUS_SESSIONS to a value equal or greater than the value you plan to use in your testing?
Can the Web server system ping the BeSim system? Connect to the BeSim Web server (telnet [BESIM_IP] [BESIM_port])?

3.2 BeSim Checks

Have you installed and configured a Web server on you BeSim system to run on port 81 (or other non-standard port number)?
Have you built and installed the BeSim application (FastCGI, NSAPI, ISAPI) in the appropriate directory under the Web server's document root?
Have you run the test_besim_*.pl scripts provided with BeSim to check that the BeSim setup is working?

3.3 Prime Client Checks

Have you checked the SPECweb_<test type>.config? Verify that BESIM_INIT_SCRIPT matches the relative path of the BeSim API installed on the BeSim Web server.
Have you checked Test.config? Verify the following parameters match your test environment:

TEST_TYPE=SPECweb_<test type>
WEB_SERVER = <SUT>
BESIM_SERVER = <BeSim system>
BESIM_PORT = 81 [or the non-80 value set up on the BeSim Web server]

Verified that the Prime Client can �ping� the other clients, the SUT and BeSim system?
Made a quick check that the prime and other clients can successfully do a �telnet [WEB_SERVER_IP] [80 and/or 443]� and that you can successfully perform a "telnet [BESIM_IP] [BESIM_port]" from the prime?

Checked that the prime (and other clients) have set per process limits for threads (i.e. max_thread_proc) and file descriptors (i.e. maxfiles) adequately for the number of simultaneous sessions you plan to test? For example, if you are running a test for 1500 simultaneous sessions and using 2 clients, each client is responsible for 750 sessions. Each session uses ~3 connections to the server, so each client process (i.e. java specwebclient) will need just over 750 * 3 threads or 2100+ threads and 750 * 3 file descriptors (socket = FD) or 2100 file descriptors. If your clients� settings aren�t adequate, then your test will fail and you�ll get a Java error telling you it couldn�t start a thread or establish a connection.

Appendix A - README file from SPECweb2005 client kit

Here are the basics for setting up SPECweb2005 Clients:

First, you should at minimum have version 1.4.1 of the Java JDK and/or JRE installed on the client machines running this code to take advantage of garbage collection optimizations in the more recent JVMs. JVM v1.4.n is strongly recommended for best performance. However, older JVM versions back to 1.3.1 may work.

Next, there are five different .config files for SPECweb2005. There is a SPECweb_<workload>.config for each of the three workloads (Banking, Ecommerce, and Support), these contain both configurable and fixed parameters specific to the workload. Test.config contains parameters that are common to all workloads and includes configurable and fixed parameters. Testbed.config contains the test bed hardware/software configuration properties.

The "specwebclient" class is the SPECweb2005 client load generator. After editing the .config files to match your test environment, you need to start this process on all client machines. To do this, simply set up your Java classpath, if needed, and at a command prompt type:
java specwebclient

Starting "specwebclient" with tuned heap sizes and garbage collection
parameters is also recommended. Specifically, for v1.4.1 of the JDK on a
multi-processor client, using parallel and CMS garbage collection is strongly
recommended. Example:

java -cp c:\sw2005 -Xms512m -Xmx512m -XX:+UseParNewGC \
-XX:+UseConcMarkSweepGC specwebclient

The usage for specwebclient is:

java specwebclient -h
Usage: java specwebclient [-v] [-h] [-p port] [-n hostname] [-s server]
[-le errorlog] [-l|-lo stdoutlog]

-v: print SPECweb benchmark version
-h: prints this message
-p: specify the port number for specwebclient to use {allows multiple instances on same machine}
-n: specify the hostname that specwebclient will run on
-s: specify the web server name (overrides WEB_SERVER in Test.config)
-le: redirect errors to log file name specified (default: specwebclient.err)
-l: redirects both errors and stdout to one file (unless -le is also used)
-lo: redirect stdout to log file name specified (default: specwebclient.out)

The "specweb" class is the SPECweb2005 test manager and is run on the Prime Client. It is started after all the "specwebclient" processes have been started. Starting specweb should begin the test. The "specweb" processor exits at the end of the test, the "specwebclient" process can be configured to terminate as well (see Test.config). To start the "specwebclient" process, simply set up your Java classpath, if needed, and at a command prompt type:

java specweb

The usage for specweb is:
java specweb -h
Usage: java specweb [-C] [-v] [-h] [-w workload] [-le errlog] [-l|-lo stdoutlog]

-C: use SPEC-compliant configuration values; non-compliant values
overwritten.
-v: print SPECweb benchmark version
-h: prints this message
-w: name of the workload to run (overrides TEST_TYPE in Test.config)
-le: redirect errors to log file name specified (default: primeclient.err)
-l: redirects both errors and stdout to one file (unless -le is also used)
-lo: redirect stdout to log file name specified (default: primeclient.out)

The "reporter" class is the SPECweb2005 report generator and is invoked by "specweb" at the end of a test to produce .TXT and .HTML reports. It can also be invoked to create a merged .raw file from results from running the 3 workloads on a given test bed for a submission to SPEC; as well as generating combined .TXT and .HTML report files. Here is the usage for the reporter:

Usage: java reporter [-h] [-v] [-c banking_rawfile ecommerce_rawfile support_rawfile] [-R submit_rawfile]
             [workload_rawfile]

java reporter -h
SPECweb2005 Reporter 1.00
Copyright (c) 2005 Standard Performance Evaluation Corporation

Command line flags:
   -h Print this message and exit
   -v Print version string and exit
   -c Combine 3 separate workload files into one submittable .raw file
   -R Regenerate HTML and ASCII reports from combined rawfile

Example Usage:
--------------
Regenerate reports from a single raw file:
java reporter results\SPECweb_Support.20050323-102204.raw

Combine three compliant workload raw files into one submittable raw file:
java reporter -c bank.raw ecom.raw support.raw submit.raw

Regenerate reports from a previously combined raw file:
java reporter -R submit.raw

Appendix B - Examples of common changes to configurable parameters

SPECweb_Banking.config:

36c36
  < PADDING_DIR = "/www/webroot/bank/dynamic_padding/"
  ---
  > PADDING_DIR = "/www/bank/dynamic_padding/"
  47,48c47,48
  < CHECK_IMAGE_DIR = "c:/www/webroot/bank/images"
  < CHECK_IMAGE_SUBDIRS = 10
  ---
  > CHECK_IMAGE_DIR = "/www/bank/images"
  > CHECK_IMAGE_SUBDIRS = 100

SPECweb_Ecommerce.config:

36c36
< PADDING_DIR = "/www/webroot/ecommerce/dynamic_padding/"
  ---
> PADDING_DIR = "/www/ecommerce/dynamic_padding/"

SPECweb_Support.config:

40c40
< PADDING_DIR = "c:/www/webroot/support/dynamic_padding/"
    ---
> PADDING_DIR = "/www/support/dynamic_padding/"
    50c50
< DOWNLOAD_SUBDIRS = 0
    ---
> DOWNLOAD_SUBDIRS = 2

Test.config:

30c30
  < CLIENTS = "localhost"
  ---
  > CLIENTS = "client50:1095 client40:1094 client30:1093 client20:1092"
  32c32
  < SIMULTANEOUS_SESSIONS = 5
  ---
  > SIMULTANEOUS_SESSIONS = 1100
  35,36c35,36
  < TEST_TYPE=SPECweb_Banking
  < # TEST_TYPE=SPECweb_Ecommerce
  ---
  > # TEST_TYPE=SPECweb_Banking
  > TEST_TYPE=SPECweb_Ecommerce
  41c41
  < WEB_SERVER = localhost
  ---
  > WEB_SERVER = sut2005
  54c54
  < BESIM_SERVER = "localhost"
  ---
  > BESIM_SERVER = "BeSimbox"
  56c56
  < BESIM_PORT = 81
  ---
  > BESIM_PORT = 82
  63a64
  > BESIM_INIT_SCRIPT = "/FastCGI/BESIM_fcgi.fcgi"
  67,70c68,71
  < SMARTY_DIR = "c:/www/php/Smarty-2.6.3/"
  < SMARTY_BANK_DIR = "c:/www/php/Smarty-2.6.3/banking/"
  < SMARTY_ECOMMERCE_DIR = "c:/www/php/Smarty-2.6.3/ecommerce/"
  < SMARTY_SUPPORT_DIR = "c:/www/php/Smarty-2.6.3/support/"
  ---
  > SMARTY_DIR = "/www/Smarty-2.6.9/"
  > SMARTY_BANK_DIR = "./"
  > SMARTY_ECOMMERCE_DIR = "./"
  > SMARTY_SUPPORT_DIR = "./"
  107c108
  < KILL_CLIENT = 0
  ---
  > KILL_CLIENT = 1

Appendix C - Wafgen README file

Invoking Wafgen:
================

    Unix: Wafgen <workload_specific>.rc

    Windows: Wafgen.bat <workload_specific>.rc

    or alternately:
      java -Xms384m -Xmx384m -jar Wafgen.jar <workload>.rc
    or:
      java -Xms384m -Xmx384m -classpath Wafgen.jar org.spec.wafgen.Wafgen <workload>.rc

Wafgen Overview
===============

Wafgen is used to create the file sets used by SPECweb2005. Each workload
has a "fixed" and a "scaling" file set. The fixed file set includes a set of
static elements that are commonly referenced by the pages that make up a given
workload. The scaling file set includes a set of directories each containing
additional static elements for the workload. The number of directories created
(or referenced by a test) is calculated by:
(SIMULTANEOUS_SESSIONS * DIRSCALING) = Total # directories

The total amount of storage required for a benchmark is also be dependent on the total number of directories in the scaling file set and the size of the contents of each directory (see Storage Requirement below). Since the total number of directories needed could exceed system limits for the number of files per directory (usually ~32,000), an additional level of subdirectories can be added using the SUBDIR* parameters. By default, banking and ecommerce use SUBDIR's. The scaling directories are round-robined across the specified
number of subdirectories.

This kit contains the resource files (.rc) for each workload that are used as input to Wafgen. There is a set tailored for both Windows and Unix/Linux in the windows and unix subdirectories. To use these files copy the appropriate set into the upper directory and edit them as needed to match your test environment. For conforming results - ONLY parameters listed under Section A of the .rc file may be modified. Double check all section A parameters, especially:
        DOCROOT (if not set will default to current directory)
        BASEDIRNAME (see examples in <workload_specific>.rc files)
        BASESUBDIRNAME (if applicable)
        SIMULTANEOUS_SESSIONS (may be larger than the corresponding value in
           SPECweb_<workload>.config)
Also verify that the SPECweb_<workload>.config file parameters in the test harness on the Prime Client are in agreement.

The .rc files for fixed file sets include:
        bank_image_props.rc
        ecommerce_image_props.rc
        support_image_props.rc

The .rc files for scaling file sets include:
        bank_usercheck_props.rc
        ecommerce_productline_props.rc
        support_downloads_props.rc

The .rc files are annotated to provide additional guidance.

Reminder: Altering .rc file parameters listed in section B will result in non-conforming tests, so only modify parameters in section A.

Storage Requirements
====================

Since the benchmark requires that the System Under Test (SUT) contain the file sets for all three workloads, it is important to understand the storage requirements for each workload. Below are the data and formulas to help estimate storage requirements for SPECweb2005.

Banking
=======

The fixed file set created by bank_image_props.rc includes:

o bank/images -
        44 files; Average size: 3570 bytes; Total size: 157096 bytes

o bank/dynamic_padding -
        15 files; Average size: 21800 bytes; Total size: 327000 bytes

The scaling file set created by bank_usercheck_props.rc creates a number
of user0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.

o Each bank/images/{subdir00mm}/user0000nnnnnn contains -
        20 files; Average size: 10431 bytes; Total size: 208625 bytes

To estimate the storage requirement for the user check directories use
the following formula:

        (SIMULTANEOUS_SESSIONS * CLIENT_SESSION_USER_IDS) * 208625

For Conforming results, CLIENT_SESSION_USER_IDS (alias for DIRSCALING)
must be 50.

Example:
    ((1000 SIMULTANEOUS_SESSIONS * 50 CLIENT_SESSION_USER_IDS) * 208625) =
        10,431,250,000 bytes

Ecommerce
=========

The fixed file set created by ecommerce_image_props.rc includes:

o ecommerce/images -
        172 files; Average size: 2032 bytes; Total size: 349557 bytes

o ecommerce/dynamic_padding -
        11 files; Average size: 73000 bytes; Total size: 803000 bytes

The scaling file set created by ecommerce_productline_props.rc creates a number
of productline0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.

o Each ecommerce/images/{subdir00mm}/productline0000nnnnnn contains -
        30 files; Average size: 18816 bytes; Total size: 564495 bytes

To estimate the storage requirement for the product line directories use
the following formula:

        (SIMULTANEOUS_SESSIONS * DIRSCALING) * 564495

For Conforming results, DIRSCALING must be 5.

Example:
    ((1000 SIMULTANEOUS_SESSIONS * 5 DIRSCALING) * 564495) =
        2,822,475,000 bytes

Support
=======

The fixed file set created by support_image_props.rc includes:

o support/images -
        31 files; Average size: 728 bytes; Total size: 22580 bytes

Warning: this file set contains several very small files (30 bytes)
so the path name must not be longer than 14 characters. So,
"support/images" is OK but "supporting/image" is NOT.

o support/dynamic_padding -
        6 files; Average size: 59616 bytes; Total size: 357700 bytes

The scaling file set created by support_downloads_props.rc creates a number
of user0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.

o Each support/downloads/{subdir00mm}/dir0000nnnnnn contains -
        16 files; Average size: 4279501 bytes; Total size: 68472021 bytes

To estimate the storage requirement for the download directories use
the following formula:

        (SIMULTANEOUS_SESSIONS * DIRSCALING) * 68472021

For conforming results, DIRSCALING must be 0.25.

Example:
    ((1000 SIMULTANEOUS_SESSIONS * .25 DIRSCALING) * 68472021) =
        17,118,005,250 bytes

Note: The heap parameters used for Wafgen are required to support the
      large files in the scaling file set for Support workload.

Tips for Building Large Filesets
================================

o To increase the size of an existing file set to support a larger number of
Simultaneous Sessions than used initially, the steps are:

        Edit the <workload>.rc file for the scaling fileset create previously
        and change these parameters:

        SIMULTANEOUS_SESSIONS=<new higher value>
        FIRSTDIRECTORY=<1+highest directory number previously create>

        For example:

        Originally a support_downloads_props.rc file used:
                SIMULTANEOUS_SESSIONS=1000
                FIRSTDIRECTORY=1

        When a file set to support 2000 Simultaneous Sessions is needed,
        change:

                SIMULTANEOUS_SESSIONS=2000
                FIRSTDIRECTORY=501

        Then run: Wafgen <updated_workload>.rc

o To run several Wafgen's in parallel to create a large file_set, copies of
the <workload>.rc file can be created for each parallel wafgen to be run; however,
appropriate changes to the SIMULTANEOUS_SESSIONS and FIRSTDIRECTORY
directories must be made. To calculate the correct values for FIRSTDIRECTORY in each copy use the formula:

        Number of Directories = SIMULTANEOUS_SESSIONS * DIRSCALING
        (or for banking the DIRSCALING alias CLIENT_SESSION_USER_IDS is used).

Use the directory scaling formula for the workload (shown above) to
calculate the correct values for FIRSTDIRECTORY in each copy.

        For example, to create a support downloads file set for a total of
        4000 Simultaneous Sessions:

        support_downloads_props.rc_1:
                SIMULTANEOUS_SESSIONS=1000
                FIRSTDIRECTORY=1

        support_downloads_props.rc_2:
                SIMULTANEOUS_SESSIONS=2000
                FIRSTDIRECTORY=501

        support_downloads_props.rc_3:
                SIMULTANEOUS_SESSIONS=3000
                FIRSTDIRECTORY=1001

        support_downloads_props.rc_4:
                SIMULTANEOUS_SESSIONS=4000
                FIRSTDIRECTORY=1501

        Then run 4 parallel wafgens:

                Wafgen support_downloads_props.rc_1 &
                Wafgen support_downloads_props.rc_2 &
                Wafgen support_downloads_props.rc_3 &
                Wafgen support_downloads_props.rc_4 &

        (Note: depending on the number of CPUs and the I/O subsystem being
         used this sequence may not take significantly less time than just
         running a single Wafgen for the whole file set.)

o To convert from a file set that was recreated without using *SUBDIR*
parameters or to add additional subdirectories, the original file set will
need to be re-created from scratch. The <workload>.rc file will need to
be edited to configure these parameters. The parmeters are preceeded by
the comment string "#subdirs#" if use of SUBDIRS is not the default
(i.e Support).

o Multiple disks or mount points may be used to hold subsets of a workload's
file sets, for example: /<docroot>/bank may have the images directory as
as separate mount point or /<docroot>/bank/images may have separate mount
points for each of 10 subdirectories (subdir0001 thru subdir0010).

Warning:

        The path names have been standarize in the wafgen <workload>.rc files
        and the SPECweb_<workload>.config files in the harness, so that
        in the DOCROOT there would be these direcories:

                bank/images
                bank/dynamic_padding

                ecommerce/images
                ecommerce/dynamic_padding

                support/images
                support/dynamic_padding
                support/downloads

        While it is possible to change these names in the .rc and .config i
        files with due care (and making sure not to exceed path length limits
        for some very small files); it is recommended that users retain this
        naming to avoid errors due to typos and to simplify debugging.

Appendix D - BeSim Make_Readme file

How to Build BeSim: UNIX/LINUX:

Type: uname -s to get name of the operating system (i.e. HP-UX)
In the Makefiles directory use one of the MakeIncl.<type>.<OS>
files as a template for your MakeIncl.<type>.<your_OS> and
update to match you environment.

To Build FastCGI version of BeSim:     make fcgi

To Build NSAPI version of BeSim:        make nsapi

To Build Zeus ISAPI version of BeSim:   make zisapi

Optionally you can specify other make targets by adding TARGET='list of targets'
For example:

        make zisapi TARGET='clean'
        make nsapi TARGET='clean all install'

The install target will use the path specified by DEST to install the BeSim
exeutatble or library, so it should be set to the appropriate place on your
BeSim Web server directory tree (i.e. /www/FastCGI).

If you plan to use FastCGI you must build and install the FastCGI development
kit which provides the FastCGI library and include files before build BeSim.
If using Apache for the BeSim Web server, that will need to include mod_fastcgi.
See: http://www.fastcgi.com/

Note: Please send any new or updated MakeIncl.* files you want included in
a future release to web2005support@spec.org.

Windows:
Project files are included as an example.

Appendix E - Common Errors and Debugging Tips

By default the DEBUG parameter is set to 1 in Test.config, if a validation error occurs this value is sufficient to print out the ERROR message related to the invalid response. The invalid response is also printed out and that is followed by a printout of the Request that produced the invalid response. Reading the returned HTML response and the associated HTTP Request is key to debugging problems.
Status 404 (file not found) errors usually mean that there is a path problem either in the .config files on on the associated Web server. In some cases it may indicate that not a large enough file set was created by Wafgen.
Errors Returned by BeSim are found in the body of the returned HTML <brseconds may="" need="" to="" be="" increased=""></brseconds>

Appendix F - FAQ

Q: How long will it take to complete a full compliant 3 iteration run for one of the 3 workloads?

A: The Minimum required time is: ~144 minutes (2.4hrs), with each iteration at 48 minutes. If WARMUP_SECONDS or THREAD_RAMUP_SECONDS have been increased, the test will run somewhat longer.
---------------------------------------

Q: My clients are printing "Java.lang.OutOfMemoryError: Java heap space" errors. What can I do to eliminate these?

A: Invoke the client JVMs with more heap space; for example, assuming the clients have sufficient RAM, the initial and maximum heap sizes can be forced to 512MB (these values are only suggestions, and will vary depending upon system):

java -Xms512m -Xmx512m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC specwebclient

More information about heap size and garbage collection can be found here:
http://java.sun.com/docs/hotspot/gc1.4.2/faq.html
http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html

---------------------------------------

Q: When trying to manually access PHP pages (i.e. from a browser), the PHP error log or the response displays "failed to open stream: No such file or directory in [PATH]\inc.common.php"

A: You must first initialize the benchmark by running 'Java specweb' with all the variables in the .config files set properly. The init.php script for each workload will write these variables to init_vars.php, so be sure user the Web server runs as has write permissions for the SPECweb2005 directories.
---------------------------------------

Q: I'm running the PHP scripts on Apache (either as the primary Web server or as the BeSim server) and I'm seeing errors.

A: If you have BESIM_PERSISTENT=1, try changing it to 0. During testing, Apache was observed to have issues with attempting to use persistent sockets to BeSim. It is suspected that this is due to the Apache architecture which forks multiple child processes to respond to requests; this makes it difficult to reuse a connection across script executions.

More information about the issue can be found here (search on Apache):
http://www.php.net/pfsockopen
---------------------------------------

Q: After running "java specweb", it tries to run the INIT_SCRIPT on the Web server but fails.

A: Ensure that the user the Web server process runs as has sufficient permissions to write to the appropriate SPECweb2005 directories. For PHP (all workloads), init.php attempts to write init_vars.php, which contains server-specific variables required to run the benchmark.
---------------------------------------

Q: Why do I get "access to globals failed" in BeSim responses?

A: For Unix, BeSim attempts to write a globals file to the /tmp directory; for Win32, the root of the C: drive. There are two possible solutions:
- Change the permissions on the appropriate directory so the user the Web server runs as can read and write this globals file.
- Modify BESIM_banking.h, BESIM_ecommerce.h, and BESIM_supportsite.h to specify different BESIM_BANKING_GLOBALS_PATH, BESIM_ECOMMERCE_GLOBALS_PATH, and BESIM_SUPPORT_GLOBALS_PATH variables respectively. You must then recompile the API.
---------------------------------------

Q: The PHP scripts don't appear to be running properly, and I don't get any output from a browser when requesting the pages manually. How do I troubleshoot this?

A: PHP ships with two ini files, php.ini-dist and php.ini-recommended. While php.ini-recommended is optimized for performance and security, for initial troubleshooting, php.ini-dist may be a better choice. This will configure PHP to display errors in the browser. You may also want to set "error_reporting = E_ALL", which will display all errors and warnings.
----------------------------------------

Q: How can I test whether I've compiled and installed BeSim correctly?

A: The test_besim_bank.pl, test_besim_ecom.pl, and test_besim_support.pl Perl scripts (included in the BeSim directory) good ways to test whether you're getting valid BeSim responses. Invoke these scripts with the URL to your compiled BeSim API, i.e.
"perl test_besim_bank.pl http://192.2.1.132/isapi-bin/BESIM_zisapi.api".
----------------------------------------

Q: I'm getting invalid account values in client logs and it use to work fine"

A: BeSim may need to be to restarted cleanly along with the rest of the testbed software. Try:
       - stopping the webserver on the SUT
      - stopping the BeSim webserver
      - renove /tmp/besim_*.globals
       - remove old besim access logs
       - check that none of your clients are running "java specwebclient" processes (kill these processes if present)
       - start the webserver on the SUT and on BeSim box
       - re-run your testA
-----------------------------------------

Q: My clients are dying with "Exceeded max allowed overthink time" errors, how do I prevent these?

A: Check the following items:
- Ensure your clients are not overloaded.
- Ensure THINK_TIME to a reasonable level.
- Increase THREAD_RAMPUP_SECONDS in Test.config.
----------------------------------------

Q: My client logs show sequences of errors that begin with:
2005-05-14 07:27:44:909 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
followed by a cople of other header and response errors.
Occasionally my log also has error sequences that begin with:
2005-05-14 07:37:02:613 Connection: [ERROR] Bad status: 503

A: Your server is overloaded or under-tuned for the number of Simultaneous Sessions the test is requesting. The SocketTimeoutException error occurs when the server was unable to send a response back before the PROTOCOL_TIMEOUT_SECONDS (60 sec) has expired. The 503 status is returned by some webservers to indicate that the service is temporarily unavailable. See if reducing the load on the server eliminates these messages, if so you may wish to investigate if there is addition tuning that could be done to the SUT.
----------------------------------------

Q: My clients reported errors during the test but they don't show up in my run results. Why?

A: If the errors occur before the line that reads "WorkloadScheduler: Clearing statistics...", then they occurred during warmup and thus are not counted as errors.
----------------------------------------

Q: When navigating the ecommerce pages in a browser, why does "Session does not exist" appear when I click the "Add to Cart" button?

A: Make sure that you are accessing the Web server with the same host name as you specified for the WEB_SERVER variable in Test.config. For example, if WEB_SERVER=192.168.0.1 in your Test.config, you should use a URI such as http://192.168.0.1/ecommerce/index.php (and not a hostname that resolves to that IP address). This is due to the way the Location redirects are done in the scripts.
----------------------------------------

Q: Why do I get a segmentation fault when using the Sun 1.4.x JVM on the x86_64 architecture?

A: This problem is not specific to SPECweb2005. It has to do with the processor having NX (No-Execute, also known as Execute Disabled) on. There are two known workarounds:

- Upgrade to a 1.5.x JVM
- Disable NX in your operating system. Under Linux, this can be done by passing the "noexec=off" command-line option to the kernel. If the kernel is 64-bit, "noexec32=off" should also be passed.
---------------------------------------

Q: How do I get the PHP scripts running on Windows Server 2003 / IIS 6.0?

A: There is a helpful tutorial available here: http://www.peterguy.com/php/install_IIS6.html

You must also configure IIS to serve all files regardless of file name extension by adding a wildcard character (*) MIME type. NOTE: This should not be performed on production Web servers. More information can be found here:
http://www.microsoft.com/resources/documentation/iis/6/all/proddocs/en-us/wsa_mimemapcfg.mspx
----------------------------------------

Q: I changed BASEDIRNAME in support_image_props_rc file from the default "support" to "supp1data". The wafgen was failing to create the image files under the image directory with an error message as: Error: path length too long for requested file size - what's wrong?

A: The message: Error: path length too long for requested file size has to do with very small files (like ccc which is 30 bytes). To be a specweb file it has to have the byte count, relative file name and at least 1 char. of data. So to handle support's smallest image file, the path can be <7chars>/<6chars>/<name>. While you can change the path to something else (sup/sup_images or sup1data/img01 or other as long as it fits into this footprint and the changes are reflected in the SPECweb_<workload>.config. The Wafgen README does include the following warning under the section on Support: Warning: this file set contains several very small files (30 bytes) so the path name must not be longer than 14 characters. So, "support/images" is OK but "supporting/image" is NOT.