JVM Client98 Help

Autorun Button

The Autorun button is similar to the Start button in that it begins execution of one or more benchmarks with the selected problem size. However, instead of running each benchmark a single time it runs each benchmark multiple times to determine the best (shortest) time. These runs are controlled by a number of parameters specified in the user properties file.

This methodology is used for measuring results for the SPECjvm98 metric. The primary reason is to illustrate a trade-off between time spent optimizing a program and time saved by executing that optimized program. I.e., if a program will run for many minutes then it makes sense to spend considerable time optimizing it in a Just-In-Time compiler; however JIT optimization time spent on a program that runs only a few seconds may well be wasted. The same trade-off also applies to dynamic optimizers like HotSpot.

The first time each of the benchmark programs executes, the JVM must perform a number of "overhead" functions which typically will make this first execution the slowest:

• It must read the program's class files from the web server
• It must perform security verification of those class files
• It must perform static initialization of the classes
• It may perform JIT compilation of some or all of the classes

On subsequent executions of the same program, these overheads may be avoided, and so typically later executions will run faster. This will not always be the case, however, for several reasons. Dynamic optimizers may defer some optimization from the first execuction of a class, until later when it has decided that the class is going to run long enough to make the optimization worth the cost. Also, garbage collection can have a significant impact on run time, and it is not in general possible to predict when it will occur. Some systems perform garbage collection continually during execution of applications. Other systems perform garbage collection only when free heap space falls below a certain point. If no garbage collection occurs during the first execution of a benchmark, and then a significantly long garbage collection occurs during the second execution of that benchmark, then that second execution may well take longer than the first.

For that reason SPECjvm98 metrics are not based on the "first" and best times, but on the "worst" and best times.

When a benchmark is run in autorun sequence shorter (untimed) executions are interspersed between the longer timed executions. That is, if you are running size 100, then it runs 100, 10, 100, 10, 100, 10, etc., and uses only the times from the size 100 executions. If you're running size 10, then it runs 10, 1, 10, 1, etc. The reason the smaller executions are interspersed is to discourage "over-optimization" of the repeatedly executed program. If a JVM is doing dynamic optimization on the program, the smaller executions will at least change various loop counts and branch directions on which it is basing its decisions, and in some cases changes the input workload completely.

Go to description of Main Window