Q. Are there any scripts or pre-loaded VMs available to help me set up the benchmark?
A. The SPEC Virtualization subcommittee provides scripts that assist you in rapidly creating a SPEC virt_sc example test environment. You can create a SPEC virt_sc tile consisting of the five different workload VMs plus a database server as well as a client VM. The guide and scripts are also useful when building from scratch. Sign up at the SPEC virt Forum, then review the first
topic for instructions on downloading the scripts and their accompany documentation.
NOTE: The versions of software referenced in the ExampleVM Setup Guide represent a known working combination of software and are used to build a working environment for example purposes only. SPEC does not endorse the software used in this example, and the fact that these versions are used in this document does not mean they would be compliant for submitting an official SPEC virt_sc result. Please consult the SPEC virt_sc
Run and Reporting Rules to check on software version compliance.
Q. How do I increase error logging during a test?
Q. How do I resolve the following Java connection problem:
java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:Q. I've completed a run that passes all validation tests and would like to submit the result, but some (or all) of the testbed information in the disclosure report .html needs to be updated. How do I regenerate updated/corrected .sub and .html files for submission?
A. The recommended way to do this is to update all of the information in the Testbed.config file (located in the /opt/SPECvirt directory). Then, copy and insert the contents of Testbed.config into the .raw file at the line just above "END SPEC virt_sc". Finally, regenerate the .sub file with the usual syntax:
Q. What version of Java do I need?
A. You need to run a supported version of Java 6 or greater Compatible software. The clients, appservers, and batchservers require Java 6 or greater JDK. All other VMs require Java 6 or greater JRE.
Q. How long does a compliant test run?
A. A compliant test must run for 7200 seconds. If POLL_INTERVAL_SEC in Control.config is greater than 7200 seconds, the test continues to run, but the run is non-compliant. At the end of the test, the harness reports a compliance error:
* Compliance Errors:
o POLL_INTERVAL_SEC value is: 3600; should be 7200 or greater.
Q. How do I fix the following webserver SSL or TLS errors:
2015-05-06 09:25:51:414 SPECweb_Support: [ERROR] STATE 0; makeHttpRequest() failed.
2015-05-06 09:25:51:475 SslConnection: [ERROR] IOException during SSL handshake: javax.net.ssl.SSLHandshake
Exception: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
...
2015-05-06 09:26:16:544 SslConnection: [ERROR] IOException during SSL handshake: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
2015-05-06 09:26:22:535 Connection: [ERROR] Write to socket failed! IOException was: javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
A. SSLv3 is disabled by default starting with JDK 8u31 and JDK 7u75. Refer to this Oracle note for details.
To enable SSLv3, log into the client and:
Q. If I want to use TLS, which cipher suites do you recommend?
A. Many Java 1.7 JRE versions do not support TLS_RSA_WITH_AES_128_GCM_SHA256. We found that the following two cipher suites perform similarly to SSLv3 and are available options for Java 1.7 JREs:
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
Note that the CPU consumption on the SPECweb prime client driver will likely be noticeably higher. Please review the documentation for your version of Java 1.7 to verify which cipher suites are supported. Also, some cipher suites must be enabled explicitly in the JRE settings.
Alternatively, Java 1.8 offers a larger range of TLS cipher suites (namely, the TLS GCM varieties). If Java 1.7 cipher options are too limited, you may want to consider upgrading to Java 1.8 to enable and experiment with the additional cipher options.
Q. How do I fix webserver validation errors?
A. This can occur if there are Besim processes (e.g. besim_fcgi.fcgi) on the infraserver VM that retain the data from a prior run. This is most likely to occur when a test is aborted and restarted but the httpd and associate Besim daemons are not restarted on the infraserver VMs. To avoid this condition, delete the old besim_support.globals file and restart the httpd on the infraserver:
ssh infraserver /etc/init.d/httpd stop
ssh infraserver rm -f /tmp/besim_support.globals
ssh infraserver /etc/init.d/httpd start
You also can configure the harness to run an initialization script that contains these commands. For example, the Control.config contains: PRIME_HOST_INIT_SCRIPT[1] = "webInit.sh" and the script webInit.sh would include the commands above.
Q. When running webserver, the test seems to be running and possibly gives one or two lines of polling data when the test ends. Examination of Clientmgr logs shows one or more SPECweb tests have large numbers of errors similar to:
Messages from: 10.17.181.118:1191
-> 2009-10-02 10:06:43:115 Creating specwebclient using RMI Registry port 1191
-> 2009-10-02 10:06:43:144 perf2.company.com:1191 ready...
-> 2009-10-02 10:09:04:082 LoadGenerator: Setting sleep delay.
-> 2009-10-02 10:09:07:373 specwebclient: setting tile ID to 1
-> 2009-10-02 10:09:08:383 LoadGenerator: Ramping up for 60 seconds.
-> 2009-10-02 10:09:08:383 LoadGenerator: Starting 500 threads.
-> 2009-10-02 10:10:08:385 LoadGenerator: All threads running, test starting
-> 2009-10-02 10:10:59:731 Connection: SocketTimeoutException waiting for end-of-header
-> 2009-10-02 10:10:59:731 SPECweb_Support: [ERROR] SocketTimeoutException encountered during run!
-> 2009-10-02 10:11:01:264 SPECweb_Support: [ERROR] Invalid title tag, expecting 'SPECweb2005: Support Category' but got 'SPECweb2005: Support Home' was found
-> 2009-10-02 10:11:01:264 SPECweb_Support: [ERROR] Expected 13 catalog listings, 7 found.
-> 2009-10-02 10:11:01:264 SPECweb_Support: [ERROR] STATE 2; Expected 27500 bytes appended to response but found 51700
-> 2009-10-02 10:11:01:264 SPECweb_Support: [ERROR] STATE 2; RESPONSE INVALID on GET /support/catalog.php?id=3936 HTTP/1.1
-> 2009-10-02 10:11:01:264 SPECweb_Support: [ERROR] Response was:
-> http/1.1 200 ok
A. Check the open file limit on the clients:
ssh client2 "ulimit -n"
If you get a number such as 1024, then the ulimit has not been set. Check the /etc/sysctl.conf file to make sure that you have a line similar to:
fs.file-max = 500000
Then check /etc/bashrc to make sure the following is included:
Then reboot the affected client(s) and rerun the test.
Q. How do I resolve HttpRequestSched errors?
A. Closely look at the network tunings for your VMs, especially for the workload that is reporting failures. The webserver does the largest amount of network traffic with the client and is usually the first to require network tuning. Looking at the published results for applied tunings in those configurations provides a good idea of where and how to start tuning.
Q. Webserver fails, and examination of Clientmgr logs shows errors similar to:
Messages from: 10.17.181.118:1191
-> 2014-07-29 15:11:52:602 SPECweb_Support: [ERROR] STATE 4; makeHttpRequest() failed.
-> 2014-07-29 15:11:52:602 SPECweb_Support: [ERROR] STATE 4; makeHttpRequest() failed.
-> 2014-07-29 15:11:50:603 SslConnection: [ERROR] IOException creating socket. Exception is:
-> java.net.ConnectException: Connection refused
-> hostServer is: webserver; port is: 443
A. Issue the following commands on the webserver:
netstat –ant|grep 443
ps –ef|grep httpd
If the netstat command returns nothing and the ps -ef command shows no httpd processes running, then you need to enable SSL or TLS. Shut down httpd and add to /etc/httpd/conf/httpd.conf:
Include conf.d/ssl.conf
Add to /etc/httpd/conf.d/ssl.conf:
ServerName webserver:443
Then restart httpd.
Q. What parameters do I use to configure open source databases to meet Atomicity, Consistency, Isolation, and Durability (ACID) requirements as described in the SPEC virt_sc Run and Reporting Rules?
A. For PostgreSQL, fsync=on and synchronous_commit=on are required. For MySQL and MariaDB, innodb_flush_log_at_trx_commit=1 is required. Depending on your configuration and products used, there may be other parameters needed as well. Please consult with your product's documentation on strict ACID configuration.
Q. How do I resolve FAILED transaction validation errors such as:
Example 1
0-0-0 = Work Order Transaction validation FAILED
0-0-1 = Largeorder Vehicle Purchase Rate FAILED
0-0-2 = Regular Vehicle Purchase Rate FAILED
Example 2 (from appserver audit report):
Work Order Transaction validationExample 3 (from html results file):
Validation Errors:A. This usually means that the application server VM and client are configured properly but require tuning. Try tuning heap size and the network between the clients and VM. Review recent
submissions and download the accompanying supporting documentation archive (TGZ) for application and database server settings.
In the case of GlassFish, try tuning the following domain.xml parameters:
To find a sample domain.xml file, extract the archive and look under the Virtualization folder (VIRT) and select any tile. Then navigate through the appserver folder to software (SOFTW) then spec2004-01 then config where domain.xml resides.
Q. How do I fix the Unsuccessful switchlog appserver error: Messages from: 10.17.181.114:1091
-> 2010-08-18 15:58:16:893 Creating jappclient using RMI Registry port 1091
-> 2010-08-18 15:58:16:931 perf2.company.com:1091 ready...
-> Driver Host: perf2company.com Tile Number:0
-> 2010-08-18 16:02:19:457 matchOut() messages set ...
-> 2010-08-18 16:02:19:458 Starting rmiregistry; bindWait = 45000
-> 2010-08-18 16:03:04:466 Starting Controller; bindWait = 45000
-> 2010-08-18 16:03:04:473 waiting for: Binding controller to /
-> rec'd notifyInterrupt(101) call
-> Binding controller to //perf2company.com:2098/Controller
-> Launcher: done in waitMatch(0)
-> 2010-08-18 16:03:49:611 Starting Agents
-> ---------------//perf2.company.com:2098/Controller
-> Calling switchLog as master
-> url[0] is : http://specemulator:8080/Emulator/EmulatorServlet?cmd=switchlog
-> url[1] is : http://specdelivery:8000/Supplier/DeliveryServlet?cmd=switchlog
-> 2010-08-18 16:04:34:775 Exception encountered executing runScript(): java.io. IOException: Unsuccessful switchlog
-> java.io.IOException: Unsuccessful switchlog
-> at org.spec.jappserver.launcher.Driver.switchLog(Driver.java:705)
-> at org.spec.jappserver.launcher.Driver.runScript(Driver.java:531)
-> at org.spec.jappserver.launcher.jappclient.setup(jappclient.java:146)
-> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
A. If the following steps are successful, it is very likely that the AppServer VM and the Database VM are configured properly. The switchlog or any other client side error could be a misconfiguration on client(s):
Verify that emulator is accessible. Otherwise check the server log where emulator application is deployed:
Verify that SPECjAppServer application is available on AppServer:
For any other response span style="font-family: Courier New,Courier,monospace;", such as 400 Error, check AppServer server log and fix the issue. For example:
Change their permissions:
Verify that database is properly configured and accessible by the SPECjAppServer application on AppServer:
The resulting text should show PASSED for the three Atomicity tests.
Q. How do I fix the NullPointerException appserver error in Clientmgrn_1088.out:
-> Error: Connection: UnknownHostException?: specdelivery
-> java.lang.NullPointerException?
-> at org.spec.jappserver.driver.http.HttpDealer?.login(HttpDealer?.java:651)
-> at org.spec.jappserver.driver.DealerEntry?.logIn(DealerEntry?.java:1193)
-> at org.spec.jappserver.driver.DealerEntry?.loginCtl(DealerEntry?.java:1134)
-> at org.spec.jappserver.driver.DealerEntry?.doBrowse(DealerEntry?.java:1046)
-> at org.spec.jappserver.driver.DealerEntry?.run(DealerEntry?.java:256)
If you get a number such as 1024, then the ulimit has not been set. Check the /etc/sysctl.conf file to make sure that you have a line similar to:
Then check /etc/bashrc to make sure the following is included:
Then reboot the affected client(s) and rerun the test.
Q. How do I build SPECjAppServer.ear using GlassFish?
A. Building SPECjAppServer.ear for any application server requires vendor-specific deployment descriptors. For GlassFish, build_glassfish.xml is the modified version of the generic build.xml to include GlassFish deployment descriptors from src/deploy/glassfish. To build SPECjAppServer.ear with GlassFish, update config/glassfish.env as per your environment and run ant from the SPECjAppServer2004 directory as follows:
Q. How do I resolve continued IMAP timeout errors during the course of mailserver runs:
A. You can create a "warmed up" mailstore. This is a one-time task. Run a two- or three-hour mailserver test and ignore the IMAP timeout errors during the first 20 or 30 minutes. When the test stops, shutdown the mailserver software and use tar to create an archive of the warmed-up mailstore for subsequent test runs. For example, if you use Dovecot:
Then restore the mailstore using this archive between test runs.
Q. How to I restore the mailstore between test runs?
A. Use a mailserver initialization script in Control.config to restore the mailstore. Using the example in the previous question, after you back up the mailstore archive to /root/BACKUPS/dovecot_500users_warmed.tgz on the mailserver, the mailserver init script that Control.config calls on the prime client might contain:
Q. How do I resolve the following repetitive validation errors:
UID_STORE_NUM_UNSET_FLAGS_SEEN
Mix
Pct Failed. Expected: 30.000, Achieved: 34.912
FETCH_NUM_RFC822HEADER Mix Pct Failed. Expected: 515.000, Achieved: 627.550
A. Restore the mailstore to its original state and rerun.
Q. Does the mailstore needed to be restored to its initial state after each benchmark run?
A. It is optional whether or not you restore the mailstore between test runs. Some submissions restored the mailstore before starting the test, while others didn't.
Q. How do I resolve the following message count errors:
2013-07-02 00:04:59:167
LoadGenerator: Threads started, test in progress...
test1 Expected 1969 messages, found 2002
test1 FATAL ERROR - Aborting run, and EXITING.
test11 Expected 1969 messages, found 2003
test11 FATAL ERROR - Aborting run, and EXITING.
A. Finding more messages than expected suggests that the mailstore needs to be reloaded or restored from a previously warmed up backup, if available.
Q. How do I resolve the following mailstore loading errors:
BAD Error in IMAP command APPEND:
Invalid arguments
A. This usually occurs when the SPECimap/IMAP*.config files are in DOS format instead of UNIX format. Edit the config files with vi and do the following to convert to UNIX format:
Q. How do I submit results?
A. Information on how and when to submit SPEC virt_sc results is available at http://www.spec.org/osg/submitting_results.html. Specific SPE Cvirt_sc information on review cycle scheduling is at http://www.spec.org/osg/calendar/2014_osg-virt.html. Additional details on the SPEC virt_sc submission and review process are available at:
SPEC does not post the email aliases for submissions or the ftp upload information. For this information, the Benchmarker must contact info@spec.org. Benchmark license numbers may be requested.
Note that benchmarkers must follow all Run and Reporting Rules when testing and generating results for submission and publication. Benchmarkers can submit anytime prior to the weekly review cycle deadline as the results are held in a locked directory. Benchmarkers can request that results can be withdrawn during the review cycle. Minor updates such as adding missing information or making editorial corrections are allowed, but the expectation is that the submission packet is essentially complete as detailed in the Run and Reporting Rules.
If you haven't solved your problem with the information above and need to contact SPEC virt_sc Support, below is a list of the information to collect, review, and submit with your problem report:
You should also consider using an example collection script from a published archive TGZ as a template to collect potentially useful debugging information. Include all key files and logs with your problem report to give SPEC virt_sc technical support volunteers the best chance to help you resolve your problem.