« previous next »
Pages: [1] 2 3 4

Author Topic: Run the benchmark failed  (Read 35981 times)

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Run the benchmark failed
« on: October 27, 2015, 09:37:51 PM »
 run runspecvirt.sh ....

0,0,2015-10-28 09:17:13:086,0,0.0,2,10.0,0,0.0,3853,25.00
0,1,2015-10-28 09:17:13:078,1289441,1289441,0,0,589479289457,5934284695,1289330,89,22,263,391940,0
0,2,2015-10-28 09:17:13:081,912576,912576,0,0,26350344,0,2938
0,3,2015-10-28 09:17:13:073,1,719,1,0,24,2120,719,0

2015-10-28 09:17:23:071 PrimeControl: stopping result polling.
2015-10-28 09:17:23:071 PrimeControl: waiting for all workloads to stop...
2015-10-28 09:17:33:071 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:17:48:072 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:18:03:073 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:18:18:073 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:18:33:073 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:18:48:073 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:19:03:074 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:19:18:074 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:19:33:074 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:19:48:075 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:20:03:075 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:20:18:075 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:20:33:075 PrimeControl: waiting for 2 masters to finish
2015-10-28 09:20:48:076 PrimeControl: waiting for 1 masters to finish
2015-10-28 09:21:03:076 PrimeControl: waiting for 1 masters to finish
2015-10-28 09:21:18:077 PrimeControl: waiting for 1 masters to finish
2015-10-28 09:21:33:077 PrimeControl: waiting for 1 masters to finish
2015-10-28 09:21:48:078 PrimeControl: waiting for 1 masters to finish
2015-10-28 09:22:03:078 PrimeControl: waiting for 1 masters to finish
2015-10-28 09:22:11:233 PrimeControl: all workloads stopped
2015-10-28 09:22:11:304 Workload validation errors reported!:
0-0-0 = Error reading file '/opt/SPECjAppServer2004/output/4/Atomicity.html'
0-0-1 = Dealer Purchase Transaction Mix: FAILED
0-0-2 = Dealer Manage Transaction Mix: FAILED
0-0-3 = Dealer Browse Transaction Mix: FAILED
0-0-4 = Dealer 90% Response Time FAILED
0-0-5 = Dealer Avg. Response Time FAILED
0-0-6 = Manage Cycle Time Avg. FAILED
0-0-7 = LargeOrderLine Vehicle Rate FAILED
0-0-8 = PlannedLines Vehicle Rate FAILED
0-0-9 = Manufacturing 90% Response Time FAILED
0-0-10 = Manufacturing Avg. Response Time FAILED
2015-10-28 09:22:11:310 PrimeControl: aggregate audit...
2015-10-28 09:22:11:311 PrimeControl: aggregate audit...
2015-10-28 09:22:11:316 PrimeControl: aggregate audit...
2015-10-28 09:22:11:377 PrimeControl: aggregate audit...
2015-10-28 09:22:11:377 PrimeControl: validating aggregate audit...
2015-10-28 09:22:13:529 PrimeControl: stopping clients.
2015-10-28 09:22:13:529 PrimeControl: stopping remote client processes
2015-10-28 09:22:15:586 PrimeControl: stopping local client threads
  > Loading Raw Result File..

2015-10-28 09:22:15:722 PrimeControl: terminating run. Please wait...
2015-10-28 09:22:16:727 specvirt: Done!
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #1 on: October 27, 2015, 10:46:58 PM »
result
Logged

lroderic

  • Moderator
  • Full Member
  • *****
  • Posts: 167
  • Karma: +6/-0
Re: Run the benchmark failed
« Reply #2 on: October 28, 2015, 10:33:11 AM »
Hello. You can see that ap/dbserver isn't working with the 25.00 second response time:

  0,0,2015-10-28 09:17:13:086,0,0.0,2,10.0,0,0.0,3853,25.00

Please set NUM_WORKLOADS=1 so that you're running only the app/dbserver to help troubleshoot. Also if you haven't done this already, please set DEBUG_LEVEL=2. Then post Clientmgr1_1088.out, Clientmgr1_1098.out, and primectrl.out. The perf.html doesn't help us at this point.
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #3 on: October 28, 2015, 10:43:02 AM »
[root@client1 SPECvirt]# cat Control.config|grep -A10 ^NUM_WORKLO
NUM_WORKLOADS = 1

WORKLOAD_LABEL[0] = "Application Server"
WORKLOAD_LABEL[1] = "Web Server"
WORKLOAD_LABEL[2] = "Mail Server"
WORKLOAD_LABEL[3] = "Batch Server"

BACKEND_VM_LABEL[0] = "Database Server"
BACKEND_VM_LABEL[1] = "Infrastructure Server"

# POLL_MASTERS controls whether or not to request polling data from the prime
[root@client1 SPECvirt]# cat Control.config|grep ^DEBUG
DEBUG_LEVEL = 2
[root@client1 SPECvirt]#

[root@client1 SPECvirt]# tail -20f primectrl.out
2015-10-28 22:40:42:476 Wed Oct 28 22:40:42 CST 2015
2015-10-28 22:40:42:480 Validator: [WARNING] NUM_WORKLOADS value is: 1; should be 4
2015-10-28 22:40:42:481 Validator: [WARNING] Non-compliant configuration.
2015-10-28 22:40:42:481 [WARNING] This will be a non-compliant benchmark result!
2015-10-28 22:40:42:543 RMI server started: client1:9990
2015-10-28 22:40:42:558 [INFO] This is a perf-only benchmark run. Skipping active idle polling interval.
2015-10-28 22:40:42:559 PrimeControl: preparing client drivers.
2015-10-28 22:40:42:619 PrimeControl: starting clients...
2015-10-28 22:40:42:632 PrimeControl: PTDs not used for this benchmark run!
2015-10-28 22:40:42:633 PrimeControl: starting 1 masters.
2015-10-28 22:42:23:883 PrimeControl: waiting on 1 prime client(s).
.
2015-10-28 22:42:24:903 setting hostsReady = true
2015-10-28 22:44:43:244 PrimeControl: Workload and prime controller builds: 79
2015-10-28 22:44:43:244 PrimeControl: awaiting runtime started signal from prime clients
2015-10-28 22:44:43:246 PrimeControl: all workloads started.
2015-10-28 22:45:12:683 specvirt: clock sync check completed successfully
2015-10-28 22:45:12:683 specvirt: initiating workload ramp-up.
2015-10-28 22:45:12:684 Polling start time = Wed Oct 28 23:05:13 CST 2015
2015-10-28 22:45:12:684 Polling end time   = Thu Oct 29 01:05:13 CST 2015
« Last Edit: October 28, 2015, 10:48:59 AM by onlyloveu2 »
Logged

lroderic

  • Moderator
  • Full Member
  • *****
  • Posts: 167
  • Karma: +6/-0
Re: Run the benchmark failed
« Reply #4 on: October 28, 2015, 01:57:55 PM »
Is that all?
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #5 on: October 28, 2015, 01:59:33 PM »
this is all
Logged

lroderic

  • Moderator
  • Full Member
  • *****
  • Posts: 167
  • Karma: +6/-0
Re: Run the benchmark failed
« Reply #6 on: October 28, 2015, 02:04:13 PM »
I can't read that 7z file. Would you please use another tool or just post the contents of the log files?
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #7 on: October 28, 2015, 02:19:40 PM »
zip
Logged

lroderic

  • Moderator
  • Full Member
  • *****
  • Posts: 167
  • Karma: +6/-0
Re: Run the benchmark failed
« Reply #8 on: October 28, 2015, 02:40:40 PM »
As we continue to debug, in Control.config please set POLL_INTERVAL_SEC = 900 so that you don't have to wait an hour as well as WARMUP_SECONDS= 300.

Check the open file limit on the client:

# ulimit -n

If you get a number such as 1024, then the ulimit hasn't been set. Check the /etc/sysctl.conf file to make sure that you have a line like:

fs.file-max = 500000

Then check /etc/bashrc to see if it has near the top:

# overide open files limit of 1024
ulimit -n 100000 > /dev/null 2>&1

Then reboot the client and rerun.
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #9 on: October 28, 2015, 06:36:45 PM »
[root@client1 SPECvirt]# cat Control.config|grep ^WARMUP_SEC
WARMUP_SECONDS= 900
[root@client1 SPECvirt]# cat Control.config|grep ^POLL_INT
POLL_INTERVAL_SEC = 900
[root@client1 SPECvirt]# uptime
 06:35:41 up 1 min,  1 user,  load average: 0.04, 0.02, 0.00
[root@client1 SPECvirt]# ulimit -n
500000
[root@client1 SPECvirt]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
fs.file-max = 1000000
vm.swappiness = 0
vm.overcommit_memory = 0
net.core.optmem_max = 20000000
net.core.rmem_default = 20000000
net.core.rmem_max = 20000000
net.core.wmem_default = 20000000
net.core.wmem_max = 20000000
net.core.somaxconn = 8192
net.ipv4.tcp_max_tw_buckets = 500000
net.ipv4.tcp_mem = 20000000 20000000 200000000
net.ipv4.tcp_rmem = 20000000 20000000 200000000
net.ipv4.tcp_wmem = 20000000 20000000 200000000
net.ipv4.ip_local_port_range = 4096 65535
net.ipv4.tcp_tw_reuse = 1
[root@client1 SPECvirt]#  tail -6 /etc/security/limits.conf
*       hard    nofile          500000
*       soft    nofile          500000
*       hard    nproc          20000
*       soft    nproc          20000


[root@client1 SPECvirt]#
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #10 on: October 28, 2015, 09:19:28 PM »
without html output.

Logged

lroderic

  • Moderator
  • Full Member
  • *****
  • Posts: 167
  • Karma: +6/-0
Re: Run the benchmark failed
« Reply #11 on: October 29, 2015, 10:16:24 AM »
From the Technical Support doc, log into the appserver and verify that emulator is accessible. Otherwise check the server log where emulator application is deployed:

# curl http://specemulator:8080/Emulator/EmulatorServlet?cmd=switchlog
200 OK

Verify that SPECjAppServer application is available on AppServer:

# curl http://specdelivery:8000/Supplier/DeliveryServlet?cmd=switchlog
200 OK

Verify that database is properly configured and accessible by the SPECjAppServer application on AppServer:

# curl http://specdelivery:8000/SPECjAppServer/app?action=atomicityTests

The resulting text should show PASSED for the three Atomicity tests.

In /etc/hosts, does appserver have the host aliases specemulator and specdelivery? That is, on the client and appserver, do you have something like:

192.168.1.4 appserver appserver1 specdelivery specemulator

Also, check ulimit on all VMs, not just the client.
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #12 on: October 29, 2015, 10:24:04 AM »
all VMs ulimit -n set 500000 ?
Logged

lroderic

  • Moderator
  • Full Member
  • *****
  • Posts: 167
  • Karma: +6/-0
Re: Run the benchmark failed
« Reply #13 on: October 29, 2015, 10:28:30 AM »
Yes. It won't hurt.
Logged

onlyloveu2

  • Newbie
  • *
  • Posts: 41
  • Karma: +0/-0
Re: Run the benchmark failed
« Reply #14 on: October 30, 2015, 06:10:27 AM »
After resize  dbserver memory to 32G, the problem (can not start glassfish service) solved.

rerun runspecvirt.sh meet a new problem again.


Logged
Pages: [1] 2 3 4
« previous next »