SPECjAppServer2004 Result: Oracle WebLogic Server Standard Edition Release 10.3.3 on Cisco UCS B230 M1 Blade Servers

SPECjAppServer®2004 Result
Copyright © 2004 Standard Performance Evaluation Corporation

Oracle WebLogic Server Standard Edition Release 10.3.3 on Cisco UCS B230 M1 Blade Servers

11,283.80 SPECjAppServer2004 JOPS@Standard

Submitter:
Cisco Systems, Inc.

SPEC license # 9019

Test date:
Aug-2010

Software Products
Oracle WebLogic Server Standard Edition Release 10.3.3
Oracle JRockit(R) 6.0 JDK (R28.0.1) (Linux x86 64bit)
Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Oracle JDBC Driver 11.2.0.1.0 (Thin)
Oracle Database Enterprise Edition Release 11.2.0.1.0

Software Configurations
J2EE Application Server
Emulator SW Config
Database SW Config
Driver SW Config Primary and Satellites

Hardware Systems
J2EE Application Server
Database Server HW
Primary Driver, Satellite Drivers & Emulator HW

System Configuration Diagram

Benchmark
Modifications
Configuration
Bill of Materials
Other Info
General Notes
Full Disclosure Archive

SUT Configuration

J2EE Server Nodes:	2	DB Server Nodes:	1
J2EE Server CPUs:	32 cores, 4 chips	DB Server CPUs:	32 cores, 4 chips
J2EE Instances:	16	DB Instances:	1

Other SUT Components:

Benchmark Settings

Dealer Injection Rate:	6,750	Ramp Up Time:	1,200 seconds
# of Dealer Agents:	30	Ramp Down Time:	300 seconds
# of Manufacturing Agents:	24	Steady State Time:	3,600 seconds
DB Load Injection Rate:	7,000	Trigger Time:	1,200 seconds

Detailed Results

Dealer Transaction Mix	Count	Tx. Mix
Purchase	6,097,245	24.99%
Manage	6,097,420	24.99%
Browse	12,202,982	50.02%
Total # Dealer Transactions	24,397,647

Total # Manufacturing Transactions	16,224,042

Response Times	Average	Maximum	90th%	Reqd
Purchase	0.37	15.98	0.90	2.00
Manage	0.27	17.53	0.60	2.00
Browse	0.47	12.67	1.10	2.00
Manufacturing	2.32	11.32	4.00	5.00

Reproducibility Run (SPECjAppServer2004 JOPS)	11,284.19

Oracle WebLogic Server Standard Edition Release 10.3.3

Vendor:	Oracle Corporation	Availability:	Apr-2010
CTS Version:	5.0	Date Passed CTS:	Feb-2007
OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64
EJB Protocol:	RMI/T3

Oracle JRockit(R) 6.0 JDK (R28.0.1) (Linux x86 64bit)

Vendor:	Oracle Corporation	Availability:	Jul-2010
OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64

Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21

Vendor:	Oracle Corporation	Availability:	Jul-2010
OS Name:	Red Hat Enterprise Linux 5 Update 5 x86_64

Oracle JDBC Driver 11.2.0.1.0 (Thin)

Vendor:	Oracle Corporation	Availability:	Oct-2009
OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64

Oracle Database Enterprise Edition Release 11.2.0.1.0

Vendor:	Oracle Corporation	Availability:	Oct-2009
OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64

J2EE Application Server

J2EE Product:	Oracle WebLogic Server Standard Edition Release 10.3.3
JVM Product:	Oracle JRockit(R) 6.0 JDK (R28.0.1) (Linux x86 64bit)
JDBC Product:	Oracle JDBC Driver 11.2.0.1.0 (Thin)
Hardware:	J2EE Application Server
# of Instances:	16
Interfaces:
	Web Interface for the Dealer Domain
	Web Interface for the Supplier Domain
	EJB Interface for the Dealer Domain
	EJB Interface for the Manufacturing Domain
	EJB Interface for the Supplier Domain

Notes / Tuning Information

-Xms6400m -Xmx6400m -Xns3360m -XXaggressive -Xgc:genpar -Xlargepages
-Xverbose:gcpause,opt,gcreport -showversion -showversion
-Doracle.jdbc.defaultRowPrefetch=200
-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=1
-Dweblogic.management.discover=false
-Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

Emulator SW Config

JVM Product:	Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Web Product:	Oracle WebLogic Server Standard Edition Release 10.3.3
Hardware:	Primary Driver, Satellite Drivers & Emulator HW
# of Instances:	1

Notes / Tuning Information

-Xmx22g -Xms22g -Xmn7g -Xss272k -XX:PermSize=95m -XX:MaxPermSize=512M
-XX:+UseLargePages -XX:+AggressiveOpts -XX:+DisableExplicitGC
-verbosegc -Xloggc:emugc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintCommandLineFlags
-Dsun.net.inetaddr.ttl=1 -Dnetworkaddress.cache.ttl=1 -Dweblogic.SocketReaders=1
-Dweblogic.management.discover=false
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

Database SW Config

DB Product:	Oracle Database Enterprise Edition Release 11.2.0.1.0
Hardware:	Database Server HW
# of Instances:	1

Notes / Tuning Information

db_name                 = spec
compatible              = 11.2.0.1.0
control_files           = ?/dbs/cntrlspec
aq_tm_processes         = 0
db_4k_cache_size        = 12000M
db_8k_cache_size        = 20000M
db_block_checking       = FALSE
db_block_checksum       = FALSE
db_block_size           = 2048
db_cache_advice         = OFF
db_cache_size           = 45000M
db_file_multiblock_read_count = 128
db_files                = 256
db_keep_cache_size      = 10000M
db_writer_processes     = 8
dml_locks               = 2200
filesystemio_options    = DirectIO
log_buffer              = 67108864
log_checkpoint_interval = 0
log_checkpoints_to_alert= TRUE
open_cursors            = 3000
parallel_max_servers    = 100
pga_aggregate_target    = 0
processes               = 3000
query_rewrite_enabled   = FALSE
replication_dependency_tracking = FALSE
session_cached_cursors  = 3000
sessions                = 3000
shared_pool_size        = 6000M
statistics_level        = BASIC
timed_statistics        = FALSE
trace_enabled           = FALSE
transactions            = 15000
transactions_per_rollback_segment = 1
undo_management         = AUTO
undo_retention          = 300
undo_tablespace         = undo_ts

Driver SW Config Primary and Satellites

JVM Product:	Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Hardware:	Primary Driver, Satellite Drivers & Emulator HW
# of Instances:	4

Notes / Tuning Information
-server -Xms2048M -Xmx2048M -XX:+DisableExplicitGC -XX:+UseParallelGC -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0

J2EE Application Server (SUT hardware)

Hardware Vendor:	Cisco	OS Vendor:	Oracle Corporation
Model Name:	Cisco UCS B230 M1 Blade Server	OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64
Processor:	Eight Core Intel(R) Xeon(R) X7560	Filesystem:	ext3
MHz:	2262	Disks:	2x 64GB 3Gb/s SATA SSD
# of CPUs:	16 cores, 2 chips, 8 cores/chip, 2 threads/core (Hyper-Threading)	Network Interface:	Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter with 2 x 10 Gigabit Ethernet and 2 x 4Gbps Fiber Channel
Memory (MB):	131072	Other Hardware:	See notes
L1 Cache:	32KB(I)+32KB(D) on chip per core	# of Systems:	2
L2 Cache:	256KB on chip per core	H/W Available:	Oct-2010
Other Cache:	24MB L3 on chip per chip	OS Available:	Apr-2010

Notes / Tuning Information

Added to /etc/sysctl.conf:
fs.file-max = 6815744
kernel.sem = 250 32000 100 142
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.netdev_max_backlog = 400000
net.core.optmem_max = 30000000
net.core.rmem_default = 30000000
net.core.rmem_max = 30000000
net.core.somaxconn = 40000
net.core.wmem_default = 30000000
net.core.wmem_max = 30000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 30000000 30000000 30000000
vm.zone_reclaim_mode=1

For each of the NICs:
set arp_announce = 2
set arp_ignore = 1
set txqueuelen 240000
enable tso and gso
bind interrupt to a core corresponding to the server instance

Server instances were started using numactl, binding 4 instances per chip.
Stop iptables.
Stop irqbalance.
Configure 32000 hugepages.

Other hardware:
EMC CLARiiON CX4-240
VAULT PACK 450GB 15k 4G Drives
EMC 450 GB 15K 4G FC HDD

System was configured with 32 drives in 2 x (8+8) RAID1/0
for jms and server logs.
2 drives configured as spares.

Database Server HW (SUT hardware)

Hardware Vendor:	Cisco	OS Vendor:	Oracle Corporation
Model Name:	Cisco UCS B440 M1 Blade Server	OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64
Processor:	Eight Core Intel(R) Xeon(R) X7560	Filesystem:	ext3
MHz:	2262	Disks:	4x 300GB SAS 10K RPM
# of CPUs:	32 cores, 4 chips, 8 cores/chip, 2 threads/core (Hyper-Threading)	Network Interface:	Cisco UCS M71KR-Q QLogic Converged Network Adapter with 2 x 10 Gigabit Ethernet and 2 x 4Gbps Fiber Channel
Memory (MB):	262144	Other Hardware:	See notes
L1 Cache:	32KB(I)+32KB(D) on chip per core	# of Systems:	1
L2 Cache:	256KB on chip per core	H/W Available:	Jun-2010
Other Cache:	24MB L3 on chip per chip	OS Available:	Apr-2010

Notes / Tuning Information

Added to /etc/sysctl.conf:
fs.aio-max-nr= 3145728
fs.file-max = 6815744
kernel.msgmni = 2878
kernel.sem = 250 32000 100 142
kernel.shmmax = 4398046511104
net.core.netdev_max_backlog = 400000
net.core.rmem_default = 30000000
net.core.rmem_max = 30000000
net.core.somaxconn = 15000
net.core.wmem_default = 30000000
net.core.wmem_max = 30000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_max_syn_backlog = 12000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 30000000 30000000 30000000

For each of the NICs used:
set arp_announce = 2
set arp_ignore = 1
set txqueuelen 240000
enable tso and gso

Stop iptables.
Stop irqbalance.
Configure 55000 hugepages.

Lgwr nice priority was set to -20
named nice priority was set to -10
8 Oracle listener processes were started and bound each to 8 Hardware Threads.

Other hardware:
EMC CLARiiON CX-480
VAULT PACK 450GB 15k 4G Drives
EMC 450 GB 15K 4G FC HDD

40 drives in 4 x (5+5) RAID1/0 for data
20 drives in 2 x (5+5) RAID1/0 for logs
2 drives configured as spares

Primary Driver, Satellite Drivers & Emulator HW (non-SUT hardware)

Hardware Vendor:	Cisco	OS Vendor:	Oracle Corporation
Model Name:	Cisco UCS B200 M2 Blade Server	OS Name:	Oracle Enterprise Linux 5 Update 5 x86_64
Processor:	Six Core Intel(R) Xeon(R) X5680	Filesystem:	ext3
MHz:	3333	Disks:	2x 300GB SAS 10K RPM
# of CPUs:	12 cores, 2 chips, 6 cores/chip, 2 threads/core (Hyper-Threading)	Network Interface:	Cisco UCS M71KR-Q QLogic Converged Network Adapter with 2 x 10 Gigabit Ethernet
Memory (MB):	98304	Other Hardware:	none
L1 Cache:	32KB(I)+32KB(D) on chip per core	# of Systems:	4
L2 Cache:	256KB on chip per core	H/W Available:	Apr-2010
Other Cache:	12MB(I+D) L3 on chip per chip	OS Available:	Apr-2010

Notes / Tuning Information

Added to /etc/sysctl.conf:
fs.file-max = 6815744
kernel.sem = 250 32000 100 142
net.core.netdev_max_backlog = 400000
net.core.optmem_max = 30000000
net.core.rmem_default = 30000000
net.core.rmem_max = 30000000
net.core.somaxconn = 40000
net.core.wmem_default = 30000000
net.core.wmem_max = 30000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 30000000 30000000 30000000

For each of the NICs used:
set arp_announce = 2
set arp_ignore = 1
set txqueuelen 240000
enable tso and gso

Stop iptables.

Benchmark Modifications
Schema Modifications: Tablespace sizes were increased to support the Injection rate. Table and index initrans were modified. The C_CUSTOMERINVENTORY, M_WORKORDER and O_ORDERLINE tables were horizontally partitioned. C_CUSTOMER, M_PARTS, M_BOM and M_INVENTORY were hash clustered. Table locks were disabled during the run for all the tables using ALTER TABLE DISABLE TABLE LOCK. Scripts to create the DB are included in the FDA.
Load Program Modifications: The load program was not modified.

Benchmark Modifications

Schema Modifications:

  Tablespace sizes were increased to support the Injection rate. Table and index
  initrans were modified. The C_CUSTOMERINVENTORY, M_WORKORDER and O_ORDERLINE tables were
  horizontally partitioned. C_CUSTOMER, M_PARTS, M_BOM and M_INVENTORY were hash clustered.
  Table locks were disabled during the run for all the tables using ALTER TABLE DISABLE TABLE LOCK.
  Scripts to create the DB are included in the FDA.

Load Program Modifications:

  The load program was not modified.

Benchmark Configuration Information
Isolation Requirement Info: The following Beans were deployed in READ_COMMITTED mode as per the benchmark requirement: Mfg - AssemblyEnt, BomEnt, ComponentEnt, LargeOrderEnt, PartEnt Orders - ItemEnt Supplier - SupplierCompEnt, SupplierEnt All other beans were deployed using pessimistic locking mode such that all selects including finders are issued with a FOR UPDATE clause in select statement: Corp - CustomerEnt, CustomerInventoryEnt Mfg - InventoryEnt, WorkOrderEnt Orders - OrderEnt, OrderLineEnt Supplier - POEnt, POLineEnt, SComponentEnt Util - SequenceEnt The ItemEnt bean was cached for 20 minute intervals by setting the following in bea-orders.xml deployment descriptor: concurrency-strategy to ReadOnly read-timeout-seconds to 1200
Durability Requirement Info: To ensure database durability, RAID 10 was used for the data files and redo logs. The application server used RAID 1 to provide durability for server logs and JMS persistence filestores. The RAID controller on the database and the application server machines were configured with Write-Back cache policy supported with battery backup unit on the controller.
Storage Requirement Info: An 85 min run at Injection Rate of 6700 increased storage by 14.35 GB. Extrapolating for 24 hrs we need 243 GB. The system is configured with over 5TB of durable storage.
Argument Passing Semantics: Oracle WebLogic Server Standard Edition uses pass-by-value as required by the EJB specification.

Bill of Materials
Supplier Description SKU # Qty -------- -------------------------------------------------------- ---------------- --- Application Server Cisco Cisco UCS B230 M1 Blade Server N20-B6730-1 2 Cisco Eight Core Intel(R) Xeon(R) Processor X7560 A01-X0200 4 24MB L3 cache, 2.26GHz, 130W, Turbo Cisco Cisco 8GB DDR3-1333MHz RDIMM/PC3-10600/2x4GB Kit A02-M308GB3-2 32 Cisco Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter N20-AQ0102 2 Intel Intel 64 gb SSD N20-D064SSD 2 EMC EMC CLARiiON CX4-240 CX4-240C 1 EMC VAULT PACK 450GB 15k 4G Drives V-CX4-45015K 1 EMC EMC 450 GB 15K 4G FC HDD CX-4G15-450 34 32 drives in 2 x (8+8) RAID1/0 for jms logs 2 drives configured as spares Database Server Cisco Cisco UCS B440 M1 Blade Server 1 Cisco Eight Core Intel(R) Xeon(R) Processor X7560 A01-X0200 4 24MB L3 cache, 2.26GHz, 130W, Turbo Cisco Cisco 16GB DDR3-1333MHz RDIMM/PC3-10600/2x8GB Kit A02-M316GB3-2 32 Cisco Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter N20-AQ0102 1 Cisco 300GB 6Gb SAS 10K RPM SFF HDD A03-D300GA2 4 EMC EMC CLARiiON CX-480 CX4-480C 1 EMC VAULT PACK 450GB 15k 4G Drives V-CX4-45015K 1 EMC EMC 450 GB 15K 4G FC HDD CX-4G15-450 62 40 drives in 4 x (5+5) RAID1/0 for data 20 drives in 2 x (5+5) RAID1/0 for logs 2 drives configured as spares Blade Server Enclosure Cisco UCS 5108 Blade Server Chassis N20-C6508 1 Cisco UCS 2104XP Fabric Extender/4 external 10Gb ports N20-I6584 2 Cisco UCS 6120XP 20-port Fabric Interconnect N10-S6100 2 Cisco 4-port 10 GE/4-port 4Gb FC/Expansion module N10-E0440 2 Cisco Cisco MDS 9134 Multilayer Fiber Switch DS-C9134AP-K9 1 Oracle Oracle Enterprise Linux Basic Support for 3 years 3 Oracle Oracle Database 11g Enterprise Edition, Per Processor Unlimited Users for 3 years 16* Oracle Partitioning, Per Processor, Unlimited Users for 3 years 16* Oracle Oracle Premium Support for 3 years 2 Oracle Oracle WebLogic Server Standard Edition Release 10.3, Per Processor for 3 years 16* Oracle Oracle Premium Support for 3 years 2 (* 16 = 0.50 * 32) Explanation: For the purposes of counting the number of processors which require licensing, an Intel multicore chip with "n" cores shall be determined by multiplying "n" cores by a factor of 0.50).

Bill of Materials

Supplier  Description                                                SKU #              Qty
--------  --------------------------------------------------------   ----------------   ---
Application Server
Cisco    Cisco UCS B230 M1 Blade Server                              N20-B6730-1          2
Cisco    Eight Core Intel(R) Xeon(R) Processor X7560                 A01-X0200            4
         24MB L3 cache, 2.26GHz, 130W, Turbo
Cisco    Cisco 8GB DDR3-1333MHz RDIMM/PC3-10600/2x4GB Kit            A02-M308GB3-2       32
Cisco    Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter      N20-AQ0102  2
Intel    Intel 64 gb SSD                                             N20-D064SSD          2
EMC      EMC CLARiiON CX4-240                                        CX4-240C             1
EMC      VAULT PACK 450GB 15k 4G Drives                              V-CX4-45015K         1
EMC      EMC 450 GB 15K 4G FC HDD                                    CX-4G15-450         34
         32 drives in 2 x (8+8) RAID1/0 for jms logs
         2 drives configured as spares


Database Server
Cisco    Cisco UCS B440 M1 Blade Server                                                   1
Cisco    Eight Core Intel(R) Xeon(R) Processor X7560                 A01-X0200            4
         24MB L3 cache, 2.26GHz, 130W, Turbo
Cisco    Cisco 16GB DDR3-1333MHz RDIMM/PC3-10600/2x8GB Kit           A02-M316GB3-2       32
Cisco    Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter      N20-AQ0102           1
Cisco    300GB 6Gb SAS 10K RPM SFF HDD                               A03-D300GA2          4
EMC      EMC CLARiiON CX-480                                         CX4-480C             1
EMC      VAULT PACK 450GB 15k 4G Drives                              V-CX4-45015K         1
EMC      EMC 450 GB 15K 4G FC HDD                                    CX-4G15-450         62
         40 drives in 4 x (5+5) RAID1/0 for data
         20 drives in 2 x (5+5) RAID1/0 for logs
         2 drives configured as spares

 

Blade Server Enclosure

Cisco     UCS 5108 Blade Server Chassis                             N20-C6508             1
Cisco     UCS 2104XP Fabric Extender/4 external 10Gb ports          N20-I6584             2
Cisco                  UCS 6120XP 20-port Fabric Interconnect       N10-S6100             2
Cisco                  4-port 10 GE/4-port 4Gb FC/Expansion module  N10-E0440             2
Cisco     Cisco MDS 9134 Multilayer Fiber Switch                    DS-C9134AP-K9         1

Oracle    Oracle Enterprise Linux Basic Support for 3 years                               3
Oracle    Oracle Database 11g Enterprise Edition, Per Processor
          Unlimited Users for 3 years                                                    16*
Oracle    Partitioning, Per Processor, Unlimited Users for 3 years                       16*
Oracle    Oracle Premium Support for 3 years                                              2
Oracle    Oracle WebLogic Server Standard Edition Release 10.3,
          Per Processor for 3 years                                                      16*
Oracle    Oracle Premium Support for 3 years                                              2



(* 16 = 0.50 * 32)  Explanation:  For the purposes of counting
the number of processors which require licensing, an Intel multicore chip with "n"
cores shall be determined by multiplying "n" cores by a factor of 0.50).

Other Benchmark Information
The xerces implementation version 2.8.1 from xml.apache.org was used as it comes packaged with Oracle Weblogic Server Standard Edition. Requests were directed to the Appserver instances using DNS round robin, with the database node acting as the DNS server.

Other Benchmark Information

  The xerces implementation version 2.8.1 from xml.apache.org was used as it comes
  packaged with Oracle Weblogic Server Standard Edition.
  
  Requests were directed to the Appserver instances using DNS round robin, with
  the database node acting as the DNS server.

General Notes

On the database, incremental checkpointing ensured that no buffer will remain
dirty (in the cache) for more than the allowed time.

The only errors in the driver error logs are the application errors generated by
this benchmark.

For questions about this result, please contact the submitter: Cisco Systems, Inc.
For other inquiries, please contact webmaster@spec.org

Benchmark run on Sat Aug 21 00:14:27 PDT 2010 by SPECjAppServer2004 v1.08
Result submitted on Wed Aug 25 03:33:03 EDT 2010
Report generated by SPECjAppServer2004 Reporter v1.03

First published at SPEC.org on 20-Sep-2010