SPECsfs97_R1.v3 Result

Exanet Inc. :	ExaStore EX200FC (High Performance Cluster)
SPECsfs97_R1.v3 =	143880 Ops/Sec (Overall Response Time = 1.47 msec)

Performance

Throughput (ops/sec)	Response (msec)
13765	0.6
27393	0.8
41151	1.0
55114	1.1
68874	1.3
82760	1.6
96577	1.7
110481	1.9
124327	2.5
138671	2.9
141382	3.2
143880	3.5

Configuration

Server Configuration and Availability
Vendor	Exanet Inc.
Hardware Available	Nov 2005
Software Available	Sep 2005
Date Tested	Dec 2005
SFS License Number	211
Licensee Locations	Hertzlia, Israel

CPU, Memory and Power

Model Name ExaStore EX200FC (High Performance Cluster)

Processor Intel(R) Xeon MP(TM) CPU 3.66GHz

# of Processors 8 cores, 8 chips, 1 core/chip (4 cores per node)

Primary Cache 8 KBD on chip

Secondary Cache 1024KB(I+D) on chip

Other Cache N/A

UPS APC SmartUPS 1500 2U

Other Hardware None

Memory Size 48 GB (24 GB per node)

NVRAM Size 52 GB (all node memory is UPS protected, and 4 GB in the 6998 Storage System)

NVRAM Type Node: UPS-protected DRAM, Storage Controller: Battery-backed RAM

NVRAM Description See notes

Server Software

OS Name and Version ExaOS V2.2

Other Software N/A

File System ExaFS

NFS version 3

Server Tuning

Buffer Cache Size Dynamic

# NFS Processes 16 (8 per node)

Fileset Size 1373.2 GB

Network Subsystem
Network Type	Jumbo Frame Gigabit Ethernet (MTU 9000)
Network Controller Desc.	Intel(R) PRO/1000 MT Quad Port Server Adapter
Number Networks	2 (N1 - Server, N2 - Cluster)
Number Network Controllers	4 (2 per node: 4 ports server, 4 ports cluster)
Protocol Type	UDP

Switch Type Nortel Baystack 5510 (N1)

Bridge Type N/A

Hub Type N/A

Other Network Hardware N/A

Disk Subsystem and Filesystems

Number Disk Controllers 2 (1 per node)

Number of Disks 148

Number of Filesystems 1 (F1)

File System Creation Ops default

File System Config 12 RAID-Groups of 12 disks each

Disk Controller IBM ServeRAID 8i SAS controller

# of Controller Type 1 (per node)

Number of Disks 2 (per node)

Disk Type Hitachi 36 GB 10K RPM

File Systems on Disks OS, write cache dump files

Special Config Notes N/A

Disk Controller Fibre Channel: QLogic Corp. QLA2342 Dual Port Fibre Channel Adapter

# of Controller Type 4

Number of Disks 144

Disk Type Seagate Cheetah ST 373453 FC 73GB 15K RPM

File Systems on Disks F1

Special Config Notes See notes

Load Generator (LG) Configuration
Number of Load Generators	16
Number of Processes per LG	32
Biod Max Read Setting	2
Biod Max Write Setting	2

LG Type LG1

LG Model SuperServer 5013C-T

Number and Type Processors 1x3.0 GHz Pentium 4, 1MB cache

Memory Size 2048 MB

Operating System Linux 2.4

Compiler gcc 3.2

Compiler Options -g -DNO_T_TYPES -DUSE_INTTYPES -DLinux

Network Type Intel e1000 Gigabit Ethernet, MTU = 9000

Testbed Configuration

LG # LG Type Network Target File Systems Notes

1/2/../16 LG1 N1 F1 N/A

Testbed Configuration
LG #	LG Type	Network	Target File Systems	Notes
1/2/../16	LG1	N1	F1	N/A

Other Notes

The system under test is ExaStore EX200FC (High Performance Cluster), which consists of two IBM x366 nodes cluster with one 144-disk Engenio 6998 FC Storage System.
The cluster is comprised of a redundant node pair.
The 6998 FC Storage System contains one 6998 controller module which contains two 6998 controllers.
The 6998 controller has four connections to four different FC adapters, two on each node.
The 6998 controller has 2 GB of battery-backed RAM for storage controller NVRAM.
NVRAM cache mirroring was disabled in the 6998 controller.
Read ahead was disabled in the 6998 controller.
The 6998 controller module holds twelve 2 Gb FC Drive Modules.
Each 2 Gb FC Drive Module holds 12 disks.
The 6998 controller module and the 2 Gb FC Drive Modules are chained together with redundant FC-ALs.
The single file system, F1, was comprised of 12 RAID-0 sets.
Each RAID set had a stripe size of 256 KB.
Uniform Access Rules (UAR):
Each Node in the cluster has four N1 network ports connected to the server network (N1) for communication
with all Load Generators.
Each node in the cluster has four N2 network ports connected to the cluster network (N2) via a trunked
connection for communication with the other node.
Each node in the cluster access all of the 6998 FC Storage System RAID sets.
Each load generator process accessed the cluster through a single node and had full access to all the data.
Each of the 2 nodes served as the access point for 1/2 of the load generator processes.
Stable Storage:
All memory in the ExaStore was used by the system for general-purpose memory and dynamically sized data cache.
There was an integral UPS for each ExaStore node, which guarantees sufficient power in the event of power failure
to dump the write cache to its local SCSI RAID. In case of unsafe situations (e.g. low battery condition),
the system uses a journal mechanism to synchronously log all write-type operations to the Engenio 6998 FC Storage System
which are connected to all nodes. When power is restored, the ExaStore system is able to recover all committed data
and metadata operations from journal files or dump files accordingly.
Server Tunings:
The nodes read ahead option was disabled.
The nodes write cache mirroring was disabled. This option enables mirroring of newly written data to a peer node to
support a higher standard of high availability in addition to the UPS backed memory.
The nodes dirty cache percent was decreased from 80% to 50% in order that written data transfer from the nodes to
the storage system will be performed sooner.
All periodical system diagnostics and monitoring jobs were disabled for reproducibility:
The cluster failover manager was disabled.
Logging of periodical statistics was disabled.
Event log compression and mail delivery were disabled.
Exanet, ExaMesh and ExaStore are trademarks of Exanet, Inc.
All other trademarks or registered trademarks are the property of their respective owners.

First published at SPEC.org on 01-Feb-2006

LG Type	LG1
LG Model	SuperServer 5013C-T
Number and Type Processors	1x3.0 GHz Pentium 4, 1MB cache
Memory Size	2048 MB
Operating System	Linux 2.4
Compiler	gcc 3.2
Compiler Options	-g -DNO_T_TYPES -DUSE_INTTYPES -DLinux
Network Type	Intel e1000 Gigabit Ethernet, MTU = 9000