SPECsfs2008_nfs.v3 Result

Throughput (ops/sec) Response (msec) 310130 0.3 621075 0.4 931969 0.4 1243548 0.5 1554753 0.5 1868974 0.6 2183089 2.7 2495548 3.5 2790350 3.5 3064602 3.8

Tested By	Huawei
Product Name	OceanStor N8500 Clustered NAS Storage System, 24 nodes
Hardware Available	July 2012
Software Available	July 2012
Date Tested	September 2012
SFS License Number	9021
Licensee Locations	West Zone Science Park of UESTC,No.88,Tianchen Road,Chengdu CHINA

The N8500 Clustered NAS Storage System is an advanced, modular, and unified storage platform (including NAS, FC-SAN, and IP SAN) designed for high-end and mid-range storage applications. Huawei OceanStor N8500 is a professional integrated storage solution that optimally utilizes storage resources. Its superior performance, reliability, ease of management, and flexible scalability maximize its cross-field application potential to span areas such as high-end computing, databases, websites, file servers, streaming media, digital video surveillance, and file backup.

Item No	Qty	Type	Vendor	Model/Name	Description
1	12	Engine Nodes	Huawei	N8500-EHS-N2M384G-DC	OceanStor N8500 Clustered NAS Storage System(including two internal nodes, totally 4*CPU and 384GB memory)
2	1	Fibre channel switch base module	Huawei	SN1-BASE	Huawei SNS5120 FC Switch,Base Module(AC,Dual CPU,Dual-Power Supply,Dual Fan)
3	6	Fibre channel switch I/O Blade	Huawei	SN1Z01FCIO	Huawei SNS5120 FC Switch,I/O Blade(8/4/2Gb FC,16 Ports,with 16*8Gb SFPs)
4	6	N8500 storage unit	Huawei	STTZ14SPES	OceanStor Dorado5100 High Performance Solid State Storage System Controller Enclosure,with HSSD Controller System Software
5	12	N8500 storage unit	Huawei	STTZ08DAE24	High Performance Solid State Storage System Disk Enclosure,2U,AC,400GB per SSD
6	6	N8500 storage unit	Huawei	SPE61C0200-S62-2C96G-DC-BASE	OceanStor S6800T Controller Enclosure(Dual Controller,DC,96GB Cache,with Huawei Storage Array Control System Software)
7	24	N8500 storage unit	Huawei	DAE12435U4-AC	DAE12435U4-03 Disk Enclosure(4U,3.5",AC,SAS Expansion Module)
8	144	Disk drive	Huawei	eMLC400G-S-3	HSSD eMLC 400GB SAS disk solid state Drive(3.5")
9	576	Disk drive	Huawei	SATA2K-7.2K	2000GB 7.2K RPM SAS-SATA Disk Drive(3.5")
10	1	10GE network switch frame	Huawei	CE12804-C	Huawei CloudEngine 12804 universal integrated cabinet
11	2	10GE network switch blade	Huawei	CE-L48XS-EA	Huawei CloudEngine 12804 48-Port 10GBASE-X Interface Card(EA,SFP+)

OS Name and Version	N8500 Clustered NAS Engine Software V200R001
Other Software	Integrated Storage Management V100R005C00
Filesystem Software	VxFS

Name	Value	Description
nfsd number	380	The number of NFS threads has been increased to 380 by editing /proc/fs/nfsd/threads
read_pref_io	1048576	Preferred read request size.
noatime	on	Mount option added to all filesystems to disable updating of access times.
vx_ninode	9000000	Maximum number of inodes to cache.
write_throttle	1	The number of dirty pages per file.

Server Tuning Notes

None

Description	Number of Disks	Usable Size
Dorado5100 storage unit, with 144 400GB SSDs in it, bound into 5+1 RAID5 LUNs	144	43.2 TB
S6800T storage unit, with 576 2T SATA HDDs in it, bound into 6+6 RAID10 LUNs	576	523.0 TB
Total	720	566.2 TB

Number of Filesystems	24
Total Exported Capacity	533299 GB
Filesystem Type	VxFS
Filesystem Creation Options	Default
Filesystem Config	Each client can access all the filesystems.
Fileset Size	363318.4 GB

Each filesystem consists of three LUNs, one LUN is from one 5+1 RAID5 of Dorado5100, and the other two from 6+6 RAID10 of S6800T. One RAID only has one LUN. The N8500 NAS system can dynamiclly and quickly recongnize hot files, and then migrate them between the Dorado5100 and the S6800T. The filesystem metadata and the hotspot are stored in Dorado5100, and the colder files are stored in the S6800T.

Item No	Network Type	Number of Ports Used	Notes
1	10 Gigabit Ethernet	48	There are 2 10GE network ports per N8500 engine node, both in use.

Network Configuration Notes

All network interfaces were connected to the 10GE network switch(Huawei CE12804).

An MTU size of 9000 was set for all connections to the switch.

Item No	Qty	Type	Description	Processing Function
1	48	CPU	CPU for N8500 engine nodes,Intel Six-Core Xeon Processor X5690 3.46GHz,12MB L3 cache	VxFS,NFS,TCP/IP
2	48	CPU	CPU for storage controllers,Intel Quad-core Xeon Processor E5540 2.53GHz,1MB L2 cache	block serving

Processing Element Notes

Each N8500 NAS Engine Node has two physical processors.

Description	Size in GB	Number of Instances	Total GB	Nonvolatile
Clustered NAS Engine Nodes memory	192	24	4608	V
The storage controllers provide write cache vault. When power failure is detected, a standby bbu will kick in to hold the controller cache contents and controller will flush them back to the vault.	96	12	1152	NV
Grand Total Memory Gigabytes			5760

Memory Notes

Each storage array has dual disk array controller units that work as an active-active failover pair.

The storage controller provide write cache vault. When power failure is detected, a standby bbu will kick in to hold the controller cache contents and controller will flush them back to the vault.

Default configuration

None

Item No	Qty	Vendor	Model/Name	Description
1	24	Huawei	RH2288	48GB RAM and Red Hat Enterprise Linux Server release 5.5

LG Type Name	RH2288
BOM Item #	1
Processor Name	X86 series-FCLGA2011-2200MHz-0.9V-64bit-95000mW-SandyBridge-EP Xeon E5-2660-8Core
Processor Speed	2.2 GHz
Number of Processors (chips)	2
Number of Cores/Chip	8
Memory Size	48 GB
Operating System	Red Hat Enterprise Linux Server release 5.5
Network Type	10 Gigabit Ethernet

Benchmark Parameters

Network Attached Storage Type	NFS V3
Number of Load Generators	24
Number of Processes per LG	576
Biod Max Read Setting	8
Biod Max Write Setting	8
Block Size	AUTO

Testbed Configuration

LG No	LG Type	Network	Target Filesystems	Notes
1..24	LG1	10 Gigabit Ethernet	/vx/fs01,/vx/fs02,/vx/fs03,...,/vx/fs24	N/A

Load Generator Configuration Notes

All 24 filesystems were mounted over 10GE by every load generator.

Each client can access all 24 filesystems from the same 10GE network switch. All the 24 filesystems are cluster mounted on all the 24 engine nodes, and each node exports all 24 filesystems. There are 24 shares on each engine node. Each engine node has a dual-port 10GE NIC, both ports are used, totally 48 IP addresses. Each filesystem is accessed over all data network interfaces (ip addresses ip1...ip48). 24 IP addresses are selected per client for mounting each filesystem, and the 24 IP addresses are selected in a round-robin manner from the list of 48 IP addresses such that each successive client used the next 24 ip addresses in the series. For example, for client 1,the mount points are ip1:/fs01,ip2:/fs01, ...ip12:/fs01, ip1:/fs02, ip2:/fs02, ...ip12:/fs02,... ,ip1:/fs24,...,ip12:/fs24, ip13:/fs01,ip14:/fs01, ...ip24:/fs01, ip13:/fs02, ip14:/fs02, ...ip24:/fs02,... ,ip13:/fs24, ip14:/fs24,...ip24:/fs24, so totally for client1, there are 576 RPOCs. For client2, repeat the same pattern with ip24 to ip48. For client 3, started with ip1 to ip24 again. This ensures that data access to every filesystem is uniformly distributed across all clients and target IP addresses. The 24 filesystems were striped evenly across all the disks from the storage units.

There were 24 filesystems: fs01, fs02, ..., fs24. All of them were cluster mounted on all the 24 server nodes. Each server node in the cluster exported all 24 filesystems. SpecSFS clients connected to a 10GE network switch to gain access to all the 24 server nodes in the cluster, and mapped all 24 NFS shares. Each specSFS client was able to do read/write IOs to all the 24 filesystems in a unified way with this setup.

• Config Diagram