Performance Testing Raid

The following tests were performed against a ReiserFS file system to validate the read and write performance of various RAID configurations. The disk storage was a Gateway 840 disk storage system, which is a 12-bay SATA cage with embedded hardware RAID and an Ultra-320 SCSI interface. The host was a Gateway 9415 dual Xeon64 server with 2 gigabytes of RAM and an embedded Ultra-320 SCSI controller. The operating system was SUSE Linux 9.2 Professional. It is my belief this configuration is probably typical of modern, indexpensive, x86 based storage servers.

bonnie - parameters passed -d /scratch1 -s 2030 -y
postmark - parameters passed set location /scratch1, set number 5000, set transactions 5000, set buffering false,set size 500 10000000

The filesystem was reformatted, and the host was power cycled between each test suite to make sure that the tests were conducted without any possibility of corruption of the results from cache. Yes, this is being pedantic.

RAID 50

The RAID 50 configuration consists of two striped RAID 5 mirrors. It is designed to be provide a compromise between the speed of RAID 1+0 and the storage capacity of RAID 50.

bonnie on RAID 50

islay:/scratch1 # bonnie -d /scratch1 -s 2030 -y
Bonnie 1.4: File '/scratch1/Bonnie.6082', size: 2128609280, volumes: 1
Writing with putc()...         done:  39426 kB/s  96.1 %CPU
Rewriting...                   done:  68278 kB/s  21.3 %CPU
Writing intelligently...       done: 112074 kB/s  32.1 %CPU
Reading with getc()...         done:  46145 kB/s  91.4 %CPU
Reading intelligently...       done: 297031 kB/s  40.8 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
              ---Sequential Output (sync)----- ---Sequential Input-- --Rnd Seek-
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
islay  1*2030 39426 96.1112074 32.1 68278 21.3 46145 91.4297031 40.8 1550.9  2.2

postmark on RAID 50

islay:~ # /scratch/postmark 
PostMark v1.5 : 3/27/01
pm&set location /scratch1
pm&set number 5000
pm&set transactions 5000
pm&set buffering false
pm&set size 500 10000000
pm&run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
        1555 seconds total
        866 seconds of transactions (5 per second)

Files:
        7482 created (4 per second)
                Creation alone: 5000 files (7 per second)
                Mixed with transactions: 2482 files (2 per second)
        2510 read (2 per second)
        2490 appended (2 per second)
        7482 deleted (4 per second)
                Deletion alone: 4964 files (97 per second)
                Mixed with transactions: 2518 files (2 per second)

Data:
        13215.77 megabytes read (8.50 megabytes per second)
        40857.20 megabytes written (26.27 megabytes per second)

RAID5

RAID5 has been a mainstay for years. It offers good redundancy and decent read performance, and maximizes the disk space. However, there is a cost, and that is in write performance.

bonnie on RAID 5

islay:~ # bonnie -d /scratch1 -s 2030 -y
Bonnie 1.4: File '/scratch1/Bonnie.5962', size: 2128609280, volumes: 1
Writing with putc()...         done:  37991 kB/s  92.7 %CPU
Rewriting...                   done:  39368 kB/s  12.8 %CPU
Writing intelligently...       done: 105574 kB/s  30.1 %CPU
Reading with getc()...         done:  46009 kB/s  91.5 %CPU
Reading intelligently...       done: 309707 kB/s  42.8 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
              ---Sequential Output (sync)----- ---Sequential Input-- --Rnd Seek-
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
islay  1*2030 37991 92.7105574 30.1 39368 12.8 46009 91.5309707 42.8 1944.4  2.8

postmark on RAID 5

islay:~ # /scratch/postmark 
PostMark v1.5 : 3/27/01
pm&set location /scratch1
pm&set number 5000
pm&set transactions 5000
pm&set buffering false
pm&set size 500 10000000
pm&run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
        1845 seconds total
        959 seconds of transactions (5 per second)

Files:
        7482 created (4 per second)
                Creation alone: 5000 files (5 per second)
                Mixed with transactions: 2482 files (2 per second)
        2510 read (2 per second)
        2490 appended (2 per second)
        7482 deleted (4 per second)
                Deletion alone: 4964 files (95 per second)
                Mixed with transactions: 2518 files (2 per second)

Data:
        13215.77 megabytes read (7.16 megabytes per second)
        40857.20 megabytes written (22.14 megabytes per second)

RAID 0

RAID 0 should, in theory, offer the highest possible read and write performance, but at a high risk of data loss.

bonnie on RAID 0

islay:~ # bonnie -d /scratch1 -s 2030 -y
Bonnie 1.4: File '/scratch1/Bonnie.5881', size: 2128609280, volumes: 1
Writing with putc()...         done:  40936 kB/s  99.8 %CPU
Rewriting...                   done:  85503 kB/s  27.7 %CPU
Writing intelligently...       done: 142080 kB/s  36.4 %CPU
Reading with getc()...         done:  45821 kB/s  91.1 %CPU
Reading intelligently...       done: 329967 kB/s  45.8 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
              ---Sequential Output (sync)----- ---Sequential Input-- --Rnd Seek-
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
islay  1*2030 40936 99.8142080 36.4 85503 27.7 45821 91.1329967 45.8 2159.4  3.0

postmark on RAID 0

islay:~ # /scratch/postmark 
PostMark v1.5 : 3/27/01
pm&set location /scratch1
pm&set number 5000
pm&set transactions 5000
pm&set buffering false
pm&set size 500 10000000
pm&run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
        1024 seconds total
        642 seconds of transactions (7 per second)

Files:
        7482 created (7 per second)
                Creation alone: 5000 files (14 per second)
                Mixed with transactions: 2482 files (3 per second)
        2510 read (3 per second)
        2490 appended (3 per second)
        7482 deleted (7 per second)
                Deletion alone: 4964 files (118 per second)
                Mixed with transactions: 2518 files (3 per second)

Data:
        13215.77 megabytes read (12.91 megabytes per second)
        40857.20 megabytes written (39.90 megabytes per second)

RAID 10

The Gateway 840 does not support RAID 1 (mirroring). However, it does support RAID 10, which is a combination of striping and mirroring. RAID 10 should offer similar performance as RAID 0 while providing disk redundancy.

bonnie on RAID 10

islay:~ # bonnie -d /scratch1 -s 2030 -y
Bonnie 1.4: File '/scratch1/Bonnie.6517', size: 2128609280, volumes: 1
Writing with putc()...         done:  39204 kB/s  95.5 %CPU
Rewriting...                   done:  66074 kB/s  21.0 %CPU
Writing intelligently...       done: 121126 kB/s  33.7 %CPU
Reading with getc()...         done:  43011 kB/s  85.2 %CPU
Reading intelligently...       done: 215739 kB/s  29.1 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
              ---Sequential Output (sync)----- ---Sequential Input-- --Rnd Seek-
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
islay  1*2030 39204 95.5121126 33.7 66074 21.0 43011 85.2215739 29.1 2425.6  3.6

postmark on RAID 10

islay:~ # /scratch/postmark 
PostMark v1.5 : 3/27/01
pm&set location /scratch1
pm&set number 5000
pm&set transactions 5000
pm&set buffering false
pm&set size 500 10000000
pm&run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
        1199 seconds total
        808 seconds of transactions (6 per second)

Files:
        7482 created (6 per second)
                Creation alone: 5000 files (14 per second)
                Mixed with transactions: 2482 files (3 per second)
        2510 read (3 per second)
        2490 appended (3 per second)
        7482 deleted (6 per second)
                Deletion alone: 4964 files (99 per second)
                Mixed with transactions: 2518 files (3 per second)

Data:
        13215.77 megabytes read (11.02 megabytes per second)
        40857.20 megabytes written (34.08 megabytes per second)

Summary of Results

RAID Level	Bonnie Write	Bonnie Read	Postmark Write	Postmark Read
RAID 50	39426 kB/s	46145 kB/s	26.27 megabytes/s	8.5 megabytes/s
RAID 5	37991 kB/s	46009 kB/s	22.14 megabytes/s	7.16 megabytes/s
RAID 0	40936 kB/s	45821 kB/s	39.90 megabytes/s	12.91 megabytes/s
RAID 10*	39204 kB/s	43001 kB/s	34.08 megabytes/s	11.02 megabytes/s

*I find it interesting to note that according to bonnie, the read/write performance of RAID 50 is noticeably better than RAID 10, while postmark shows that RAID 10 has 30% better throughput over RAID 50. My subjective perception has been that in real world applications, RAID 10 offers better performance, so maybe bonnie is not that valid of a test.

Bonnie is included in SUSE 9.2. Postmark was downloaded from Network Appliances at the url http://www.netapp.com/tech_library/postmark.html.