bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs onsoftware raid 5

CONFIG:

Software RAID 5 (400GB x 6): Default mkfs parameters for all filesystems.
Kernel was 2.6.21 or 2.6.22, did these awhile ago.
Hardware was SATA with PCI-e only, nothing on the PCI bus.

ZFS was userspace+fuse of course.
Reiser was V3.
EXT4 was created using the recommended options on its project page.

RAW:

ext2,7760M,56728,96.3333,180505,51,85484,17.3333,50946.7,80. 3333,235541,21.3333,373.667,0,16:100000:16/64,2354,27,0,0,84 55.67,14.6667,2211.67,26.3333,0,0,9724,22.3333
ext3,7760M,52702.7,94.3333,165005,60,82294.7,20.6667,52664,8 3.6667,258788,33.3333,335.8,0,16:100000:16/64,858.333,10.666 7,10250.3,28.6667,4084,15,897,12.6667,4024.33,12.3333,2754,1 1.3333
ext4,7760M,53129.7,95,164515,59.3333,101678,31.6667,62194.3, 98.6667,266716,22.3333,405.767,0,16:100000:16/64,1963.67,23. 6667,0,0,20859,73.6667,1731,21.3333,9022,23.6667,16410,65.66 67
jfs,7760M,54606,92,191997,52,112764,33.6667,63585.3,99,27492 1,22.3333,383.8,0,16:100000:16/64,344,1,0,0,539.667,0,297.66 7,1,0,0,340,0
reiserfs,7760M,51056.7,96,180607,67,106907,38.3333,61231.3,9 7.6667,275339,29.3333,441.167,0,16:100000:16/64,2516,60.6667 ,19174.3,60.6667,8194.33,54.3333,2011,42.6667,6963.67,19.666 7,9168.33,68.6667
xfs,7760M,52985.7,93,158342,45,79682,14,60547.3,98,239101,20 .3333,359.667,0,16:100000:16/64,415,4,0,0,1774.67,10.6667,45 4,4.66667,14526.3,40,1572,12.6667
zfs,7760M,25601,43.3333,32198.7,4,13266.3,2,44145.3,68.6667, 129278,9,245.167,0,16:100000:16/64,218.333,2,2698.33,11.6667 ,7434.67,14.3333,244,2,2191.33,11.6667,5613.33,13.3333

HTML

http://home.comcast.net/~jpiszcz/benchmark/allfs.html

THOUGHTS

Overall JFS seems the fastest but reviewing the mailing list for JFS it
seems like there a lot of problems, especially when people who use JFS > 1
year, their speed goes to 5 MiB/s over time and the defragfs tool has been
removed(?) from the source/Makefile and on Google it says not to use it
due to corruption.

Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs on software raid 5

[trimmed all but linux-raid from the cc]

On 7/30/07, Justin Piszcz wrote:
> CONFIG:
>
> Software RAID 5 (400GB x 6): Default mkfs parameters for all filesystems.
> Kernel was 2.6.21 or 2.6.22, did these awhile ago.
Can you give 2.6.22.1-iop1 a try to see what affect it has on
sequential write performance?

Download:
http://downloads.sourceforge.net/xscaleiop/patches-2.6.22.1- iop1-x86fix.tar.gz

Unpack into your 2.6.22.1 source tree. Install the x86 series file
"cp patches/series.x86 patches/series". Apply the series with quilt
"quilt push -a".

I recommend trying the default chunk size and default
stripe_cache_size as my tests have shown improvement without needing
to perform any tuning.

Thanks,
Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs on software raid 5

Justin Piszcz wrote:
> CONFIG:
>
> Software RAID 5 (400GB x 6): Default mkfs parameters for all filesystems.
> Kernel was 2.6.21 or 2.6.22, did these awhile ago.
> Hardware was SATA with PCI-e only, nothing on the PCI bus.
>
> ZFS was userspace+fuse of course.

Wow! Userspace and still that efficient.

> Reiser was V3.
> EXT4 was created using the recommended options on its project page.
>
> RAW:
>
> ext2,7760M,56728,96.3333,180505,51,85484,17.3333,50946.7,80. 3333,235541,21
>.3333,373.667,0,16:100000:16/64,2354,27,0,0,8455.67,14.6667 ,2211.67,26.3333
>,0,0,9724,22.3333
> ext3,7760M,52702.7,94.3333,165005,60,82294.7,20.6667,52664,8 3.6667,258788,
>33.3333,335.8,0,16:100000:16/64,858.333,10.6667,10250.3,28. 6667,4084,15,897
>,12.6667,4024.33,12.3333,2754,11.3333
> ext4,7760M,53129.7,95,164515,59.3333,101678,31.6667,62194.3, 98.6667,266716
>,22.3333,405.767,0,16:100000:16/64,1963.67,23.6667,0,0,2085 9,73.6667,1731,2
>1.3333,9022,23.6667,16410,65.6667
> jfs,7760M,54606,92,191997,52,112764,33.6667,63585.3,99,27492 1,22.3333,383.
>8,0,16:100000:16/64,344,1,0,0,539.667,0,297.667,1,0,0,340,0
> reiserfs,7760M,51056.7,96,180607,67,106907,38.3333,61231.3,9 7.6667,275339,
>29.3333,441.167,0,16:100000:16/64,2516,60.6667,19174.3,60.6 667,8194.33,54.3
>333,2011,42.6667,6963.67,19.6667,9168.33,68.6667
> xfs,7760M,52985.7,93,158342,45,79682,14,60547.3,98,239101,20 .3333,359.667,
>0,16:100000:16/64,415,4,0,0,1774.67,10.6667,454,4.66667,145 26.3,40,1572,12.
>6667

> zfs,7760M,

Dissecting some of these numbers:

speed %cpu
> 25601,43.3333,
> 32198.7,4,
> 13266.3, 2,
> 44145.3,68.6667,
> 129278,9,
> 245.167,0,

> 16:100000:16/64,

speed %cpu
> 218.333,2,
> 2698.33,11.6667,
> 7434.67,14.3333,
> 244,2,
> 2191.33,11.6667,
> 5613.33,13.3333

Extrapolating these %cpu number makes ZFS the fastest.

Are you sure these numbers are correct?


Thanks!

--
Al

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs on software raid 5

> Extrapolating these %cpu number makes ZFS the fastest.
>
> Are you sure these numbers are correct?

Note, that %cpu numbers for fuse filesystems are inherently skewed,
because the CPU usage of the filesystem process itself is not taken
into account.

So the numbers are not all that good, but according to the zfs-fuse
author it hasn't been optimized yet, so they may improve.

Miklos
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs on

On Mon, 2007-07-30 at 10:29 -0400, Justin Piszcz wrote:

> Overall JFS seems the fastest but reviewing the mailing list for JFS it
> seems like there a lot of problems, especially when people who use JFS > 1
> year, their speed goes to 5 MiB/s over time and the defragfs tool has been
> removed(?) from the source/Makefile and on Google it says not to use it
> due to corruption.

The defragfs tool was an unported holdover from OS/2, which is why it
was removed. There never was a working Linux version. I have some
ideas to improve jfs allocation to avoid fragmentation problems, but jfs
isn't my full-time job anymore, so I can't promise anything. I'm not
sure about the corruption claims. I'd like to hear some specifics on
that.

Anyway, for enterprise use, I couldn't recommend jfs, since there is no
full-time maintainer.

Thanks,
Shaggy
--
David Kleikamp
IBM Linux Technology Center

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs onsoftware raid 5

On Mon, 30 Jul 2007, Miklos Szeredi wrote:

>> Extrapolating these %cpu number makes ZFS the fastest.
>>
>> Are you sure these numbers are correct?
>
> Note, that %cpu numbers for fuse filesystems are inherently skewed,
> because the CPU usage of the filesystem process itself is not taken
> into account.
>
> So the numbers are not all that good, but according to the zfs-fuse
> author it hasn't been optimized yet, so they may improve.
>
> Miklos
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

This was performed on an E6300, 1 core was ZFS/FUSE (or quite a bit of it
anyway)

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs onsoftware raid 5

On Mon, 30 Jul 2007, Dan Williams wrote:

> [trimmed all but linux-raid from the cc]
>
> On 7/30/07, Justin Piszcz wrote:
>> CONFIG:
>>
>> Software RAID 5 (400GB x 6): Default mkfs parameters for all filesystems.
>> Kernel was 2.6.21 or 2.6.22, did these awhile ago.
> Can you give 2.6.22.1-iop1 a try to see what affect it has on
> sequential write performance?
>
> Download:
> http://downloads.sourceforge.net/xscaleiop/patches-2.6.22.1- iop1-x86fix.tar.gz
>
> Unpack into your 2.6.22.1 source tree. Install the x86 series file
> "cp patches/series.x86 patches/series". Apply the series with quilt
> "quilt push -a".
>
> I recommend trying the default chunk size and default
> stripe_cache_size as my tests have shown improvement without needing
> to perform any tuning.
>
> Thanks,
> Dan
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

Will keep in mind for next test, but like I said these were from a while
ago.

Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: bonnie++ benchmarks for ext2,ext3,ext4,jfs,reiserfs,xfs,zfs on software raid 5

On Mon, Jul 30, 2007 at 09:39:39PM +0200, Miklos Szeredi wrote:
> > Extrapolating these %cpu number makes ZFS the fastest.
> >
> > Are you sure these numbers are correct?
>
> Note, that %cpu numbers for fuse filesystems are inherently skewed,
> because the CPU usage of the filesystem process itself is not taken
> into account.
>
> So the numbers are not all that good, but according to the zfs-fuse
> author it hasn't been optimized yet, so they may improve.

Also, something which is data i/o intensive is going to be the best
case for a FUSE filesystem. If you try something which is much more
metadata intensive (i.e., lots of file creates and deletes, chmods,
etc.) like say with a Postmark benchmark, you would almost certainly
get very different results. That's not to say that bonnie++
benchmarks aren't useful, but when doing comparisons between
filesystems, it's a good idea to use a wide variety of benchmarks to
avoid getting potentially misleading results.

- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html