New Ext3 vs ReiserFS benchmarks 191
An anonymous reader writes "Saw this new benchmark on the linux-kernel mailing list. Although NAMESYS, the developers of ReiserFS has many benchmarks on their site, they only have one Ext3 benchmark. The new benchmark tests Ext3 in ordered and writeback mode versus ReiserFS with and without the notail mount option. Better than expected results for Ext3. Big difference between ordered and writeback modes."
Anyone care to explain ordered mode? (Score:1, Interesting)
Re:Anyone care to explain ordered mode? (Score:4, Informative)
Writeback kicking it (Score:3, Informative)
Re:Writeback kicking it (Score:2)
I remember in the mid-80s, Stratus and Tandem would duel over TPC benchmarks, and while Stratus did respectably on conventional disk-based writes, they did try to get the TPC council to allow writes to their resilient (duplicated), battery-backed memory to count too. I don't think they succeeded then, and IMHO some rather cruddy PC memory system should not be allowed to count now.
Re:Writeback kicking it (Score:2)
Re:Writeback kicking it (Score:3, Informative)
Re:Writeback kicking it (Score:3, Insightful)
But if you look at the NAMESYS benchmark comparing ext2 to ext3 and ResierFS then it is clear that for sheer throughput ext2 wins...
IF Speed is your reason for choosing a Filesystem then writeback wins on almost everything in these examples...
But using a Journaled Filesystem isn't usually done for Speed... Unless you count speed booting after a crash. It's done to (more or less) guarantee filesystem integrity after a crash. You may lose data, but you only lose writes that never completed.
So, if you are choosing ext3 with writeback, is it faster than native ext2? I don't know. But it doesn't sound like it is any safer.
Of course, if you're worried about data integrity, you will have a mirror across multiple striped drives using multiple controllers. And then use a Journaled Filesystem to improve boot time.
Z.
Re:Writeback kicking it (Score:3, Interesting)
Now, data corruption can be a Very Bad Thing, depending on what you're doing... but in many cases, preventing metadata corruption (and thus being sure that your filesystem is always usable) is Good Enough.
Re:Writeback kicking it (Score:2, Informative)
This is a misinformed opinion, at best. Your RAID setup will only save data in the case of hardware failure (i.e. one of your disks fails). It will do nothing about incomplete writes. The whole purpose of journaled filesystems is to ensure that writes completed, to minimize filesystem corruption. It just so happens that the way it does this allows for a faster boot, which is an added bonus.
Re:Writeback kicking it (Score:2)
I've seen such an option on big external RAID arrays. Makes sense, lets the write cache be written to disk before the power goes out.
I'm curious, though, do any hard drives have this feature? Maybe not a full battery, but perhaps a capacitor to store enough juice to write that 8 MB of cache data down to disk before it's gone for good? Or perhaps some sort of bolt-on option for existing internal drives?
I ask this as I'm an average joe with home-brew and cheap-label servers (I built most, a few are PII Dells and Gateways). My machines are pretty stable, but I only have about 70 minutes of battery backup from my UPS... and there's no way I could justify buying a generator.
Re:Writeback kicking it (Score:1)
Re:Writeback kicking it (Score:2, Informative)
Also worth nothing -- we have our Exchange server begin shutdown almost immediately after the power goes out, as it takes exchange nearly 15 minutes just to shut down. We are actively looking for an alternative to Exchange.
Re:Writeback kicking it (Score:3, Interesting)
This all assumes that you have the network on a UPS and with the power out all machines can still talk.
Pretty nice tool with tons of options. http://www.apcupsd.org (oddly, with the exception of the what's new pages of the docs, the url isn't listed in the docs.)
Of course, I like my option - buy a UPS with enough capacity to hold the whole room for about 30 minutes (40KW) and a big ole generator in case things go down for a while.
Re:Writeback kicking it (Score:2)
A few months ago, I had a short (2 minute) power outage and of course my UPS kicked in and my server stayed online as you might expect. However, when power was restored, the apcupsd scripts were (by default) configured to reboot the server after a return to utility power. Why this is the case, I cannot answer, however I'm sure there is a logical explanation. In my case, I found this very unsettling as it caused my 100+ days of uptime to return to zero whence they came. The scripts were easy to fix, but hopefully this will serve as a warning for those of you who cannot afford the restart.
On a slightly different note, I'm still not understanding the whole journalling file system issue; I understand the benefits, but are you really crashing that much (which must be hard locks), that you need to do a hard reset and let the journal replay the transactions? Personally, I have a tape backup, and a UPS. Do I really need a journalling file system, other than the obvious advantage of impressing the ladies? At the moment, I'm interested in XFS because of the ACLs and the "intensive disk usage" features SGI has in the IRIX version, and I'm hoping those make it into the "final" Linux version (if there ever will be a "final" version).
I'm curious... (Score:2)
Re:I'm curious... (Score:2)
The system wasn't halted. The UPS kicked in and ran on batteries for a couple minutes then switched back to mains. The server remained up and running. The apcupsd daemon was set to run a script when hen the utility power returned, and the script was configured to be "shutdown -r now"
At no point during the process was the system halted.
Re:Writeback kicking it (Score:2)
On one of our old systems, the network admin asked what a button did as he pushed it. It was the power button. At another time the same guy accidentally dropped a pencil that hit the same power button (actually a rocker switch) again. Someone else was curious as to what the inside of that machine looked like, so they opened the swinging back door of the case, which caused the system to power down (oh that poor TI 1500)
Power cords get tripped over. UPS's fail. UPS software does odd things. Hardware fails.
Yeah...you have backups. Those fail as well, and restores take time. A journaling file system takes a few seconds after an abnormal startup to fix itself.
Just think of it as yet another layer of protection beyond the UPS and backup tapes. And of course it helps get the ladies
Re:Writeback kicking it (Score:2)
Of course, if you dont need a feature for feature match with Exchange there are unlimited cheap alternatives for mail servers.
Re:Writeback kicking it (Score:1)
There are cases where a UPS won't prevent an "unexpected downtime". In these cases, it might be helpful if the drives were able to finish their last write on their own power. It might give you something to boot after you correct the problem.
I'm spoiled (Score:1, Funny)
Re:I'm spoiled (Score:2)
He's actually referring to the MadOnion 3DMark2001 benchmark.
http://www.madonion.com
If you've never seen it, it's a killer.
Simon
Journaled ext3 vs Reiserfs (Score:3, Informative)
http://labs.zianet.com
There are some decent benchmarks there that compare the two as well as extensive NFS tests.
ReiserFS loses data (Score:1, Informative)
Speaking of losing data... (Score:1)
Re:Speaking of losing data... (Score:4, Funny)
Awww, and here I thought you were trying to give an example of what the ReiserFS did to your data during a hash collision.
Re:ReiserFS loses data (Score:4, Interesting)
Interesting (Score:1)
Re:Interesting (Score:2, Insightful)
Re:Interesting (Score:2)
To prove you theory you could take the hash function in reiserfs and replace it with a function that always returns '1'. You would probably have to reformat your partitions though for that test though. The filesystem should still work. If it doesn't that's a bug.
The chances of their being a bug in reiserfs is about 100%. Same is true of ext3 though.
Re:ReiserFS loses data (Score:2, Informative)
I'm 110% sure it's saved more files when I've lost power or when something's hung requiring a hard reset than it'd deleted due to hash clashes. What's the likelihood of two files generating the same hash? You talk of increasing likeliness, but don't mention any figures. It's hard to judge without some stats.
As an aside, why didn't you restore your large project from your backup? What do you mean you didn't have...
Re:Can you document that? (Score:5, Insightful)
If this is true, surely it must be documented somewhere, or have been discussed in a credible forum? I did a little searching, and didn't find anything. Please post a URL to elevate your comment from unsubstantiated rumor to informative information.
In most hash-based indexing algorithms I know of, hash collisions incur a perfomance penalty, but not a data loss.
Re:Can you document that? (Score:2, Funny)
Informative information? I really ought to use "Preview" before "Submit".
Not a troll! (Score:1, Troll)
Your bong had a hash collision (Score:4, Funny)
Is it possible that there is a bug in reiserfs? Sure. I just don't trust anecdotal evidence from some dood on
Re:ReiserFS loses data (Score:2)
This is not necessarily a bug if the probability of that happening in real world scenarios is negligible. After all, you risk data loss from many sources.
Unfortunately, programmers often seem a bit unreasonable about probabilities. They complain about a (say) 1:10^20 chance of losing a file, while at the same time writing the whole file system in C, which basically guarantees a several-fold increase in the probability of undetected software faults compared to alternatives. In fact, the fix for such a remote possibility may not only kill performance, it may actually increase the overall probability of a fault that causes data loss--because the extra code may have bugs.
So, no, this doesn't bother me. I suspect that if Reiser knows about it and he isn't fixing it, he probably thought about it and decided the probability is too remote. If you disagree, I would like to see a more detailed analysis from you.
Re:ReiserFS loses data (Score:3, Insightful)
Provide at least one pair of filepaths which generate a hash collision under whatever scenario you care to specify, so that others can test and verify the resulting effect, even if it's probabilistic and requires billions of reruns to trigger -- no problem.
If the effect isn't seen by anyone else under any conditions, then the problem doesn't exist. Conversely, if it does happen under some repeatable conditions (even if only extremely rarely) then it *is* a problem, and will be fixed.
If you want to be constructive about it, take this issue out of mythology and onto firmer ground.
Small (Big?) Surprise. (Score:2)
that reiser would do so well on the heavy-throughput/large file test.
I've been laboring under the perception that reiser was good for randomly accessing small files, but paid a performance penalty when going after large ones.
Guess I'm still waiting to prove that no one can be wrong about everything!
Cheers.
my decision (Score:5, Insightful)
Also I like the idea that I can read the drive with an ext2 driver from an older kernel or from FreeBSD just in case. In case of what? I don't know, but somehow it makes me feel better.
Re:my decision (Score:1)
Hard disk crash? no problem -- drop in a new drive and the cd with your partition image and you're up in 15 minutes.
Note: I'm not affliated with PowerQuest -- I just buy their software when I've got money left over from buying a book of the new 37 cent US first class stamps...
Re: (Score:2, Interesting)
Gurulabs background picture? (Score:1)
Offtopic, but seems to me that the picture that gurulabs is using as background for their web page is ripped from the cover artwork of the album "Rally of Love" by the Finnish band 22-Pistepirkko. Wonder if they have permission for that?
Of course, could be that the album cover is a copy of something that is in the public domain...
Re:Gurulabs background picture? (Score:1, Informative)
Offtopic, good class though.
OT: 22 spotted ladybug (Score:2)
Bare bone nest is one of 5 records that keeps the LP player hooked up to the stereo.
ext3 writeback vs ext2? (Score:2)
Re:ext3 writeback vs ext2? (Score:1)
But for some people, that appears to be enough...
Re:ext3 writeback vs ext2? (Score:3, Insightful)
so what's the point of running ext3 in writeback if (as the faq says) it's exactly equivalent to ext2 "with a very fast fsck"?
Consider a large tmp volume.
Anywhere where the consequences of finding stale data in a file are no worse than having the data simply missing after a crash. Even a src directory if you do a lot of big makes (since you're best off with make clean ; make after a crash anyway). Just be sure to sync after writing out a source file.
However, as long as performance is adequate, probably better safe than sorry when it comes to filesystems.
Re:ext3 writeback vs ext2? (Score:3, Informative)
Well, ext3 with data=writeback is equivalent to how reiserfs has always operated (i.e. if you crash you can lose data in files that were being written to). Using data=ordered is an extra benefit that doesn't have any noticable performance hit unless you are trashing the disk and RAM in a benchmark. FYI, there are now beta patches for reiserfs that implement data=ordered.
Only the fsck time can be a big deal if you have to wait 8 hours while your 1TB storage array is fscking (8 hours is a guess, I don't have that much disk...)
Re:ext3 writeback vs ext2? (Score:3, Funny)
Re:ext3 writeback vs ext2? (Score:1)
Re:ext3 writeback vs ext2? (Score:1)
...what I would like to see (Score:2, Interesting)
Re:...what I would like to see (Score:1)
You might also want to make sure you compiled ext3 support into your kernel. Not trying to be a jackass, but not everyone has the latst kernel, I just upgraded from 2.2.17 for ext3 personally. Giving sloppy advice like that to somebody could be bad. Shame on you.
I have to wonder about the competence... (Score:1, Offtopic)
Re: (Score:2)
Re:I have to wonder about the competence... (Score:2)
Re: (Score:2)
Slashdot fight agains standards. (Score:3, Informative)
Completely true. I've filed a bug to the slashdot bug report page in sourceforge to add some semantic tags to the ones we are allowed to use. I'd like to use , , etc. The bug was deleted as quick as it was posible, with no explanation.
Besides, not only the HTML code doesn't validate. but also Slashdot has blocked [w3.org] the W3C validator!. That's very stupid, as anyone can just download and validate the page uploading it to the validator. Here is the validation result [technisys.com.ar].
Re: (Score:2)
XFS? (Score:3, Interesting)
Re:XFS? (Score:1)
i'd personally use XFS over any of the others any day, mainly since its fsck free and is a file system that is known to work well (its been used/owned by SGI, yea know).
UFS + soft updates (Score:1)
It's fast, stable, no fscking after a
dirty reboot. Anyone know of benchmarks
comparing this to ext3 or riser?
Danger, Will Robinson (Score:2, Informative)
Here is what you are missing. Soft updates is a method of ensuring that disk metadata is recoverably consistent without the normal speed penalty imposed by synchronous mounting. The only guarantee that softupdates makes is that your file system can be recovered to a consistent state by running fsck. Soft updates is designed to aid the running of fsck, but does not eliminate the need.
Better get out your Palm add running fsck to your "to-do" list.
Why always Linux? (Score:2, Interesting)
Better yet, why did EXT get to be the defacto Linux filesystem, rather than UFS? It outperforms, and supports much large files/filesystems.
A comparison of UFS from a platform other than FreeBSD might be in order.
Re:Why always Linux? (Score:1)
My understading of the sitation is that it was because until softupdates were implemented UFS was painful. Now, had softupdates been implemented, say, 7-10 YEARS ago when EXT became the Linux de-faco filesystem there might have been a chance.
On the flip side, seeing a good Linux implementation of a BSDish UFS with softupdates would be very nice.
- RustyTaco
Re:Why always Linux? (Score:2)
Re:Why always Linux? (Score:2, Interesting)
On a fresh install of FreeBSD 4.6 using UFS, bonnie reported more than 30 MB/s on the same machine.
I know this isn't really what you were looking for but it surprised me that there was that much of a difference.
Re:Why always Linux? (Score:2, Interesting)
Just for the hell of it I ran the same benchmarks on one of my test boxes (FreeBSD running -current). The performance basically comes down to how much write latency you are willing to endure... the longer the latency, the better the benchmark results for the first two tests.
So, for example, with the (conservative) system defaults I only got around 250 trans/sec for mixed creations with the first postmark test, because the system doesn't allow more then around 18MB of dirty buffers to build up before it starts forcing the data out, and also does not allow large sequential blocks of dirty data to sit around. When I bump up the allowance to 80MB and turn off full-block write_behind the trans rate went up to 2776/sec. I got similar characteristics for the second test as well. Unfortunately I have only one 7200 rpm hard drive on this box so I couldn't repeat the third test in any meaningful way (which is a measure mostly of disk bandwidth).
In anycase, the point is clear, and the authors even mention it by suggesting that the ext3 write-back mode should only be used with NVRAM. Still, I don't think they realize that their RedHat box likely isn't even *writing* the data to the disk/NVRAM until it absolutely has to, so arbitrarily delaying writes for what is supposed to be a mail system is not a good evaluation of performance. Postmark does not fsync() any of the operations it tests whereas any real mail system worth its salt does, and even with three drives striped together this would put a big crimp on the reported numbers unless you have a whole lot of NVRAM in the RAID controller.
I do not believe RedHat does the write-behind optimization that FreeBSD does. This optimization exists specifically to maximize sequential performance without blowing up system caches (vs just accumulating dirty buffers). But while this optimization is good in most production situations it also typically screws up non-sequential benchmark numbers by actually doing I/O to the drive when the benchmark results depend on I/O not having been done :-).
Last thought. Note that the FreeBSD 4.6 release has a performance issue with non-truncated file overwrites (not appends, but the 'rewrite without truncation' type of operation). This was fixed post-release in -stable.
-Matt
Re:Journaling (Score:2)
OK, asshole. How about we start with Journaling Versus Soft Updates: Asynchronous Meta-data Protection in File Systems [usenix.org] presented at USENIX 2000? The first three authors should need no introduction, so I think it satisfies the "well known" requirement; in fact, one could hardly find a group of six people more qualified to comment on the matter. Even in the abstract, the authors clearly state the similarity between goals of journaling and soft updates:
The similarity is mentioned repeatedly elsewhere in the paper, all the way to the conclusion, but I'll let you do your homework this time.
Anybody who knows anything about filesytems - and I've been working on them for over a decade - recognizes the similarity in goals between journaling, soft updates, and phase trees. Usually it's considered too obvious even to require comment, unless an ignorant troll like you comes along demanding that the obvious be spelled out.
As always, it depends on what is on the filesystem (Score:5, Informative)
We benchmark ReiserFS versus all other Linux filesystems about once every 6 months or so, and the last one from about 3 months ago still places Reiser in the "significantly faster" category for our workloads, specifically web caching with Squid.
ext3 is a nice filesystem, and I use it on my home machine and my laptop. But for some high performance environments, ReiserFS is still superior by a large margin. It is also worth mentioning that I could crash a machine running ext3 at will the last time we ran some Squid benchmarks (this was on 2.4.9-31 kernel RPM from Red Hat, so things have probably been fixed by now).
All that said, I'll be giving ext3 vs. ReiserFS another run real soon now, since there does seem to be some serious performance and stability work going into ext3.
Why I like reiserfs (Score:1, Insightful)
Sure ext3 may benchmark slightly faster in certain scenarios. But unless you know ahead of time that those are the only scenarios you are going to put on the file system, I recommend reiserfs.
Testimony (Score:1)
But ext3.. I've been using it since the day RH7.3 was released, during which time I'll bet power to my machine has been cut at least 150 times (we had a bad circuit breaker that would randomly flip. I replaced it a few days ago). Often power was repeatedly lost many times in a short period of time (if that would matter), and in the middle of big disk write operations.
Every single time I have been able to immediately reboot without any apparent data loss (except for the data being written at that very second) or filesystem corruption (a couple of times I forced a check just to make sure nothing was wrong, and nothing ever was).
I can't testify to the relative quality of ext3 compared to ReiserFS, but I can certainly say I have been quite pleased with the stability of ext3.
-Alan
EXT3 is to EXT2 as VFAT is to FAT (Score:1)
ReiserFS has eaten more megabytes then I would have liked...but that was 2 years ago. Comparing Resier which is a mature, next generation FS to EXT3, a revamp which isn't even done yet, is a bad idea.
Re:EXT3 is to EXT2 as VFAT is to FAT (Score:2)
"We need to choose a file system. Let's try to experimentally determine which of out two prime contenders is best."
You may feel that their selection of contenders is incorrect, but to select between them based on experiment is called "the experimental method" (sometimes mistakenly "the scientific method". This is the basis of science, engineering, and technology. I.e.: Don't assume ahead of time that you know the right answer, check.
If they didn't find the problems that you expected, then perhaps you need to examine why. But a hand-waving "explanation" doesn't explain very much, so I don't even really know what problems you think they should have found. FWIW, I haven't noticed any instability problems with ext3.
For multimedia playback? (Score:3, Informative)
Re:For multimedia playback? (Score:1)
I use both (Score:3, Insightful)
Re:I use both (Score:2)
Re:I use both (Score:1)
Re:I use both (Score:2)
OTOH, a year (I think) earlier, when Mandrake released a Reiser file system option, I tried it, had disk corruption, and couldn't find any tools that helped recovery.
Now these are single data points, so you shouldn't take them too seriously. Also, around the same time that I had file corruption under Reasser, I also had an ext2 file system become corrupt. I even know that the problem was caused by fsck. (I was running from a secondary hard disk. I think that this may have been a kernel problem.) The point is, I was able to recover from the ext2 file system corruption, but was unable to recover from the Reisser file system corruption.
So I didn't find either system to be more reliable than the other. But ext2 was recoverable, and I was unable to recover the Reiser file system.
Again, let me stress, this was under light use. The system was one that I was using for development and experimentation, not one that I did serving from or kept serious data on. So usage patterns wouldn't match a production machine.
I just have one question... (Score:1)
With Red Hat, at least, journaling is a pain. (Score:1, Flamebait)
I don't like the fact that ext3 is now included as a module. The default filesystem driver should be compiled as part of the kernel.
SGI's version of Red Hat is far preferable to Red Hat's own release for this reason.
Now, I must create and maintain an initrd on my IDE system (which was never required before), and I've also been in a crazy situation where attempting to mount an installed filesystem under ext3 caused and Oops, but changing fstab to ext2 was fine.
Down with Red Hat's use of ext3 as a module! Red Hat has never handled journaling in a reasonable manner.
Re:The results are in (Score:1)
Re:As a profane fucking map maker (Score:1)
So, did you not read the article? Or did you just fail to notice the nice bar graphs in the result section?
Re:But Remember (Score:1)
Personally I'd rather have this one:
3:14pm up 321 days, 22:23, 124 users, load average: 0.84, 0.37, 0.56
Re:But Remember (Score:4, Informative)
tune2fs -c 0 -C 0
However, you should also read this, from the tune2fs man page:
You should strongly consider the consequences of disabling mount-count-dependent checking entirely. Bad disk drives, cables, memory, and kernel bugs could all corrupt a filesystem without marking the filesystem dirty or in error. If you are using journaling on your filesystem, your filesystem will never be marked dirty, so it will not normally be checked. A filesystem error detected by the kernel will still force an fsck on the next reboot, but it may already be too late to prevent data loss at that point.
I cannot speak to the inode issue - I've never run into it myself.
Re:But Remember (Score:1)
My linux box (not quite as many files) recovers its ext3 journal seemingly instantly after any crash (oh wait, it doesn't crash) or forced reboot (I'll admit that sometimes it's just easier to reboot the machine than try to restart X when the screensaver won't power my monitor back up)
Re:But Remember (Score:1)
FAT32, on the other hand, will always run a chkdsk whenever it wasn't unmounted cleanly. For a disk with that many small files, it would likely take even longer than a full NTFS chkdsk (whatever the reason is that that's even running), not to mention the horrific slack waste..
Re:Rebooting is easier than killing X ? (Score:1)
Re:Rebooting is easier than killing X ? (Score:1)
oh dear God man! (Score:2)
Oh dear God man, I would never want that! Is it really possible for that to happen to my Linux box with ext3? I'm switching to ReiserFS right away!
Thanks for the warning!
Re:But Remember (Score:1)
Next time, I suggest standing outside with a golf club outstretched to the sky.
Re:No way.. (Score:2)
No mindcraft, please.
Re:No way.. (Score:1)
Live and learn I suppose.
Re:No way.. (Score:2)
If you're stuck doing video work on an NT kernel, (and many people are, since linux is definately behind in this area), do you really want no files bigger than 4gig?
graspee
Re:No way.. (Score:1)
Didn't post said that NTFS can beat both ext3 and ReiserFS? "NTFS can beat both of them!" he said. That's why I posted some proof that NTFS is slower?
Obviously, HE cares, and I was replying to him.
Now, can anybody explain me how that was "offtopic"? The original poster was talking about NTFS's speed, and so was I.
Re:Benchmarks (Score:2)
First problem: They weren't trying to make anyone look good, this was a 3rd party test.
Second: Why would they try to make anyone look good, neither of the "products" tested are for profit projects. They have nothing to gain from false benchmarks.
Third: How could that be taken as Linux bashing? Both filesystems are linux only, they aren't being compared to anything non-linux nor are you comparing them to anything non-linux.
Please read both the article and posting before you take the "How to post" (early and often) of slashdot very seriously.
Re:Benchmarks (Score:1)
I agree with the rest of your points though
Re:Benchmarks (Score:1, Informative)
Since ext3fs writes in a way compatible with ext2fs, shouldn't you get (at least somewhat close to) the same speed reading it nomatter how it was written?
Re:Ext3 is fine... (Score:1)
Re:Why not ext2?? (Score:1)