Reiser4 Benchmarks

A user writes "Hans Reiser has benchmarked Reiser4 against ext3 and Reiserfs 3. Reiser4 turns out to be way faster than V3, and for ext3, why don't you check out the results yourself ? Hans Reiser states, "these benchmarks mean to me that our performance is now good enough to ship V4 to users", and he will be probably sending in a patch within the next couple of weeks to be included in the 2.6/2.5 kernel."

Reliability

[prestwich]

My one concern is reliability and recovery from failure; I've had a few cases where my belief in ReiserFS has been questioned; however I can't get Ext3 to build on larger than 500GB arrays.

At this point I'd happily choose based on reliability/recoverability/stability not raw speed.

Conversion?

[avalys]

Does anyone know if there will be a conversion utility available - i.e, to convert ReiserFS v3 partitions to v4?

Honest Portability Question

[jstockdale]

I am curious as to whether there are any projects to port Reiser4 to *BSD, particularly FreeBSD 5.x. Does anyone have any thoughts on how difficult a port might be? Can somone more versed in filesystems on *nix enlighten me as to the implimentation differences?

Re:ok

[ElGuapoGolf]

You'll probably have to compile it in yourself for now.

RH probably will include it in the future, but probably won't give you the option to install on it without jumping thru major hoops.

RH seems to suffer from a big case of "not-invented-here-itis", and RH users sometimes suffer for it. Not having ReiserFS is one way in which they do.

Which to choose for DBs?

[Openadvocate]

I realise that it is a bit early to adopt V4, but stable issues aside, which filesystem would YOU choose to for database volumes for fx. Oracle or MySQL?

XFS?

[leoboiko]

How does it compare against everyone's favorite, XFS?

but will it make it

[DemiKnute]

So he's submitting it to 2.6, but what are the chances it'll get submitted? Isn't this what caused all of Reiser's bitching a couple of years ago? He waited to long to get RFS into the kernel and ran into the feature freeze, and then pitched a hissy fit.

Filesystems for the laptop user?

[niko9]

Anybody know what, if any, features are being added for the laptop user? Last time a checked, journaled filesystems, like ext3, were generally a no-no if you wanted you battery to last.

Maybe a filesystem just for laptop/tablet pc users?

Which is best?

[Dodge This]

OK so there seems to have been a lot of Reiser flaming going on here, so what would people recommend? Taking into account Speed, Reliability and Compatability.

I know a lot of people will pull their hair out when they hear this, but: Speed is my primary concern. On long compiles of new programs or kernels for example the speed difference on a good FS can be important. I'm not saying that I'm willing to have a FS that corrupts every last file and directory, only that given two FSs which both have seemingly similar stability I would prefer the speed boost.

I have tried one or two of the FSs but I haven't used them for any length of time to be able to compare one against another.

ext2?

[Coneasfast]

I know ext2 isn't a journal fs, but it would still be interesting to see a direct comparison again reiser4.

Re:Reliability

[cvd6262]

We had a massive failure of our primary database server while I was out of the country. (Trust me, nothing puts a damper on your day more than having one of your techs call you at midnight from 7,000 miles away.) I blame Reiser. Not because it caused the outage (it was hardware), but because it was so good, it made us a bit lax.

We're just a small grant lab at a university, so it's not like this was a corporate system or anything, and there had been hardware problems before. Given that most of the people are not techies, they did not know how to ssh in and shutdown -r now, so they would just hit the reset button whenever they thought something was wrong and I wasn't around.

Anyway, because of Reiser's journalling, the system would come right back up after a forced reboot. I think that the guys in the lab cut the power a couple of times to many and the hard drive just gave out.

By the way, I just had a tech install a new drive, and Debian base with ssh. I knew the password he would use for root, and I was able to rebuild the entire system and restored 250,000 records in half a day.... From North Africa.

Try that with a non-*nix.

In other news...

[wirelessbuzzers]

Apple benchmarked their new G6 processor against the latest 10 GHz Pentium V. They say that despite its lower clock speed, it runs their suite of PhotoShop 8 filters almost four time faster than the Pentium.

Seriously, Hans Reiser is benchmarking his own file system, and he's using benchmarks that make his system look good. Like the SpriteLFS, his filesystem has a log structure for sequential writing, which makes it look really good in tests like he performed where you write the files once.

Compare a database load, where you write small chunks of big files all the time. Without the repacker (like the cleaner in LFS), the disk becomes horribly fragmented. With the repacker, you have to include the slowdown of this background process defragging your hard disk. Ick.

I'll trust his benchmarks when he presents a final, stable release, with the repacker on, and tests it under workloads such as would be encountered on a server. I might use it on my homebox even if it sucks on a server, but it would be nice to know that he considers his structure's impact on other workloads.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Should I bother?

[anthony_dipierro]

I'm not putting thousands of files in a single directory. I'm not using tons of small files, and my hard drives are more than big enough to hold my data. Is there any reason to use reiserfs instead of ext3?

Re:Computer's names translation

[hansreiser]

These are the names of the two dogs that were sent into outerspace by Russia.

Re:Which to choose for DBs?

[DarkVein]

I'd go with XFS. I've never heard of corruption from regular use, while I've heard far too many horror stories about ReiserFS. Aside from the ineptitude at reliability, Reiser's one of the most unpleasant characters I've ever run into in Free/Open software. Stallman at least has reason for what he does. Reiser sends off flames to dev mailing lists (sometimes legally) threatening people for removing the kernel message and command line commercials from his code. Usually, he fires off fiction about "the next version of GPL" that someone from the FSF invariable shoots down. Bleh.

If you ever get a soap commercial in the output of dmesg, you'll know who to blame.

Upgrading...

[SealBeater]

What about upgradablity? The nice thing about ext3 is that upgrading to a new
version is a reboot away. With ReiserFS, you had to re-format the drive, in
order to upgrade. Has this changed?

SealBeater

Re:Reliability

[Billings]

Yeah, you won't lose files, but you'll lose data. It's been noted elsewhere in this article's comments in more technical jargon, but it is a known flaw in ReiserFS that blocks of data can be written to flat out wrong areas. As an example, I had an outage while I was working with my config files and running an apt-get update;apt-get dist-upgrade. Reiser then managed to write the middle of a debian package file to whatever config file I was working with.

Had me confused to hell until I saw a newsgroup discussion that mentioned the exact problem I was having. Does Hans Reiser know about this problem? Oh, yeah. He does. Is he concerned about it? No, he's not. In his own words he's not. And ReiserFS fails silently; you'll never know until you find it.

When I setup ReiserFS on my machine, I was aware of similar complaints, but I dismissed them as fear of trying something unproven. And I was happy with ReiserFS for quite awhile, because I never saw anything wrong (unlike ext2/3). But I really can't support a FS that has these kinds of data integrity issues if the team has that kind of attitude towards them.

some questions abou ReiserFS

[Pflipp]

I have some questions abou ReiserFS and was wondering whether someone out there would be able to answer them.

First off, there's this stuff with ReiserFS storing fine-grained data. Does this imply that using ReiserFS (v3 or v4) directly as a database would be efficient? I know RFS doesn't have Relational features, but these might very easily be implemented in userspace if you can store e.g.: .../customers/0001/name .../customers/0001/phone .../customers/0001/agent -> .../personnel/0021
(...etc.)

Am I losing this or getting this???

My other question was about this metadata-as-file thing. Hans can implement whatever he wants, but it just so appears that Linux behaves like Unix. I've just made a ReiserFS partition to check, and there's no way I can "touch foo; ls foo/" to see e.g. permissions etc.

Now I'm aware that this might be v4 stuff, but I wonder if anything of this is ever to be seen back in Linux userland? E.g., will it be possible for projects to use ReiserFS to change the paradigm used for metadata using a straight Linux kernel?

See, from time to time I just happen to be quite impressed with the "everything is a file" applied to metadata, and I hope we can make the shift to this future one day, and finally get rid of file extensions, MIME guesses and app association registries in Linux, and store this stuff in metadata space.

Re:but will it make it

[Nothinman]

not many people are going to use 2.6.0, and I'm sure V4 will make it into 2.6.1. The wait will be what, like a month?

The point is that unless reiser4 is 100% self contained and stable it shouldn't make it into 2.6 at all now because there's been a feature freeze and a new filesystem and anything else it adds to the kernel are features. Hans waited too long, again, and technically reiser4 shouldn't be included in the standard/Linus kernel until 2.7 now.

XFS wasn't allowed to be included in 2.4 officially because of those reasons but the SGI developers didn't cry about it, they just kept their patches up to date and waited for 2.5 to start so they could get included.

If the filesystem is actually stable and has real benefits over ext3, XFS, JFS, etc it'll probably be patched into distros like RedHat anyway so it's not a huge deal.

Re:Reliability

[SaDan]

The root partition is on a 20gig drive that is not part of the RAID array. It ONLY has the operating system stored on it.

I'll say it another way... We don't boot from the array, and we don't store any operating system information on the array.

Guess I should have mentioned that earlier, although I didn't consider that my decisions on how to lay out the filesystems would come under fire here.

Re:Reliability

[shokk]

When you're talking about that much data, backups become something you're only going to pull out for catastrophic recovery. Something like snapshots become more and more important. Are any of the upcoming Linux filesystems going to address this? I imagine that something like that would begin to take customers away from EMC and Network Appliance, and would put high reliability in the hands of smaller organizations.

Re:Conversion?

[Jah-Wren Ryel]

Hans - How is scalability? SGI seems to think that ReiserFS doesn't scale at all with multiple CPUs, unlike ext2. At least according to the paper they presented at the Ottawa linux symposium last month:

(see page 9 of this PDF for the graph) [linuxsymposium.org]

The implication is a lack of fine-grain locking. Does this new all-atomic, all-the-time implementation automagically bring better locking too?

Re:Conversion?

[Anonymous Coward]

it creates a loop back mounted target filesystem inside a file inside the source filesystem

It sure sounds like a nice idea, but it sounds like something that would require the source filesystem to support sparse files as well as the FIBMAP ioctl. With those features in place, and enough free space for extra metadata, it should be possible. I'm curious to how they actually reshufle the sectors, I believe they will need extra metadata beyond what is used by the filesystems to do it in a safe way.

Re:Reliability

[GigsVT]

I'm using ext3 on a 1.2 TB array, a 2.0 TB array, a 1.9TB array, a 550GB array, another 2.0 TB array, and a 1.0 TB array (If I remembered them all right). They are all ATA or SATA arrays, some straight 3ware, some 3ware with software RAID on top to bind multiple cards together, and some are AC&C ATA-SCSI boxes.

Anyway, haven't had any trouble, which is more than I can say for when I last tried Reiser. It couldn't handle files larger than 2GB about a year ago, that's since been fixed. It also caused some strange stability problems.

The 1.9 TB and the 1.0TB array had XFS on it for a while, I'm not sure if I ever got around to changing the 1.9 to ext3, but XFS (linux) has also performed flawlessly. Of course our IRIX boxes also run XFS flawlessly on some smaller legacy SCSI-only arrays.

Re:Patch for FreeBSD request??

[Billly Gates]

Hans. Thanks for all your cool fs work. I have a request. Would you be interested in porting Reiser to other os's and maybe creating a bsd licensed version for them?

I like the UFS2 FS for FreeBSD. Its stable but a little sluggish. I think it would be cool to have internal competition but the MS GPL == viral crap has made a dent into the BSD developers. They fear linking to anything gpl would make their kernel gpl as well.

Anyway this is just a pretty please with a cherry on top. Especially since you are being paid for by grants from DARPA who use Solaris, Linux, FreeBSD, and every Unix os under the sun.

Re:XFS is not mature... not in linux anyway

[Anonymous Coward]

For my usage patterns/needs and in my experience; I have found it far more difficult to produce catastrophic failure in XFS.

Anything referred to as Irix-isms is Irix has a proper implementation, and Linux needs to do catch up.

As far as problems go, I have seen from time to time a few errata/bugs associated with release code, but I also see quite a bit of bizarre errata/bugs for Reiser and EXT3. Some of which are not corner cases but foolish oversights.

I have to say being multidextrous with various *nix, and being agnostic towards OSes - my encounters with FreeBSD shows that filesystems don't have to be a kludged mess.

I'm looking forward to 2.6 for a number or reasons (and hope to see a lot of thrashing eliminated that has become linux's trademark on even bigmem systems), but for XFS being bolted in as part of the kernel.

Read the EXT3 mailing lists or check out a few newsgroups. Its not without problems. And as far as Reiser goes, it may be faster in some cases, but I dont take R3 very seriously, especially after all the whining hans has done towards the linux kernel and towards compilers. Sure, Linux has flaws and RedHat's compilers produced suspect code, but filesystem code isnt a place to be messing around with code that is easily mangled.

XFS and somewhat JFS are in my estimation the only serious filesystems for Linux.

Re:Reliability

[jdew]

ick, 3ware... gotta hate raid cards with no onboard cache, and do retarded tricks with the drives cache for performance :(

Features

[einhverfr]

ReiserFS is really good at some things (and I actually recommend it for some things) but for my main system filesystems, I find that Reiserfs lacks some of the things that EXT2/3 has that are very useful--

File attributes including things like "Append Only" although these can also be ways of screwing up your system (who'da thunk Qmail doesn't like append-only log files?).... See man chattr for more info....

So I am happy to use it for directories where I don't have special needs for the filesystem, but for others, I have to still use ext3.

Re:some questions abou ReiserFS

[hansreiser]

I get reports (not verified by me) that ReiserFS V3 is an order of magnitude faster when used as a backend for an XML database than relational databases that were tried. So, if your data happens to have a hierarchical structure, or, you can put it into one, then you are likely to get a performance gain. If your data does not have a hierarchical structure, then you need to wait for V6 where we plan to expand the semantics.

If you want to be able to "cat filenameX/..owner" to see who owns "filenameX", you need to use V4.

Re:Reliability

[Anonymous Coward]

Sorry, but: a) which options to use to also add data journaling to reiserfs v3, and since when this feature is avalable? b) you said that if one is writing in a middle of crash the file can contains garbage, but I wonder, why that also happens in file which aren't written at all. E.g. /etc/modules.conf, is not a file generally under writing, unless user is editing it by hand. So why it contained garbage (but garbage taken from system log files) after a power-loss (it was not edited/touched since months)? And ditto sometimes happened for other system files (at least files you find because of system malfunctioning). Thanks. Bye.

Reboot requirement?

[Anonymous Coward]

An argument has been going on at one of the distro mailing lists. It basically boils down to this:

ReiserFS, currently, requires a reboot after the filesystem is installed, but prior to completion of the installation of the distro, does it not?

I know that this was a requirement earlier. Whether this was "fixed", or the reboot requirement was removed in a subsequent update, I do not know. But I do remember reading this at the Reiser site, and elsewhere. I also remember reading that the reboot requirement would be removed with Reiser4. Whether this reboot requirement has been removed somewhere in the 3. series, I do not know.

The distro authors have justified their non-reboot installation by saying that it is not required under Suse either. I remember reading elsewhere, though I can't remember where, that Suse had developed specific patches to Reiser 3. series to eliminate the reboot requirement, and a comment was made as to a hat tipping or something similar, to the coding abilities of Suse.

Since I can't find the above information, and can't remember where I read it, the concerns have been dismissed. But the reboot issue has cropped up again on the list.

Other than Suse, does the current, ReiserFS, the versions prior to Reiser4, require a reboot? If not, starting at which dot release has the reboot requirement been removed?

And thanks for a great filesystem! I use it for everything.

As a side note, one or several distros require to dump everything into /, without separate partitions. I attempted to use the tools to check and repair the filesystem. If you must mount / to run the computer, how do you check the / filesystem if you can't unmount it while running it? Other partitions are easy to unmount and check. How do you do it with / ?