Reiser4 Benchmarks 414
A user writes "Hans Reiser has benchmarked Reiser4 against ext3 and Reiserfs 3. Reiser4 turns out to be way faster than V3, and for ext3, why don't you check out the results yourself ? Hans Reiser states, "these benchmarks mean to me that our performance is now good enough to ship V4 to users", and he will be probably sending in a patch within the next couple of weeks to be included in the 2.6/2.5 kernel."
Re:Conversion? (Score:3, Informative)
Computer's names translation (Score:5, Informative)
Wonder what naming system they're using. I use names from Alice in Wonderland.
Re:wait! (Score:3, Informative)
Yep. Testing a beta filesystem is something you do on a spare machine that you are not using at the moment. It would be really cool to help test ReiserFS out, but I only have one machine and can't risk losing all of my work!
Re:Honest Portability Question (Score:1, Informative)
Re:ok (Score:5, Informative)
Re:Reliability (Score:5, Informative)
The drivers are crap, and the box dies about every week or so. Haven't lost a single file yet, and we're at 91% filesystem useage (millions of files).
The / filesystem is ext3. It's about 20gigs, and has had to have files restored several times.
I have a lot of confidence in ReiserFS, after seeing the incredible amount of abuse on this one particular machine. I have run ReiserFS for quite a while now (ever since it was part of the kernel) for all of my home systems, and have never had a single issue with those filesystems.
Looking forward to what ReiserFS4 will bring.
Re:Computer's names translation (Score:5, Informative)
Quote from first google search result: "Belka("Squirrel")and Strelka("Little Arrow") were launched into space on board Sputnik 5 on August 19, 1960. They were accompanied on their historic flight by 40 mice, 2 rats and a number of plants. Belka and Strelka were safely recovered after spending a day in orbit. Strelka eventually gave birth to a litter of 6 healthy puppies, one of which was given to President Kennedy as a gift."
"but you won't need to fsck" (Score:2, Informative)
Once you realize that this maintains your filesystem structure but turns your files into Swiss cheese (this happened to me several times) you'll switch to ext3 or XFS.
Since reiserfs continuously rebalances the block tree, files that are only open for reading can be trashed by having a block exchanged with the last file written before a crash.
Hans is recklessly pursuing optimization of one aspect of filesystem performance. I honestly believe DARPA should reconsider funding his current track of work.
Re:So how much better is this really? (Score:3, Informative)
The columns marked B/A and C/A are the performance ratios of ext3 data journaling and ext3 to reiserV4 respectively. A ratio greater than 1 means more time was needed for the operation than reiserV4 while a ratio less than 1 means faster performance.
Re:I don't understand the statistics (Score:4, Informative)
B. ext3 data journalling
C. ext3
That's how the filesystems are lettered. In the column headers, you see this:
A B/A C/A
So it looks like the first column is the actual time for reiser4 (A), the second column is the ratio between ext3-j (B) and reiser4 (A) which is B divided by A, and the third is ext3 divided by reiser4. So if the number is > 1 (red), it means reiser4 took less time and might be "better". If the number is 0 (green) it means reiser took MORE time.
I think.
ReiserFS == BrokenFS (Score:1, Informative)
If you want to use some serious FileSystem for Linux then continue using EXT2/EXT3 or switch to XFS. everything else is simply a risk.
You may think that I may some standart troll or something but then you should ask yourself why there was a need to rewrite ReiserFS at all. Because there was a requirement to get rid of all these known issues.
Re:Reiser4? Competition? (Score:3, Informative)
About reiser4 (Score:5, Informative)
Basically, reiser4 is optimized for the case where you unpack a large tarball, say the Linux kernel, and have enough memory to hold it all in cache, which is true for most of us these days. reiser4 will then choose the optimal disk layout for these files and flush them to disk.
Hans also has aspects of a log structured file system in reiser4, which means you don't write to the file, you write to a log file which basically encompasses the whole disk. The up side is that you mostly write linearly, the down side is that the files get badly fragmented if they are updated at all. Most files are not updated, just written once at installation of the package. The files that are updated frequently tend to be source code from CVS, which are small enough to fit in memory completely and have reiser4 choose an optimal disk layout again.
The case where this model completely sucks is the case where you update many portions of a large file. For example, running an SQL database with files on a reiser4 file system as backend, or maybe a DNS server with DDNS, or a berkeley db backend for Postfix or qmail to keep the SMTP AUTH users or something. Also, log files will probably be badly fragmented.
Hans proposes to have something like a transparent defragmenter running in the background, which he calls "repacker". This would run in the kernel space, as part of the file system, and defragment badly fragmented files that are accessed frequently. This would solve most of the down sides of his approach, but this repacker is not finished yet.
My personal view of reiser4 is: it looks like it is optimized to perform well in benchmarks. It tries to be fastest for updating databases, but buys the performance by being slower when reading the data afterwards. The critical question is whether the repacker can alleviate these concerns, and as long as it is not finished, reiser4 is basically out of the question except for a little testing here and there. I reckon reiser4 would be a great filesystem for keeping your mozilla and gcc CVS checkout handy. But until the repacker is done, I will not even use it for testing, because the repacker really is the crucial component that makes or breaks this.
By the way: my previous experiences with reiserfs were less than stellar. Some people call it shredderfs instead. The main complaint with reiserfs is and always was that the fsck is not nearly as trustworthy or stable as the one from ext2/ext3. So even if I use reiserfs at all, it's only for data I can afford to lose completely, like my CVS checkouts or the squid cache directories or something like that.
The benchmarks do look good though, and I am glad that at least someone is still trying major innovations in this area. Since most Unix vendors or divisions are no longer profit centers, file system innovations have largely stalled or moved to specialized companies who regard them as proprietary (Veritas) instead of releasing them as free software like IBM and SGI did.
Re:ok (Score:3, Informative)
rh supports reiser just fine - they simply hide the option, probably because they feel ext3 is a safer choice.
Re:It may be interesting to know... (Score:2, Informative)
XFS is especially good in dealing with large(and when I say large I mean *LARGE*) files, and IIRC, the all idea behind the Reiser filesystem is to deal with small data objects.
Anyway, it'd be interesting to see such a benchmark as XFS has the reputation of one of the best(the best???) fs to be used with high hard-disk I/O-s and even higher system loads.....
Re:Conversion? (Score:2, Informative)
Re:Should I bother? (Score:2, Informative)
- plugin-based architecture
- new semantics that allow files to also be directories
When was the last time you remember hearing about new features in a filesystem? Journaling was the last one I can remember and that's a few years old now.Re:Conversion? (Score:5, Informative)
However, there is a tool called convertfs (as well as tar) which can convert V3 to V4. It can also convert ext2 to V3 or V4 or V3 or V4 to ext2. It is pretty clever (and written by someone outside our team), in that it creates a loop back mounted target filesystem inside a file inside the source filesystem, copies everything from the source to the target, and then reshuffles the blocks of the file so that they are at the offsets on the device that they were at within the file.
Re:Which to choose for DBs? (Score:5, Informative)
Re:I don't understand the statistics (Score:5, Informative)
I would have fixed this before posting to lkml, but I had to catch a plane, sorry about that.
Re:Which is best? (Score:2, Informative)
If that's a little extreme, XFS is probably the way to go. Ext3 is good for /, since you still have an ext2 way out, if you need it. ReiserFS works well with small files -- it makes a great proxy cache. XFS is good all around.
Re:linus (Score:2, Informative)
Re:"but you won't need to fsck" (Score:5, Informative)
That was V3. V4 is an atomic filesystem, which means that every filesystem operation is performed as a fully atomic transaction. This is more secure than the guarantees of data journaling, as data journaling doesn't necessarily guarantee that the write will complete.
The reason we are able to do this in V4 but not in V3 is that V4 uses what I call wandering logs. With wandering logs, instead of copying data first to the journal, and then after commit copying it from the journal to the rest of the filesystem, (thereby writing the data twice), we just change our definition of where the journal is. I don't think that data journaling is worth going half-speed for most users. With V4, we not only don't go half-speed, we go faster than V3 ever went.
For more details, please take a look at http:www.namesys.com/v4/v4.html [namesys.com]
Re:Gee Re:Reliability (Score:3, Informative)
Re:Reliability (Score:3, Informative)
First I don't get what you mean by "huge amount of stuff in the partition"... In no place does he actually say that _anything_ is being stored on the root partition (we can assume it's an OS only partition).
It's probably a server with an internal 20GB drive connected to the 1TB raid. If you're suggesting the 20GB is a partition off the raid, well then you have no clue how these things work.
Re:About reiser4 (Score:5, Informative)
The difference between us and an LSF is that we perform well BEFORE you run the repacker, and we merely perform even better after you run it. LSF's required that you run the repacker to get good read performance, we don't. V4 kicks V3's butt without the repacker by a lot (due to dancing trees, allocate on flush, extents, and ending the use of BLOBS, among other things). With the repacker, it will just kick it harder.
Our approach synthesizes a lot of approaches, rather than considering one technique to be the answer to everything. This makes our performance more robust, as the different approaches each cover over each other's lackings. There are some situations in which using a repacker is higher performance than making lots of little changes while constantly maintaining optimal allocation of files.
The repacker will be ready in a few weeks.
Re:Which to choose for DBs? (Score:3, Informative)
1. If you're using a filesystem for databases, then the chances are you're keeping other stuff besides the database on them. Otherwise, you'd use a raw partition.
2. I believe that by default data changes to files aren't journalled in either of the systems that were being compared; only filesystem metadata changes. This doesn't cost much in terms of resources, particularly if the metadata rarely changes (as I suspect it doesn't on oracle data files, which I assume are fixed size allocations that are opened when the server is started and never closed...).
Re:Reliability (Score:5, Informative)
ReiserFS does have speed as a goal; however, with ReiserFS 4, all filesystem operations are now atomic, which is functionally equivalent to having full data (not just metadata) journalling.
In addition, having the fastest CPU in the world won't make ext[23] better at things for which ReiserFS is fast.
CPU speeds are increasing. Storage space is increasing. RAM is cheap.
However, none of that equates to "disks are fast". Having a fast CPU with a slow filesystem is like having a gigabit LAN connected to the Internet via dialup. Sure, internally you're quite good, but throughput will still suck eggs.
The fact of the matter is, it is easier to have no clue what you're talking about than to read a little bit before posting.
Re:URL? (Score:1, Informative)
http://tzukanov.narod.ru/convertfs/ [narod.ru]
But it looks outdated compared to what Mr. Reiser has described.
Right. (Score:2, Informative)
But let's wait and see how fast and stable reiser4 is once it matures into the stock kernel.
XFS is not mature... not in linux anyway (Score:3, Informative)
XFS is still not a fully linux-native fs, it is IRIX code pasted into Linux through a special translation layer. And thats where most of the problems arise, as it has to translate IRIX-isms to Linux-isms.
Re:Colors (Score:1, Informative)
The script that creates the comparison tables divides the other filesystems by the base filesystem. The problem is that Reiser4 was used as the base filesystem in one of the benchmarks, but not the other. So in one benchmark, green is good, and the other benchmark, red is good. I would have fixed this before posting to lkml, but I had to catch a plane, sorry about that.
Re:Reiser4? Competition? (Score:3, Informative)
I think they are using the Plan 9 network filesystem protocol (9P2000), not the Plan 9 local filesystem (which isn't anything special AFAIK).
Re:Warning (Score:2, Informative)
Red Hat is providing an ABI guarantee that Linux normally doesn't provide though.
Specifically, other vendors (Oracle, Veritas, IBM, etc) have kernel modules compiled specifically for the Red Hat enterprise 2.4.9 kernel.
Red Hat has committed to keeping the source and binary interfaces stable and compatible for the duration of the 5 year life cycle of Red Hat Linux.
Updating the kernel would break all of that.
BTW, yesterday Red Hat released a beta of the RHEL 3.0
Re:Reliability (Score:4, Informative)
RAID5 would give you a 1.75TB partition with the equivalent space of one drive reserved for fault tolerance. My preference would be RAID5+Spare with that much space which would give you a 1.5TB partition with 250GB reserved for fault tolerance and one hot spare drive in case one of the others fails. With that setup, you're okay until you lose three drives.
Jason
Re:Out of the server world.... (Score:2, Informative)
Re:Conversion? (Score:3, Informative)
Re:Reliability (Score:2, Informative)
RAID5 stores N drives' worth of data on N + 1 drives, for any N >= 2. For each range of sectors, one of the N + 1 drives is used for parity of the other drives' data (this is equivalent to reserving one drive for parity, but produces less contention).
Your spare machine is more resistant to drive failure (you don't lose data unless corresponding drives in each machine fail) but you're paying for N * 2 drives (like RAID1) plus another computer and enough bandwidth to keep them in sync.
Broken controller? Viruses? If your computers are broken, nothing can save you, certainly not buying more of them.
Re:Reliability (Score:5, Informative)
Be warned though, that all fixed location journals double the transfer time cost of performing writes because the data must first be written to the journal, and then written somewhere else. This is why we don't make data journaling the default in v3. Trust me, full data journaling would have been far easier to code first than meta data journaling, but it isn't in the interest of the 'average' user.
Now V4 is an atomic filesystem, which is much better than data journaling, because it means that all filesystem operations are performed fully atomically. Your write syscall either fully happens or it does not. Applications can have multiple filesystem operations performed atomically. We do this without writing the data twice through use of a technique called wandering logs, which I describe in a posting below (and on our website).
Re:Features (Score:4, Informative)
Are there features other than attributes provided by ext[23] which ReiserFS can't provide?
Regarding the attributes, ReiserFS can't do the following "useful" things:
- Excluding a single file from access-time logging.
- Append-only would be useful in certain cases.
- 'c' (compress file contents) -- Ext[23] doesn't do this yet, and ReiserFS doesn't either, but they both will support it later.
- 'D' (dirsync) attribute -- Other than ensuring that filesystem metadata operations are completed successfully before proceeding, I can't imagine a beneficial use of this attribute. ReiserFS already journals all metadata, and V4 journals everything (and does it faster).
- 'd' (do not back up using dump) -- AFAIK you can't use dump to back up a ReiserFS partition anyways, but tar still exists, and I'm sure there are wrapper scripts for the die-hard dump users.
- 'E' (error in compression) doesn't seem to apply to general (current) use of ext[23], and doesn't have an analogue in ReiserFS.
- 'I' (hash-indexed directory) -- ReiserFS' directory algorithms are much (much!) faster than ext[23]. ReiserFS will eventually support directory plugins, which allow one to set a per-directory hashing scheme. I note that ext[23] doesn't allow modification of this attribute, so I'm not sure if/how it allows you to choose a hashing scheme.
- Immutable ('i') bit -- This might be useful in some circumstances, such as a setuid binary. Not supported by ReiserFS.
- The 'j' option is meant to get around the fact that ext[23] only journals metadata unless mounted with "data=journal". As mentioned elsewhere, ReiserFS' long-term goal is to provide fully atomic file operations, which provides the benefits of data journalling with none of the burdens. Thus, ReiserFS V3 doesn't support this attribute, as the time spent in adding this feature would have been better spent working on V4. Note that with V4 all operations are atomic, which has the effect of turning this flag on for every file.
- 's' (zero file contents upon delete) -- ReiserFS doesn't support this.
- 'S' (write file synchronously) is likely used in cases where preserving file integrity is important. ReiserFS' journalling (especially in V4) provides this "for free".
- 'T' (top of directory heirarchy) is used to get around an ext[23] weakness. I don't recall details, but I believe it's something related to preventing fragmentation and grouping similar files together for enhanced performance. ReiserFS (especially V4) is already quite fast, and with code-level optimizations will likely get faster.
- 't' (don't merge tails) does not apply to current ext3 code, as tails aren't yet merged anyways. This will be used as a flag to write the file to disk in such a way as to be compatible with LILO and other boot loaders. At present I'm not aware of any special issues with ReiserFS' tail packing (which does exist, by the way) with respect to LILO. Note that a ReiserFS partition can be mounted with the "notail" option if you want, which is feature-wise equal to ext[23], which don't seem to provide tail packing at all right now (and if they do, someone needs to update the chattr man page).
- 'u' (back up upon deletion) -- ReiserFS doesn't support this.
- 'X' and 'Z' relate to compression, and do not apply to ext[23] or ReiserFS at the moment. ReiserFS will eventually support compression plugins, as well as encryption plugins.
So of the attributes, the only ones which might actually be of some use in the context of ReiserFS would be exclusion from atime logging, append-only, the immutable bit, zero-contents-upon-delete (although future crypto plugins will destroy the need for this feature), and back up upon deletion.
Anything I've missed?
Re:Gee Re:Reliability (Score:2, Informative)
Re:reiserfs 3.6 is also very fast (Score:2, Informative)
I've been doing similar testing and have seen significant differences in numbers based on the io load / pattern. Each FS has its own strengths + weaknesses.
Was it: random or sequential? big files or small files? consecutive io or single reader/writer?
This stuff makes a BIG difference in the relvance of those numbers. Yuo probably know that if yu're doing testing, but it'w worth psting for those wh may not.