Ext4 Advances As Interim Step To Btrfs 510
Heise.de's Kernel Log has a look at the ext4 filesystem as Linus Torvalds has integrated a large collection of patches for it into the kernel main branch. "This signals that with the next kernel version 2.6.28, the successor to ext3 will finally leave behind its 'hot' development phase." The article notes that ext4 developer Theodore Ts'o (tytso) is in favor of ultimately moving Linux to a modern, "next-generation" file system. His preferred choice is btrfs, and Heise notes an email Ts'o sent to the Linux Kernel Mailing List a week back positioning ext4 as a bridge to btrfs.
Re:Why not ZFS? (Score:5, Informative)
Re:Why not ZFS? (Score:5, Informative)
The ZFS developers specifically wanted the open sourced code to be under a GPL incompatible license, hence it has been released under CDDL (there was a interview with the Sun open source rep, can someone provide info/links about this). So ZFS cannot be part of the kernel, but there is a FUSE port of ZFS and according to http://en.wikipedia.org/wiki/ZFS#Linux Sun is investigating a Linux port, so there may be something good coming
Re:Why not ZFS? (Score:5, Informative)
I don't know about the patents, but the current major obstacle is the license. ZFS, as part of the OpenSolaris kernel, is available under the CDDL. The CDDL is incompatible with the GPL, ruling out ZFS inclusion directly in the Linux kernel. Sun has hinted that they could dual license the Solaris kernel under CDDL and GPL, but that hasn't happened yet. Small parts of the ZFS filesystem code have been GPLed so they could be added to grub to support booting ZFS root filesystems.
There is a userspace port of the ZFS code and utilities which avoids the license problem by using FUSE to separate the filesystem code into a separate process: ZFS-FUSE [blogspot.com].
If Sun were to ever dual-license ZFS, the ZFS-FUSE codebase would be a good place to start for porting the code to direct kernel inclusion. (Note: Sun, via their subsidiary, Cluster File Systems, now employes the author of ZFS-FUSE to use his port as an optional backend for the Lustre file system.)
Re:What I'd like (Score:2, Informative)
wayback [sourceforge.net], copyfs [n0x.org], and ext3cow [ext3cow.com] are all fairly stable versioning filesystems for linux. I'm not sure if they let you stop non-root users from getting old versions, but I don't see why you'd want people to have to ask an admin to get old versions of their files?
Re:Why not ZFS? (Score:5, Informative)
It wouldn't be possible to duplicate RAID-Z with LVM. Other features of ZFS are very handy, but RAID-Z is by far my favorite. Same storage density as RAID 5 but without the horrible write performance. RAID-Z uses copy-on-write to avoid RAID 5's required read for every non-cached write.
Being able to create filesystems just as easily as creating directories is quite handy as well, though. IIRC, the filesystem sizes in ZFS are controlled by a quota style system. That is much simpler than shrinking an LV (if your filesystem supports shrinking), then adding a new LV, and then creating a filesystem. I don't know about you, but I am always a bit nervous when I have to resize an LV.
Re:Why not ZFS? (Score:4, Informative)
Sun has some patents on ZFS; the CDDL grants a license to these patents if you're deriving from the original ZFS source, but then you can't link it to linux.
FWIW, I doubt ZFS-FUSE would be a good place to start - FUSE is totally different from Linux's actual vfs layer, after all.
Re:Why not ZFS? (Score:5, Informative)
Rather, GPL is incompatible with anything else that can't be re-licensed as GPL, and that includes GPL v2 and v3, which can't even be mixed among themselves. May first we clear that mess, right ?
ZFS is present in both Mac OSX and FreeBSD, thank you! They have no license issues whatsoever.
Re:Why not ZFS? (Score:1, Informative)
that includes GPL v2 and v3, which can't even be mixed among themselves.
[citation needed] My memory of the GPL compatibility matrix doesn't support that, but it may be true for GPLv2-only rather than the ordinary GPLv2-or-any-later-version (which does cover the Linux kernel).
That said, I thought Sun was trying to GPLv3 ZFS? It's certainly a lovely filesystem, though I guess they're fighting with NetApp over patents on it.
Re:Why not ZFS? (Score:3, Informative)
I'm confused: if we ask people why not run ZFS using FUSE, they reply because it's slow (I'm assuming it's possible to load ZFS at boot time using an initrd). And if we ask people which is better monolithic or microkernel, they reply microkernel. But ZFS using FUSE would be like a microkernel driver, so which is it?
If you want a blazingly fast file system.... (Score:3, Informative)
Then look no farther then NSS [wikipedia.org] ( Novell Storage Services ).
It is Open Source, you get the full source if you download SLES.
It has more of the desired features [wikipedia.org] then anything else on the block right now.
This should be the default file system for Linux. It has years of very heavy duty R&D behind it, it is pretty much completely de-bugged and ready to rock.
Re:Why not ZFS? (Score:3, Informative)
No, it wouldn't. A microkernel loads modules into the kernel space. You're talking about running in user space. So when an application makes a system call, the kernel has to translate it to the FUSE layer into user space. So there's an extra layer consuming time. On top of that, kernel space isn't generally swapped out, but user space can be. Obviously it should never happen, but wouldn't it suck if your disk driver was swapped out?
See the diagram at the bottom of this page: http://fuse.sourceforge.net/ [sourceforge.net]
Also, ZFS (like ReiserFS) handles its metadata differently from ext3, so you have to translate the differences between the virtual file system and ZFS. This is why writes are significantly slower. Reads are not so bad. The NFS penalty would be huge. See http://www.linux.com/feature/138452 [linux.com]
Re:Why not ZFS? (Score:5, Informative)
> Rather, GPL is incompatible with anything else that can't be re-licensed as GPL, and
> that includes GPL v2 and v3, which can't even be mixed among themselves.
Saying that GPLv2 and GPLv3 "can't even be mixed among themselves" is wrong and
misleading.
Section 14 of GPLv2 specifically deals with the problem of later versions of the
licence and sets out the options. A copyright holder can choose to allow work to be used
with later versions, such as GPLv3, or can choose not to. There are also more
complex options. The licence itself doesn't force the choice one way or the other.
Matt
Re:Why not ZFS? (Score:3, Informative)
I've been able to do this forever with LVM snapshots under Linux.
Re:B-tree based Filesystem (Score:2, Informative)
A B-Tree can have N children per node, where N is determined by the number of child links you can fit in one block. You are thinking of a binary-tree.
Re:Why not ZFS? (Score:5, Informative)
Rather, GPL is incompatible with anything else that can't be re-licensed as GPL, and that includes GPL v2 and v3, which can't even be mixed among themselves. May first we clear that mess, right?
With a copyleft license, you intend to secure certain rights to the end user to the work as a whole. It is at the very essence of what the GPL tries to do compared to non-copyleft open source licenses or the LGPL that only covers the parts consisting of LGPL code, not any sort of "flaw" or "mess". Licenses work so that you must simultaniously fulfill all of them, so the GPL denies using GPL code with code that denies end users the four freedoms the FSF profess. That is the intention by design, but then there is some collateral damage as well-intended licenses are rendered GPL-incompatible due to details since the GPL (or any copyleft license) couldn't allow open-ended arbitrary restrictions without losing all meaning. The GPLv2 was particularly flawed in this area since it was made fairly long ago with this not much in mind, and in the GPLv3 they did a lot of work to improve compatibility leading to section 7 that among other things say:
Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or authors of the material; or
e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors.
That vastly improves compatibility with the licenses the GPL wants to be compatible with so collateral damage is reduced to a minimum. It's still very easy to write a license, even a free software license, that isn't GPL compatible though. If you look at the reason the CDDL and GPL are incompatible it's that the CDDLs copyleft conditions and the GPLs copyleft conditions clash because they both try to do the same thing. It's almost impossible to write two copyleft licenses where one (or both) doesn't see the other as adding "additional restrictions" on the end user. Even the GPL can't escape that as it tries to improve the GPL unless you have the "and later" clause. Then again, there's no reason such a license should have to be revised often - it took 16 years before releasing version three and it'll probably be longer until next time it's needed.
Re:Why not ZFS? (Score:3, Informative)
I don't know about that either. There are consistent reports that ZFS is slower than Ext3 on many common workloads. Also reports of instability.
While I do respect some of the engineering achievements in ZFS, I do not consider it to be the last word in filesystem design, or even the best filesystem for many applications. I also have doubts about the wisdom of some of the design decisions, such as inhaling the LVM into the filesystem, using 128 byte block pointers, and making a distinction between filesystem snapshots and clones.
Re:Why not ZFS? (Score:1, Informative)
And that is exactly what ZFS achieves:
http://blogs.sun.com/eschrock/entry/ufs_svm_vs_zfs_code
UFS+SVM = 320k lines of code
ZFS = 70k lines of code
And ZFS offers massively more functionality, flexibility and performance that UFS+SVM.
Reversing your analogy - sure you can do anything you want with grep, sed, awk and cut. But sometimes you just make your day easier with a 3 line perl script. It doesnt mean the earlier tools are redundant, it just means you have a better way of doing things.
Re:Back when there was only fat16, ntfs, ext2 used (Score:5, Informative)
I hope you're joking.
ext2 is nice and simple, but it's neither fast not reliable. It uses a linear search to find directory entries, which means it's very slow on large directories, like Maildir mailboxes. It doesn't do tail packing which means it wastes space and is slower with small files. It's not reliable because without a journal it needs a fsck after a bad shutdown which takes ages on a modern disk, and recovers it worse than a journal would.
Just search for benchmarks, something like reiserfs beats ext2 by huge margins when it comes to important workloads such as a mail server.
There are very good reasons why distributions generally go with ext3, or one of the other filesystems. I haven't seen ext2 as the default option for the root FS in a very long time.
Re:Why not ZFS? (Score:4, Informative)
I'm definitely in the layered-design-is-good, ZFS-is-an-abomination camp. But I do have to point out that mlockall would keep a userspace filesystem server from being swapped out, and with realtime priority, the process could even have some guaranteed CPU time. Userspace isn't that bad.
Re:Why not ZFS? (Score:5, Informative)
One of the differences I can find between btrfs and ZFS is that ZFS explicitely avoided [opensolaris.org] a fsck utility, and btrfs is explicitely designed with features designed to make fsck even more powerful than it's on usual filesystems like ext3. In btrfs, data structures have "back references", and the fsck can be used while the filesystem is mounted.
IMO, this is a a btrfs advantage. ZFS has checksums and will find errors, but only will be able to self-heal the errors in a redundant configuration. On a single disk, ZFS will find the error thanks to checksums but will not be able to recover your data. Since ZFS was mainly designed for systems that will use redundant configurations, it may have sense there, but desktops are not never going to do such things. IMO the ZFS people were a bit elitist here - "let's going to build a filesystem so good that we won't need a fsck". But in the real world you _are_ going to need a fsck util. Only in excepcional and very rare cases, but you're going to need it.
Of course that doesn't makes ZFS a bad filesystem, but it's an advantage for btrfs and linux.
Re:Why not ZFS? (Score:4, Informative)
Re:Why not ZFS? (Score:3, Informative)
How does those "back references" recover your data in case of a corrupted sector? Honest question, I do not know brfs.
AFAIK ZFS has no fsck because there is no failure case where it would really help.
Back references could help you reconstruct the file system tree during fsck, but if random data is getting corrupted, you're not going to get it back without redundancy (or forward error correction, I suppose, but that amounts to the same thing).
I can't think of many scenarios where the only kind of data corruption I'm worried about is corruption to file system metadata (which is incidentally all journaling is supposed to protect you from), but who knows.
Re:Why not ZFS? (Score:3, Informative)
That piece of text isn't part of the license itself, it's part of a separate standard notice that states that the software is copyrighted and gives permission to redistribute or modify it under the terms of the GPL. It could just as easily have said "either version 2 of the License, or (at your option) any license you want in exchange for buying Linus a beer" and still be under version 2 of the GPL.
Re:Why not ZFS? (Score:3, Informative)
Which is why it got edited out. Note the "oldid" bit in the URL.