Forgot your password?
typodupeerror
Sun Microsystems Software Linux

ZFS On Linux - It's Alive! 281

Posted by Zonk
from the insert-mad-science-laugh-here dept.
lymeca writes "LinuxWorld reports that Sun Microsystem's ZFS filesystem has been converted from its incarnation in OpenSolaris to a module capable of running in the Linux user-space filsystem project, FUSE. Because of the license incompatibilities with the Linux kernel, it has not yet been integrated for distribution within the kernel itself. This project, called ZFS on FUSE, aims to enable GNU/Linux users to use ZFS as a process in userspace, bypassing the legal barrier inherent in having the filesystem coded into the Linux kernel itself. Booting from a ZFS partition has been confirmed to work. The performance currently clocks in at about half as fast as XFS, but with all the success the NTFS-3g project has had creating a high performance FUSE implementation of the NTFS filesystem, there's hope that performance tweaking could yield a practical elimination of barriers for GNU/Linux users to make use of all that ZFS has to offer."
This discussion has been archived. No new comments can be posted.

ZFS On Linux - It's Alive!

Comments Filter:
  • by seanadams.com (463190) * on Tuesday June 19, 2007 @02:48PM (#19568473) Homepage
    The in-kernel vs userland distinction has always struck me as quite arbitrary. So in one case you're linked at compile time and in another case you compile them separately and go through system calls. Why should that make one of them a derivative work and the other not? In either case the file system can be taken out and you still have a perfectly functional kernel that can run other file systems. Same goes for graphics drivers.

    The GPL doesn't attempt to codify all the intricate details that it would take to define such a distinction in the license. It's only described as an accepted rule of thumb in the FAQ. So what's the deal? It seems like this rule is really holding back some commercial support for Linux - is the current situation what we really want, and at any rate how did we get here? Would we be better off if such a separable, non-essential feature could be linked in somehow instead of needing to be put behind extra layers of abstraction?
    • Re: (Score:3, Informative)

      by vialation (885786)
      It has been decided that the deciding factor with the GPL is whether or not something links to it. Instead of a userspace program which simply makes a system call by tripping an interrupt or using the SYSENTER opcode (which would not require kernel source code to do), a kernel module (which, without FUSE, a filesystem must be) actually gets linked into kernel space, and has its symbols added to the kernel. Any calls to other parts of the kernel directly call the kernel. That is the distinction.
      • Re: (Score:3, Insightful)

        by Paul Jakma (2677)
        It has been decided that the deciding factor with the GPL is whether or not something links to it.

        This is absolutely not true. "It has been decided": When? By whom? This claim is not only unfounded, it is also quite false (just ask a lawyer..).

        The parent absolutely should NOT be modded as informative.
        • by jimicus (737525) on Tuesday June 19, 2007 @03:39PM (#19569249)
          Alright then, how about "The de-facto understanding is..."?

          The existence of the LGPL goes some way to suggesting that this indeed is the generally accepted understanding. It's not been codified in any legal judgement that I'm aware of, but that's only becaue there have been relatively few legal judgements worldwide on this particular aspect of the GPL. Most breaches seem to be far more flagrant.
          • by Per Abrahamsen (1397) on Tuesday June 19, 2007 @03:58PM (#19569595) Homepage
            The FSF has used the syscall interface as a guideline to determine whether something is a derived work or not. It is a guideline, not a hard rule though, and I suspect they would consider user-space ZFS for a derived work using a technical trick to avoid being linked into the kernel. I.e. infringing. However, since the FSF doesn't own the kernel, their opinion on the subject doesn't matter.
            • by pthisis (27352) on Tuesday June 19, 2007 @04:38PM (#19570281) Homepage Journal
              The FSF has used the syscall interface as a guideline to determine whether something is a derived work or not. It is a guideline, not a hard rule though, and I suspect they would consider user-space ZFS for a derived work using a technical trick to avoid being linked into the kernel. I.e. infringing. However, since the FSF doesn't own the kernel, their opinion on the subject doesn't matter.

              OTOH, some respected kernel developers (e.g. Alan Cox) certainly have explicitly said they believe binary kernel modules must be GPL and that things like nvidia's drivers are probably infringing. Not exactly the same thing but it's enough that I'd still call this a grey area until it's tested.
            • by Paul Jakma (2677) <paul+slashdot@jakma.org> on Tuesday June 19, 2007 @07:53PM (#19572751) Homepage Journal
              The FSF has used the syscall interface as a guideline to determine whether something is a derived work or not.

              I don't believe that's true. I think you're thinking of the explicit exemption Linus put in the COPYING file of the Linux kernel to say that the syscall interface was a GPL interface (there are Linux contributors who disagree to an extent with him on that).
        • by ChrisA90278 (905188) on Tuesday June 19, 2007 @06:28PM (#19571863)
          "It has been decided": When? By whom?"

          Linus decided. He wrote specific terms into the modified version of the GPL he uses with Linux. He makes it very clear. Linux does NOT use the "standard GLP" it makes a few changes for example he removed the "and later versions" part.
          • by Schraegstrichpunkt (931443) on Tuesday June 19, 2007 @06:53PM (#19572153) Homepage

            Linus decided. He wrote specific terms into the modified version of the GPL he uses with Linux.

            Retroactively, AND claiming to cover code for which he is not the copyright holder. The notice at the top of the COPYING file in the kernel source code more-or-less amounts to his opinion---and if you do a little research (read: Groklaw), you'll find that Linus clearly isn't a lawyer and doesn't know what he's talking about when it comes to legal matters.

            "Linus said so" is just not a valid argument about a legal matter, unless you're trying to defend yourself against a copyright infringement claim that he's making about code he holds the copyrights for.

          • by the_womble (580291) on Tuesday June 19, 2007 @09:55PM (#19573835) Homepage Journal

            Linux does NOT use the "standard GLP" it makes a few changes for example he removed the "and later versions" part.
            Wrong. Here is the COPYING file for the Linux kernel [kernel.org].

            He has added a note clarifying that userland programs making system calls are not derivative works. That is clarifying his interpretation of the license. At the most, it adds an exemption, which is common practice.

            The "and later versions" clause is not part of the GPL, it lies outside it. In effect, it makes the work automatically multiple licensed as and when new versions of the GPL comes out: so if you distribute something under GPL v2 with that clause, when v3 comes you it will become dual licensed under 2 and 3, when v 4 comes out it will be triple licensed etc. This allows people to redistribute under v2 or any later version, with or without the "and later versions" clause.

      • by afidel (530433)
        Why not just use a kernel shim like the commercial closed source drivers do? It would still be faster than usermode!
      • by msauve (701917)
        hold the copyright to the Linux kernel? It seems to me that he must, as everyone seems to agree that he has control over what license it is issued under (see discussions of GPL v2 vs. v3). If the kernel copyright is not entirely held by Linus, then he would have to get unanimous consent to make any change (impossible?), if he at some point wanted to move to GPL3, right?

        So, if he controls the copyright, he can place it under a "GPL plus you're allowed to link ZFS" license. Where's the problem?
        • by Sancho (17056) on Tuesday June 19, 2007 @06:20PM (#19571741) Homepage
          He certainly owns the rights to parts of the Kernel, and thus his opinion on the matter has some importance. However he has incorporated patches into the source tree without requiring that the copyright be turned over to him, and thus, just about everyone who has contributed to the kernel collectively 'holds the copyright' to it.

          The Linux kernel, for better or for worse, is going to be stuck at GPLv2.
          • Ah, well then... (Score:3, Insightful)

            by msauve (701917)
            Thank you for the informative response, it was much better than the other "no" one.

            If I understand things correctly, it is possible for users to legally compile the Linux kernel and ZFS together, the sources may even be distributed together as long as there are no pre-linked binaries (ready to run). That should make it possible for a distribution to compile ZFS support into the kernel at install time (i.e. by the user, for their own use, not for distribution).
            • Re: (Score:3, Informative)

              by Sancho (17056)
              I'm not qualified to comment on this (read: I am not a lawyer), however I think that this still may be legally questionable. Because the CDDL ZFS files won't compile cleanly without modification to make them work with the Linux kernel, and because those modifications could be considered derivative works, it's possible that you'd be creating source files which are derivative works of both ZFS (CDDL-licensed) and Linux (GPLv2-licensed). Such a derivative work cannot be distributed legally.
    • by TeknoHog (164938) on Tuesday June 19, 2007 @02:57PM (#19568627) Homepage Journal

      The in-kernel vs userland distinction has always struck me as quite arbitrary. So in one case you're linked at compile time and in another case you compile them separately and go through system calls. Why should that make one of them a derivative work and the other not?

      I agree the kernel vs. userland issue is arbitrary. However, think about all the closed-source software running on Linux, or opensource with other licenses but GPL v2. These are legally possible only because we make the distinction.

    • by LWATCDR (28044) on Tuesday June 19, 2007 @03:01PM (#19568675) Homepage Journal
      I am not a kernel hacker but I believe you are confusing loadable kernel modules with user space code.
      I am pretty sure that kernel modules run at the kernel level and can access all the same structures that a driver compiled into the kernel can.
      FUSE file systems run at the same privilege level as a user program does. In theory it is a slower but more robust system. If a FUSE file system crashes you can just restart the filesystem and remount the drive. If a kernel level file system crashes it can cause a kernel panic and bring down the entire system.
      So the distinction between user-land and kernel drivers is anything but arbitrary.

      • Re: (Score:3, Informative)

        by Vancorps (746090)

        This is and was always one of the chief complaints about Windows as well, everything in the kernel so a malfunction can take the whole system down. Now the push to user-space makes everything more robust. Of course you also take a performance hit hence the slowness that is Vista.

        Still, if ZFS under FUSE is half as fast as XFS in kernel-mode then you're not talking about an FS that is particularly slow, you would still gain the data protection from ZFS which in my mind at least is the whole point. It would

        • Re: (Score:3, Interesting)

          by LWATCDR (28044)
          Actually if you take a look you will see that NTFS-G under Fuse is actually faster then XFS. There seems to be some memory fragmentation issues with ZFS currently.
          I am a big fan of Microkernels. I keep hoping that Minix 3 will really start to take off soon. I love the idea of self healing systems and that can really only be done with a Microkernel.
          I don't think Vista is slow because of any moves into user-space. I think it is more an issue of memory usage and DRM. Let's face it if you are "managing rights"
    • by notamisfit (995619) on Tuesday June 19, 2007 @03:26PM (#19569017)

      The version of the GPL included with the Linux kernel states at the top:

      • NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work".

      Not sure how far back this clarification really goes, but I think it predates the GPLv2-only one, making it at least six years old.

  • Yet to be included? (Score:2, Informative)

    by vialation (885786)
    The summary makes it sound as if ZFS will ever be included in the kernel. Anything FUSE will never be in the kernel, except the FUSE driver itself. Userspace programs and kernelspace are considered separate for a reason.

    Of course, this will all change if both Sun and Torvalds switch to GPL3...
    • by Anonymous Coward
      There's a Linux filesystem under development that might be able to compare favourably with ZFS if shown some love by developers:

      * http://oss.oracle.com/projects/btrfs/ [oracle.com]
      * http://kerneltrap.org/node/8376 [kerneltrap.org]

      Avoid the license squabbles and do what we do best: build it ourselves, only better.
    • Re: (Score:2, Interesting)

      by salahx (100975)
      Not necessarily; NTFS is both in the kernel AND in FUSE. However, the kernel version is less full-featured than the user-space version (ntfs-3G); because of lack of manpower and that author strict quality control - the old NTFS driver ate filesystems.

      That said, I can't see ZFS ever being in the kernel - even licensing problems aside; its a HUGE layering violation. Some say they can do a ZFS without the layering problem; an ambitious project - btrfs [oracle.com] exists to try do exactly that. Of course its nowhere near d
  • Grub (Score:5, Informative)

    by jshriverWVU (810740) on Tuesday June 19, 2007 @02:51PM (#19568507)
    Booting from a ZFS partition has been confirmed to work

    Grub has supported ZFS booting for a while (forget which branch though).

    • Re: (Score:2, Informative)

      by vialation (885786)
      Getting GRUB to load ZFS was easy since Sun released GPL boot code which could minimally read a ZFS partition. The hard part is making it so that the kernel can use the filesystem as root, which requires ZFS to be built into the kernel. Unless you are using an initrd or some other FS for the boot partition. Direct booting is extremely difficult.
  • Can't you just make a binary blob kernel module? I know the GPL zealots hate the idea, but wouldn't this also get around the liscencing problems?

  • Never mind ZFS (Score:3, Informative)

    by overshoot (39700) on Tuesday June 19, 2007 @02:56PM (#19568595)
    What I desperately need is a reliable caching filesystem with decent performance.

    The $COMPANY network is loaded with Linux workstations and servers, all with their own lotsabyte drives -- and the only things those drives are used for is a tiny system image. Meanwhile the network is getting hammered.

    I might not kill to get a several-hundred-gigabyte local network cache -- but don't tempt me.

    • Re:Never mind ZFS (Score:4, Informative)

      by Krondor (306666) on Tuesday June 19, 2007 @03:02PM (#19568691) Homepage
      What I desperately need is a reliable caching filesystem with decent performance.

      The $COMPANY network is loaded with Linux workstations and servers, all with their own lotsabyte drives -- and the only things those drives are used for is a tiny system image. Meanwhile the network is getting hammered.


      Are you asking for a network based filesystem like AFS [wikipedia.org]? Did I misunderstand your issue?
      • by belg4mit (152620)
        Network yes, AFS, probably not. Seems to me he wants a cheap distributed filesystem,
        probably closer to something Google uses... networked JBOD. Of course he doesn't say
        what he wants the cache for: Squid?

        I'm guessing the hassle of managing AFS and its requirements (kerberos, synchronized
        time, client-side cache) are more than he'd like just to recover some disk space.
      • Re: (Score:2, Informative)

        by andrei_r (560152)
        One thing would be to migrate your Linux workstations and servers to Solaris x86, and use cacheFS.

        "When you enable cachefs file system, the data read from the remote file system or CD-ROM is stored in a disk-based cache on the local system. Subsequent read requests to the same data are fulfilled by the local cache, which improves read performance."

        Of course, a migration is not always an option, and I'm not sure if something like this available for Linux.
        • Re: (Score:3, Informative)

          by bill_mcgonigle (4333) *
          Of course, a migration is not always an option, and I'm not sure if something like this available for Linux.

          Try the I'm Feeling Lucky search for 'linux cachefs'.
    • by mhall119 (1035984)
      I don't know about several-hundred-gigabyte, but Linux's smbfs does pretty good caching, better than NFS in my experience.
  • by kaleco (801384) <greig...marshall2@@@btinternet...com> on Tuesday June 19, 2007 @03:05PM (#19568727)
    If you can suffer the bizarre presentation style Sun have used for this video [google.com], it's quite informative about the benefits of ZFS.
  • by pair-a-noyd (594371) on Tuesday June 19, 2007 @03:13PM (#19568815)
    Really? Stick an old 20gb drive in as your boot drive and boot from whatever you have to to get up and going, load ZFS modules, mount all drives and enjoy. What's so terrible about booting from a different drive / file system? Most mobos now let you hang boatloads of drives of all types on them.
    I can't think of any reason why it would be so terrible to boot up from an old 20gb with ext2/ext3 or anything else, then run the rest of your system under whatever. I'm doing that now anyway, I boot from ext2 then everything else is ext3. Doesn't make my performance suffer any that I can tell.

    Besides, I suspect that most people that would run ZFS are the type of people that leave their machines up for months at a time. In that case, why the panic attacks over booting issues?

    I hope they can find some way to resolve the license issues, I'm excited about ZFS (in concept and theory) and I would love to give it a go. Finally a system that's up with the times.
    • Re: (Score:2, Flamebait)

      by drinkypoo (153816)

      Really? Stick an old 20gb drive in as your boot drive and boot from whatever you have to to get up and going, load ZFS modules, mount all drives and enjoy. What's so terrible about booting from a different drive / file system? Most mobos now let you hang boatloads of drives of all types on them.

      By using a single boot drive you are introducing a single point of failure. But booting from a ZFS volume spread across multiple physical volumes means that even the boot process is protected by redundancy. It also

      • by wytcld (179112)

        a machine is up for months at a time, then it clearly did not get critical security updates, which often are in the kernel. You should never have uptime that high.

        Depends on what the machine's doing, my friend. If you have dozens of people you don't know on shell accounts, sure, you better keep the kernel current - scheduled downtimes to do that are fine, and the inevitable kernel panics because the latest kernel introduces a new bug for your particular hardware configuration - well, the users will understa

        • by drinkypoo (153816)

          On the other hand, if you have no shell accounts, but just are running one or a few public-facing daemons, if you keep those daemons current, and follow solid security practices in configuring and using them, and lock down your firewall - well, if the kernel's stable, and your business's customers expect you up 24x7, why the heck should you want to be on the front lines testing the latest kernel release?

          Because all it takes is one remote hole in the kernel, perhaps in the IP stack or the firewalling code,

    • Re: (Score:3, Informative)

      by jimicus (737525)
      What's so terrible about booting from a different drive / file system?

      Nothing per se, but AIUI ZFS allows you to specify how much (if any) redundancy you want, which with other filesystems requires setting up RAID separately which in turn adds complexity.

      I suspect that most people that would run ZFS are the type of people that leave their machines up for months at a time. In that case, why the panic attacks over booting issues?

      Generally speaking, the kind of system that you want to have up for months at a
      • Re: (Score:3, Informative)

        by larien (5608)

        Not sure about "up with the times". I have yet to see any convincing explanation of what amazing features ZFS offers which hasn't been in AIX's logical volume management system for years.

        Ok, I'll bite on this one.

        • Ability to detect & disk corruption. Say you get a bad block/corruption on the bus/bug in a driver; your lovely mirror now has 2 different sets of data. Under every other volume manager I'm aware of, you have to guess which one has the correct data and resync (generally the whole drive).
    • NBD (Score:3, Informative)

      by bill_mcgonigle (4333) *
      Stick an old 20gb drive in as your boot drive

      That's *way* overkill. Grub can read minimal ZFS - just have Grub pull off an initrd image from your '/' drive (ZFS), load it into RAM (it'll be ext2 or squashfs or something the kernel can handle) and include a zfs.ko module on the initrd. Load it into RAM, do a pivotroot onto your ZFS / drive, and continue along your merry way.

      The 'you can't boot off of ZFS' canard is just a bit of hand-waving by the Sun-haters who are afraid of ZFS. We Sun-haters who like ZF
  • there's hope that performance tweaking could yield a practical elimination of barriers

    Hows that memory copy from userspace comming, has it healed up yet?
  • I almost jumped out of my skin when I read the headline... then they threw in the little tidbit of information that its running through FUSE. I certainly appreciate the work that went into it, but I'm quiet certain FUSE will never catch up to in-kernel filesystems for speed and performance.
    • Either way, its a start towards getting support on whatever the underlying OS is to support someodd filesystem until the licensing bs can be worked around...
  • by andrewd18 (989408) on Tuesday June 19, 2007 @03:17PM (#19568867)
    According to Darren Moffat at SUN, parts of ZFS are already licensed under the GPL v2. Maybe there is still hope for a native solution. Not, of course, that I mind using FUSE.

    http://blogs.sun.com/darren/entry/zfs_under_gplv2_ already_exists [sun.com]

    Now about that headline, yes I really did say that ZFS code is already available under the GPLv2. I will be completely honest though and make it clear that it isn't all of the ZFS source. It is, sufficient amount to be able to boot an OpenSolaris based system from GRUB, that means that support for mirroring and the checksum and compression support is there but radiz isn't nor are the userland commands. It is possible that this might be enough to get someone started. Still don't believe me check out the updated GRUB source on opensolaris.org, specifically all the files with zfs in their name - every single one of them under the GPLv2 or later.
    • by thesandbender (911391) on Tuesday June 19, 2007 @03:55PM (#19569525)
      Lets be more specific... ZFS (as a whole) is not GPLv2'd.

      From the TFP : Now about that headline, yes I really did say that ZFS code is already available under the GPLv2. I will be completely honest though and make it clear that it isn't all of the ZFS source.

      Well that's fantastic... which parts do we get? The ones that make ZFS revolutionary or the ones that make it a rehashed XFS, JFS, Rieser, etc? I don't see how this is any different than any of the bait-n-switch scams that people post to /., digg or other sites. Yes... you can use "part" of the FS but if you want the whole thing you'll have to use Solaris or FUSE (or BSD as others have pointed out).

      FUSE defeats the entire purpose. ZFS is meant to run and support a large/huge file store. What admin in their right mind would do that through userspace unless it's solely for backup?

      The point is, ZFS is not functionally viable for Linux on the environments for which it was intended.
      • Re: (Score:3, Interesting)

        by irgu (673471)
        "FUSE defeats the entire purpose. ZFS is meant to run and support a large/huge file store. What admin in their right mind would do that through userspace unless it's solely for backup?"

        Who doesn't want his system to crash when a transient hardware error hit non-redundant ZFS kernel code.
  • by Savage-Rabbit (308260) on Tuesday June 19, 2007 @03:19PM (#19568901)

    ZFS On Linux - It's Alive!
    Shouldn't it be more like:

    ZFS On Linux - It's Alive!! IT IS ALIVE!!! MWUHAHAHAHAHAHAHA!!!!!

    The manic laughter is especially important!
    • Re: (Score:3, Insightful)

      by Fred Ferrigno (122319)
      Maybe I've been watching too much late night TV, but in my head I "heard" it as the intro to Robot Chicken. No maniacal laughter, just a chicken clucking.
  • Legal question (Score:4, Interesting)

    by TopSpin (753) * on Tuesday June 19, 2007 @03:29PM (#19569081) Journal
    Why couldn't ZFS be distributed separately in kernel module form and installed by the user? Ubuntu and others integrate mscorefonts, NVidia drivers and others this way. Is the OpenSolaris license so heinous that it's worse than those examples?

    I doubt it.

    • by Yvanhoe (564877)
      Well yes, it is that heinous. It would simply be illegal under current terms to run it this way. It would be interesting, however, to develop such a rogue project and see if Sun is really OSS friendly or if they would be ready to send cease and desist letters to such a project. Linus suggests that Sun's apparent friendliness toward linux and OSS is only a pose, let's figure !
  • by FunkyELF (609131) on Tuesday June 19, 2007 @03:36PM (#19569189)
    Why couldn't this be implemented in the kernel and have the patches to that kernel be hosted in a country which doesn't care too much about licensing?
    • Re:kernel patch? (Score:4, Informative)

      by burndive (855848) on Tuesday June 19, 2007 @07:14PM (#19572395) Homepage

      Why couldn't this be implemented in the kernel and have the patches to that kernel be hosted in a country which doesn't care too much about licensing?

      Why hide? Nothing in the CCDL or GPL licenses prevents you from publishing a CCDL patch to a GPL kernel. Publishing the patched kernel source or binary might run you into some problems, but I don't see anything preventing you from publishing the patch itself.

      Please correct me if I'm wrong.

  • by greginnj (891863) on Tuesday June 19, 2007 @03:51PM (#19569459) Homepage Journal

    ... but does it run on Plan 9 ?



    /me ducks ....
    • by Etyenne (4915)

      ... but does it run on Plan 9 ?
      Nobody care about Plan 9. The real question is: does it run on Haiku ?
  • But does it run on Windows?
  • I have been wondering this for awhile now and I finally have a relevant story to ask it on: Is ZFS 128-bit or not? It claims to be 128-bit, but Wikipedia [wikipedia.org] gives:

    • 2^48 - Number of snapshots in any file system (2 × 1014)
    • 2^48 - Number of files in any individual file system (2 × 1014)
    • 16 EiB (2^64 bytes) - Maximum size of a file system
    • 16 EiB - Maximum size of a single file
    • 16 EiB - Maximum size of any attribute
    • 256 ZiB (2^78 bytes) - Maximum size of any zpool
    • 2^56 - Number of attributes of a file
  • Interesting (Score:3, Interesting)

    by drix (4602) on Tuesday June 19, 2007 @06:30PM (#19571885) Homepage
    This is an interesting development in light of recent comments [lkml.org] made by Linus about Sun and ZFS in particular, to which Jonathan Schwartz wrote a personal response [sun.com].
  • Noooooo!!!! (Score:5, Funny)

    by r00t (33219) on Wednesday June 20, 2007 @12:59AM (#19574907) Journal
    Linux is becoming a microkernel. Linus might even get a passing grade.

At the source of every error which is blamed on the computer you will find at least two human errors, including the error of blaming it on the computer.

Working...