Forgot your password?
typodupeerror
Data Storage Red Hat Software Upgrades Linux

Fedora 16 To Use Btrfs Filesystem By Default 198

Posted by timothy
from the taking-a-bold-line dept.
dkd903 writes "According to proposals for Fedora 16, Btrfs will be the default filesystem used in that release. The proposal has been approved by the Fedora Engineering Steering Committee. In Fedora 16, the switch from EXT4 to Btrfs will be a 'simple switch' — it means that major Btrfs features such as RAID and LVM capabilities will not be forced onto users."
This discussion has been archived. No new comments can be posted.

Fedora 16 To Use Btrfs Filesystem By Default

Comments Filter:
  • Cool. (Score:3, Funny)

    by Anonymous Coward on Thursday June 09, 2011 @10:09AM (#36387576)

    Can't wait to turn all my files into butter.

    • Re:Cool. (Score:5, Insightful)

      by jetole (1242490) on Thursday June 09, 2011 @02:25PM (#36391606)
      Turn your files into butter is right. Though I don't use Fedora, I was interested to look into btrfs again when I read this post on slashdot.

      Much to my surprise, directly from the main btrfs wiki: "Note that Btrfs does not yet have a fsck tool that can fix errors. While Btrfs is stable on a stable machine, it is currently possible to corrupt a filesystem irrecoverably if your machine crashes or loses power on disks that don't handle flush requests correctly. This will be fixed when the fsck tool is ready."

      Why would RedHat choose such a file system that does not have basic support for recovery of a file system after a system crashes? This has been an essential part of file systems since the as far back as I can remember.
      • by jjohnson (62583)

        Because it's Fedora, which is explicitly a testbed distro for newer technologies, and the problem is limited to disks that don't handle flush requests correctly. If you care about a 100% reliable system, you shouldn't be using Fedora in the first place. If you're willing to accept a very small risk of losing everything and having to restore from backup, you get to play with the shiny new toys.

        • Re: (Score:3, Insightful)

          by jetole (1242490)
          "a very small risk"? Have you never had a system crash that you have had to reboot? Have you never had to run fsck to scan and correct a disk? Everyone else I know who uses Linux has done both multiple times. This particular "shiny new toys", as you put it, is not production ready if it does not contain fsck capabilities and according to it's authors it does not. When I say "production ready", I am not referring to whether or not it's stable enough to run a production server farm but I mean whether or not i
          • Re:Cool. (Score:4, Insightful)

            by jjohnson (62583) on Thursday June 09, 2011 @03:26PM (#36392500) Homepage

            You get the reliability you select. All you need to do to avoid this issue with brtfs is 1) use ext4 instead, or 2) use a hard disk that doesn't lie about whether or not it's buffering.

            Fedora is not production ready even in the sense of a daily use desktop with data that can't be recovered from backup, even without this problem. It's explicitly a testbed distro, and always has been.

  • Does this mean we are a step closer to the possibility of snapshotting system states and rolling back to before a bunch of updates were installed?
    • by rwa2 (4391) *

      Sure, you can sort of do that with LVM now.

      What doesn't work with LVM, unfortunately, is using snapshots that allow you to "roll forward" if the system updates work out all right and you want to accept the updates into your main volume :-/

      • by Znork (31774)

        The LVM merge feature has been available for around a year now, at least it's available in F14. Or are there some issues with it that make it unsuitable for doing that?

      • Sure, you can sort of do that with LVM now.

        What doesn't work with LVM, unfortunately, is using snapshots that allow you to "roll forward" if the system updates work out all right and you want to accept the updates into your main volume :-/

        I bet you couldn't find five sysadmins in the (one one) country who do this in production.

        Brtf alone isn't going to bring to Linux:

        create new boot environment 'foo'
        mount 'foo' to /mnt
        install patches/updates/new software to root at /mnt
        umount 'foo'
        activate 'foo'
        reboot

        Then from the boot menu, pick 'previous_foo' if 'foo' is broken.

        That's a lot of really different projects that would have to come together for Linux to pull it off, one being dramatically different distro to distro. We've already heard folks sh

        • by Rich0 (548339)

          I believe in btrfs that a snapshot is itself a mountable structure, so in theory all you have to do is do a snapshot called "previous_foo" and then install your changes, and if it doesn't work boot with (real_)root= on your kernel line.

          Not a whole lot needs to actually come together unless you want it all automated. Then again, I run gentoo so my grub.conf is hand-configured anyway. :)

        • Nexenta does this already. They have an apt-clone utility that wraps apt-get and ZFS snapshots. It takes a snapshot, then does the upgrade. If you don't like it (e.g. stuff breaks), you roll back to the snapshot. If you do, then you just delete the snapshot and continue.

          We've already heard folks shout "layering violation" at basic parts of ZFS itself

          And the person who said this clarified it saying that it wasn't a criticism. It's also wrong. ZFS has very clean layering, it just doesn't put the layers in the same place as System V.

        • In apt systems at least...
          Create an overlay filesystem based on /
          chroot into the overlay
          run updates
          have a handy dandy "boot into overlay" option (I've done this, knoppix does this, it isn't hard and they CAN put it in there with no hassle)
          test
          test more
          give it the okay!
          reboot into a "collapse the changes into the mainline root" mode which will then reboot normally (into the freshly collapsed root). One can collapse the two using rsync. It will be very fast
          Profit.

          Take you two hours to do or about three weeks

    • by Bob Loblaw (545027) on Thursday June 09, 2011 @11:46AM (#36388940)

      There was already a yum plugin for this (yum-plugin-fs-shapshot) as far back as F13 as mentioned here: http://fedoraproject.org/wiki/Btrfs_in_Fedora_13 [fedoraproject.org]

      It will do an automatic btrfs snapshot of affected filesystems before every yum transaction so that you can go back to whatever point you want. Also, since it is partition dependent, you can rollback your system partition and not undo changes you may have made to your home directories if you have those on different partitions.

      btrfs is quite powerful but I have found that the user/GUI tools have not come up to speed yet. I have been using btrfs from my F15 netbook and it seems to have caused no issues so far. However, enabling transparent compression and any tweaking has entailed editing /etc/fstab (never a thing to do lightly) and command lines.

      Hopefully some of the GUI disk management tools will start to make available some of the capabilities of btrfs.

  • by drinkypoo (153816) <martin.espinoza@gmail.com> on Thursday June 09, 2011 @10:14AM (#36387648) Homepage Journal

    Summary: We like it when you test new stuff for us, and our customers are clamoring for this filesystem in RHEL, so we're going to let you try it out on Fedora for a while and experience the hiccoughs. And speaking of new stuff, we're going to finally get around to moving up to grub2 like everyone else, which we haven't bothered to implement even though it's much better, and we allegedly like new stuff.

    • by BreezeC (2040184)
      Maybe the boss redhat don't like grub2.
    • Re: (Score:3, Informative)

      by grasshoppa (657393)

      Yes, and? I thought it was well known that fedora is the test bed for RHEL.

      • by drinkypoo (153816)

        Yes, and therefore "When it comes to adopting the newest technologies, Fedora is always at the front among the major Linux distributions" is a dirty lie. It should be "When it comes to convincing fools to beta-test RHEL for us, Fedora is the shiznit!" What does "major Linux distributions" mean, anyway? I'd give that to Gentoo, which often makes functionality available very early.

        Fedora is of course at the forefront of adopting the newest technologies developed by Red Hat. Some of what they have written has

        • by Rich0 (548339) on Thursday June 09, 2011 @11:33AM (#36388672) Homepage

          So, I'm a Gentoo Dev and I'd qualify that statement a little. Gentoo tends to make a lot of stuff available very early, but it doesn't tend to go all-in with experimental stuff for the base user experience. The default Gentoo stable experience is legacy Grub, for example. And Gentoo hasn't bought into Unity/Systemd/etc. Although, there is talk of adding systemd to the list, and Chrome OS is a Gentoo-based distro that uses unity - so clearly it can be done.

          Gentoo tends to be about giving users a default experience that is reasonably stable, but making it a lot easier for users to branch out and try different things, while still keeping much of the update automation in place.

          Gentoo tends to be less about the polished desktop experience, so I suspect we'll always lag something like Ubuntu in that regard - at least when it comes to ripping apart everything and trying something new.

          • by drinkypoo (153816)

            When I was running Gentoo it was trivially easy to jump on the bleeding edge. Is that no longer true? That was my second-favorite thing about it (number one being that I actually ended up with a build for the K6 I was using it on.)

            • by Rich0 (548339)

              It is, but keep in mind that there are many bleeding edges.

              You could be running bleeding edge postfix, or you could be running bleeding edge sendmail - the distro doesn't care which.

              You could be running bleeding edge KDE or Gnome - again we don't care.

              You could be running the default openrc, or perhaps one day you could be running systemd (right now we care - some would like to change that).

              There is a default experience if you follow the guidebook and don't mess with certain flags/packages/etc. It can be s

  • by Hylandr (813770) on Thursday June 09, 2011 @10:14AM (#36387650) Homepage

    I was handed a block of drives that had been in a server recently with software raid and asked to rebuild the array and recover the data. This had been assembled with mdadm. Will this change make such recovery a non issue with snapshots and the like, providing of course the array had been running btrfs?

    - Dan.

    • by guruevi (827432) <evi @ s m o k i n g c u be.be> on Thursday June 09, 2011 @10:56AM (#36388148) Homepage

      Much like ZFS, mdadm will simply be replaced with another set of commands. If a drive crashes and the array is not correctly set up, you will also lose data and it will be a pain in the butt to repair it (I've done ZFS recovery of a corrupted pool, no fun). Again, RAID (or any software checksum based alternative) is not a backup. You should have hourly/daily/weekly snapshots and backups depending on the importance of your data.

      The good thing about ZFS and Btrfs vs RAID is that they fail graciously. Most of the time it will be able to indicate which files are corrupt, allow you to mount read only and at least copy portions of it over so there is some more intelligence built-in than a simple XOR but that's just the progression of technology.

      • by Rich0 (548339) on Thursday June 09, 2011 @11:42AM (#36388846) Homepage

        I think the bigger issue with btrfs vs mdadm and ext4 will just be maturity. Btrfs repair tools just haven't evolved to the same place the other tools are at, but there is no fundamental reason why they won't eventually make it there.

        As you say btrfs has the advantage that it knows something about how the space is used, so if space was just free it could just nuke it and not worry about trying to salvage garbage data. It could also use free space to leverage recovery. And, the concept of COW means that strips are less likely to be in a transitory state of having meaningful data partially overwritten by other meaningful data, since the filesystem would first try to write the stripe over unused space leaving a fully intact backup if it is interrupted and the array is already degraded/etc.

        The last time I tried test-converting an existing ext4 into a btrfs on RAID it paniced, but it has been almost a year now...

      • Actually btrfs merely uses lvm and md devices on the underside. You can still run the lvscan / lvdisplay / mdadm, however what you'll see will look rather obtuse.
  • Sources for this? (Score:5, Informative)

    by Dan Dankleton (1898312) on Thursday June 09, 2011 @10:15AM (#36387656)

    Surely there could be a better source for this than one blog post (I know, high UID so I must be new here.)

    But linking to something like http://fedoraproject.org/wiki/Features/F16BtrfsDefaultFs [fedoraproject.org] or http://thread.gmane.org/gmane.linux.redhat.fedora.devel/149196/focus%3D149215 [gmane.org] to lend a little authority might have been nice...

    • by Lunix Nutcase (1092239) on Thursday June 09, 2011 @10:21AM (#36387728)

      But how does that drive ad clicks for digitizor if one links to the official source? Silly you.

    • by _Sprocket_ (42527)
      Your links are nice background / support. But they're hardly read as well. Sure - one can hash out the details via the wiki link. But I could see those details being missed without the narrative. So in this case, I would say the blog entry added something to the story unlike some blog entries we see that are simply re-hashing other blogs without adding anything themselves. If I had written the submission, I would have picked the blog entry as the primary link with your wiki link as supporting. If I h
  • by Thrakkerzog (7580) on Thursday June 09, 2011 @10:19AM (#36387704)

    In the long run btrfs will be good to have, especially with solid state drives gaining popularity. Even embedded devices can easily have multi-gigabyte flash chips, and btrfs would be faster and more efficient on these when compared to jffs2 and yaffs.

    • by Zoidmann (869204) on Thursday June 09, 2011 @10:26AM (#36387796)
      It seems like they haven't: https://btrfs.wiki.kernel.org/index.php/FAQ#When_will_Btrfs_have_a_fsck_like_tool [kernel.org]

      Hopefully they know something about it being released soon since making the decision to use it.
    • by cronius (813431)

      Regarding fsck, from: https://fedoraproject.org/wiki/Talk:Features/F16BtrfsDefaultFs#Recovery_strategy [fedoraproject.org]

      fsck must work. It should be tested for a longer period to see if it's really working.

      Would be nice if this speeds up the work on btrfs.fsck.

      • by arth1 (260657)

        No plans for an fsr either.
        So far, you need to stay at ext2/3 or xfs if you want defragmentation.

    • Is there a need an fsck? For example, ZFS doesn't have one and I haven't heard of anybody working on it (or of anybody actually wanting one).

      • by cratermoon (765155) on Thursday June 09, 2011 @11:22AM (#36388490) Homepage

        It says right here on the btrfs wiki [kernel.org]: "While Btrfs is stable on a stable machine, it is currently possible to corrupt a filesystem irrecoverably if your machine crashes or loses power on disks that don't handle flush requests correctly. This will be fixed when the fsck tool is ready. "

        So I'd say yah, that's a pretty important piece to be missing if you're talking about making it the default for a distro, even one as free-wheeling as Fedor.

        • by Mullen (14656) on Thursday June 09, 2011 @11:37AM (#36388754)

          I have seen this issue with other filesystems. It comes from when the hard drives *lie* to the RAID or I/O controller stating that they don't buffer or are not buffering when in reality, they are. It's really frustrating since you can't trust the drives have written out the data.

        • Re: (Score:3, Insightful)

          This. I run btrfs on my home file server and lost a half year of data due to a power outage corrupting the filesystem. Afterwards I was looking around for its fsck utility only to find it does not yet exist. Btrfs may be the way of the future, but as is, it's still too soon.

        • by shentino (1139071)

          Translation:

          "We can get fucked over when a device lies to us about buffering just like any other filesystem, and it's not our fault if you have shitty hardware that doesn't follow instructions like it's supposed to."

      • by bill_mcgonigle (4333) * on Thursday June 09, 2011 @11:34AM (#36388698) Homepage Journal

        Is there a need an fsck? For example, ZFS doesn't have one and I haven't heard of anybody working on it (or of anybody actually wanting one).

        Um, yeah, read zfs-discuss. There are helpful folks on there who help people recover their ZFS volumes, but having a tool to do it would be much better.

        fsck for ZFS or btrfs means something different than it does for ext* but it's still needed. I just had a client's new 18TB ZFS zvol go TU when the power failed and the UPS->host communication wasn't properly connected. Fortunately it wasn't very important and the important zvol wasn't active when the power failed.

        btrfs will be better than ZFS for many use cases once the fsck is stable. For others ZFS will remain better, but you better have battery-backed disk cache or a monitored UPS (neither of which are appropriate requirements for large swaths of the Fedora user base).

        • by Chirs (87576)

          For others ZFS will remain better, but you better have battery-backed disk cache or a monitored UPS (neither of which are appropriate requirements for large swaths of the Fedora user base).

          Actually, a large swath of the Fedora user base runs laptops, which depending on how you look at it could be considered either a battery-backed disk cache or a monitored UPS.

          • Actually, a large swath of the Fedora user base runs laptops, which depending on how you look at it could be considered either a battery-backed disk cache or a monitored UPS.

            Ideally. I've had my Fedora run out of battery while shutting down (triggered by power management) often enough that I wouldn't trust it. I run my ext4 with data=journal.

      • by nairnr (314138)
        Well, ZFS has scrub which goes through and verifies all checksums, for mirror or raidz it repairs and discrepancies so it is fsck like.
      • No matter how well you design your FS, errors can happen. Generally it is because of hardware below it but the cause isn't important, the recovery is. That's why NTFS has chkdsk. Many people don't know it and assume it isn't needed because it runs extremely rarely, a user will likely never see it. It isn't like the old FAT32 days where you needed to run it on every crash. However it is still there, and we (IT) still use it from time to time. If something goes wrong, the filesystem can be left in a state whe

  • by Gaygirlie (1657131) <gaygirlie@hotma i l . c om> on Thursday June 09, 2011 @10:33AM (#36387898) Homepage

    For some reason I'm getting really low performance on btrfs, both on a single disk and on raid1 configurations. I have tried with -nodatacow and with and without -compress, but it seems it doesn't have any effect. Also, I have 90 gigabytes of free space on Storage1 but I get drive full error when I try to write there. Rebalancing it didn't fix the issue. The btrfs command-line tool is, well, rather incomplete and somewhat buggy, like e.g. when I query 'btrfs fi df /media/Storage2' -- with Storage2 being the raid1 pool -- it reports the size and usage of the smallest disk on it, not the whole thing. I don't understand why. I also have had some filesystem corruption which caused me to lose quite a bit of data, and again the only way to fix it was rebalancing the whole thing which takes the whole damn day.

    I do understand that it's a filesystem that's still under development, but the tools atleast need a lot more work. They're just too incomplete at the moment. I'm not really sure pushing it as the default filesystem for end-users is a good idea yet.

    • by arth1 (260657) on Thursday June 09, 2011 @10:46AM (#36388030) Homepage Journal

      . The btrfs command-line tool is, well, rather incomplete and somewhat buggy, like e.g. when I query 'btrfs fi df /media/Storage2' -- with Storage2 being the raid1 pool -- it reports the size and usage of the smallest disk on it, not the whole thing. I don't understand why.

      With RAID 1 (simple mirroring), your volume is limited to the smallest of the components. A 40 GB and a 2 TB drive put together yields a 40 GB RAID 1 volume.

      • My mistake, I meant raid0, ie. with striping. The odd thing is that the tools report it as raid1. And yes, I have way more stuff there than the smallest drive, so it can't be raid1.

        • by arth1 (260657) on Thursday June 09, 2011 @11:13AM (#36388356) Homepage Journal

          With RAID 0 you get twice the amount of the smallest unit.
          Using a 40 GB and an 2 TB HD will give you an 80 GB RAID 0.

          • by Rich0 (548339)

            Does btrfs actually stripe like this, or does it simply keep a copy on two volumes for RAID1, and spread data around for RAID0?

            You can tell btrfs to even keep two copies on the same volume if you want, I think.

            They call it RAID, but I don't think it is really the same thing. The RAID5-like features and reshaping don't exist yet (the last time I checked). Then again, maybe it supports both modes of operations - if there is one thing about btrs is that it has a million optional features.

    • by Bengie (1121981)

      I read one person's blog a few weeks back about how he had BTRFS set on attempting to fix something, but not sure what. It was reading the HD at full speed for over a day. It was effectively stuck in a loop. It was eating up all of his IO and also a large portion of his CPU, and a modern 8 core server with several gigs of ram. Also, because the FS was stuck trying to fix the data, the computer refused to shutdown as the FS would not release.

      He had to do a forceful shutdown and that messed up his FS even mor

  • What about fsck support? The last time I checked, btrfs does not support this important feature to allow recovering from hard disk issues.
    • by caseih (160668)

      I've ran BtrFS's arch rival, ZFS for years, and I have never ever had to fsck the disk, even after power outages and crashes (yes Solaris crashes rather spectacularly sometimes). In theory a good file system should always be in a consistent state and never require fscking. Of course you could mean trying to fsck a disk when a hard drive starts failing, but I still don't see how that is useful; fscking a failed drive might just stir the data more. I'll settle for being able to pull off as much data as I c

      • by jvillain (546827)

        It's fairly new so most people probably don't know about it. It has been available in Rawhide for a couple of months or so now.

  • Never mind (Score:2, Funny)

    by bigjocker (113512) *

    ReiserFS will kill it

  • by mgmartin (580921) on Thursday June 09, 2011 @11:34AM (#36388704)
    Quoted from man mkfs.btrfs on Fedora 15.

    ...Btrfs is currently under heavy development, and not suitable for any uses other than benchmarking and review.

    That sure limits the uses of a default Fedora installation.

    • by Gothmolly (148874)

      You do know what Fedora is supposed to be, right ?

    • by rdnetto (955205)

      Is anyone else thinking of when Ubuntu implement ext4 support? Turned out there was a major bug that caused kernel lockups when emptying the trash.
      I can only imagine something similar will happen here...

  • by drunkahol (143049) on Thursday June 09, 2011 @11:45AM (#36388912)

    I'd like to see btrfs implement a proper block tiering system. They're doing something for storing "hot" blocks on SSD, but what about giving us the full monty? Where I can rank storage types myself, assigning a different cost to each type. Hotest blocks in RAMdrive (battery backed of course), next step down fast SSD and then slower SSD, followed by Fibre, SAS, SATA and finally tape. Yes tape. Just create snapshots as backups. These blocks then sit there and drift down to tape storage when required.

    Funny how this has all been done before when disks were really slow. I suppose it's the big gap of incredibly fast SSD's (compared to mechanical) that's resurrecting these ideas. With this done, btrfs could be stuck in as a relatively cheap SAN/NAS solution. All done in a big tower case in my loft.

  • Red Hat turned against we IT geeks who got them accepted into the corporate environment, and instead made the hobbyist user to have a guinea pig distro with unstable things in it like BTRFS. Sure, I'm excited about the day when butterfs will be a stable option, but that isn't yet. Screw Fedora, Screw Red Hat, use something else where the care about stability and where the hobbyist/user/server versions are all just kernel and package changes on the same distro.
    • by mu51c10rd (187182)

      The fragmentation already exists. Need a stable, supported release? Use RHEL. Need to use it without the Red Hat strings attached but still have the stability? Use CentOS. Want the hobbyist bleeding edge? Use Fedora.

  • I don't really need to say much more. Fedora is just too bleeding edge even for non-production IMHO. I don't want to boot up my box and have my file system screwed. Sorry Red Hat. You should go back to the old model of eating your own dog food. Perhaps a model like Ubuntu with a Red Hat LTS and a Red Hat Latest Version, but the latter being a little more stable than Fedora. I digress, this is why I am on Ubuntu now. It's not perfect, but it has been a much more positive experience for me than running Fedora

"For the man who has everything... Penicillin." -- F. Borquin

Working...