Facebook To Begin Deploying Btrfs 115
An anonymous reader writes "After hiring the lead Btrfs developers and Linux kernel block maintainers last year, Facebook is beginning trial deployments of Btrfs. Facebook will start using the next-generation file-system within their web-tier and they will be among the first major public deployments of Btrfs."
Give me a big plate of beans (Score:4, Funny)
and I'll start deploying all kinds of btrfs.
BUTTERFACE! (Score:1)
Backfield in motion.
How about real problems (Score:5, Funny)
When are they going to make the users of their website tolerable human beings instead of insane caricatures designed to make you lose all faith in humanity?
Re: (Score:3, Insightful)
Bad news, those are the real human beings. The tolerable ones you want to rule the place are actually marketing sockpuppets.
Re: (Score:2)
Re: (Score:1)
Get better friends.
Re:How about real problems (Score:5, Funny)
And ... this is different from Slashdot, how? ;-)
Re: (Score:3)
And ... this is different from Slashdot, how? ;-)
Slashdot has more jokes about beowulf clusters. And if you act like a jerk, someone will call you an insenstitve clod.
Re: (Score:3, Funny)
Re: (Score:3)
The jokes here are better.
Re: (Score:1)
Re: (Score:2)
And ... this is different from Slashdot, how? ;-)
Slashdot has more jokes about beowulf clusters. And if you act like a jerk, someone will call you an insenstitve clod.
Is this misidentification of jerks a fairly recent problem? I wonder what steps we could take to rectify this issue...
Re: (Score:2)
I guess Minecraft will stop using it (Score:3, Funny)
Re:I guess Minecraft will stop using it (Score:5, Interesting)
If Facebook likes something, it must be evil.
If facebook buys something outright, it will be turned to social-shit. FTFY.
If I happen to share Zuckershits taste in coffee makers, that won't affect my enjoyment of the coffee maker. On the other hand, if Facebook buys out my favorite coffee maker... then yeah, my next coffee maker will be a brand that doesn't try and change my status to "making facebook coffee" everytime I brew a cup. (Well... I don't have a facebook account so that's moot... although I expect a facebook coffeemaker would require one before allowing coffee to be brewed so... there's that.)
Re: (Score:2)
Re:I guess Minecraft will stop using it (Score:4, Interesting)
Well they "bought out" (that is, "hired") the lead Btrfs developers, so how do you count that?
I still see it as a jointly developed GPL project, that facebook cannot own.
Re: (Score:2)
In the same way that Mozilla cannot "own" Firefox?
Re: (Score:2)
Mozilla owns the Firefox brand, not the code. Hence Iceweasel etc.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Well, considering G+ didn't exist when Youtube was purchased... Its not that good of an analogy.
Try comparing it to the Microsoft skype purchase.
Funny note, Microsoft is now hosting developer video hangouts over G+. Despite owning skype outright, and owning a part of facebook.
I also hate the Youtube G+ Integration, for the opposite reason. G+ is great, youtube and the comments there are horrible and shouldn't be brought to light anywhere civilized people gather.
Re: (Score:3)
Both Instagram and Whats App are working as they originally did prior to their acquisition.
Well except where everything you do is fed into facebook's data mining backend, and its only a matter of time before they add a back channel to start showing you facebook ads.
They will monetize it. Its just a matter of time.
Re: (Score:2)
Re: (Score:2)
So, you're saying that would never happen if they stayed independent?
It would be instagram ads and whatsapp ads. Not facebook ads. That's a big difference.
The amount of information whatsapp has on me is pretty limited compared to the profiles facebook or google are trying to build. I'm ok with small individual companies having small limited profiles of my activity with that company. I strongly object to the macro-scale surveillance that google and facebook attempt.
Re: (Score:2)
Re: (Score:2)
Meh, I'd still rather it be whatsapp than facebook. I perceived them to be much smaller, even if still fairly big.
But for what its worth I've never had a whatsapp account nor an instagram one, nor a facebook one. I find nearly all internet "social" to be a monumental waste of time bundled with massive privacy invasion and avoid it pretty thoroughly.
I do have a skype account though, and would love to find an alternative to that too. Because the ads bug me, and the attachment now to microsoft bugs me.
Re: (Score:2)
Meh, I'd still rather it be whatsapp than facebook. I perceived them to be much smaller, even if still fairly big.
I understand that feeling. But its not based on facts or even objective... People often perceive big companies as "evil", and forget that, in that regard, all companies are evil in some way or another.
But for what its worth I've never had a whatsapp account nor an instagram one, nor a facebook one. I find nearly all internet "social" to be a monumental waste of time bundled with massive privacy invasion and avoid it pretty thoroughly.
Well, I had a whatsapp account and a facebook account. While I tend to agree regarding the waste of time, it is also a way for me to connect with friends and colleagues from past companies, and often discuss technical stuff and what's happening in the world. Isolation is good often, but its not good always. An
Re: (Score:2)
all companies are evil in some way or another.
Yes, all companies are evil. Bigger companies have the capacity to do more evil.
And while I have no objection to the usual "build a real product, sell that product to customers, make money" business model, even if it leads to evil, Facebook (and social in general) have a business model i abhor. So there's that too.
And to top if off, I dislike Zuckerberg intensely as a person, have no respect for him, and wish to do nothing to enrich him, while there are any numb
Re: (Score:2)
And to top if off, I dislike Zuckerberg intensely as a person, have no respect for him,
So. these sound like personal feelings and not a rational evaluation; I',m fine with that, but its not really a valid argument. And regarding Facebok, I'd be really surprised if Zuckerberg had any kind of real control over the company.
here is the service, run it on your own servers, and we'll sell you support.
You realize that, without a centralized system, the social model would not exist, right?
Not the brand, but what the tool is and does. Is it a secure two way messenger that I have complete control over?
Its not. It's more of a 2-way p2p messaging system. And I'd take skype proprietary encryption over any SIP-based hardware device. I'd really doubt that what you describe even exists. And ev
Re: (Score:2)
So. these sound like personal feelings and not a rational evaluation
I don't know how you can presume to weigh in whether or not whether I like someone is "rational" or not.
You realize that, without a centralized system, the social model would not exist, right?
Good riddance. I can be social without "the social model". Because I was participating on forums before facebook. And IRC. And usenet. And email.
And I'd take skype proprietary encryption over [...]
Microsoft can and does monitor the chat. They collect
Re: (Score:1)
The Timeline:
Re: (Score:1)
Dude, seriously, you don't need to provide a link explaining what Jumping the Shark means.
Re: (Score:2, Informative)
While all the hipsters were using Btrfs, the real geeks were using ZFS and not worrying about bad designs such as the inability to query free space ...
https://btrfs.wiki.kernel.org/... [kernel.org]
Re: (Score:3)
Re: (Score:2)
Agreed. ZFS holistic approach of file system, volume manager, and RAID is beautiful, and safer to boot (pardon the pun!)
i.e.
* https://blogs.oracle.com/bonwi... [oracle.com]
* https://blogs.oracle.com/bonwi... [oracle.com]
Sounds about right (Score:5, Funny)
> Btrfs
tl;dr, I assume this is a button to let you tag people as a butterface?
Re: (Score:2)
There may be, but... (Score:2)
There may be such a button, but since the code will be stored on btrfs, it'll corrupt itself in a few months and disappear.
Re: (Score:1)
Re: (Score:2)
That'll work only if it really, truly wants to change.
Thanks, Facebook! (Score:4, Insightful)
FB admins - thank you for paying the developers for the open source work they do. I've been using flashcache with great success in one deployment for almost two years now and am looking to start with hhvm. I didn't even know about the block work.
Obviously kudos to the developers too for spending valuable years on it as well.
Re: Thanks, Facebook! (Score:1)
Re: (Score:2)
While I'll agree that there a few companies that only take advantage of Open Source work, I happen to work for and know several that have full time developers working on nothing but Open Source projects as their full time job. (No, I dislike Facebook and would probably never work there).
I have no idea why so many people try and paint everything as black or white. The world is grey, enjoy it!!
Re: (Score:2)
Where I work, a couple hundred developers and I get paid to work on nothing but GPL software. And that's just our project (which I'll not name, but believe me, it's pretty much everywhere). There's at least that many more getting paid for working on other FOSS projects including the Linux kernel. And it is not a company that people normally associate with FOSS--quite the opposite, in fact.
Re:hhvm (Score:2)
Trial by fire... (Score:4, Insightful)
IMHO, this is a very good thing. btrfs doesn't have as many capabilities that ZFS or Storage Spaces/ReFS possesses.
However, it is finally time that Linux has a filesystem that supports the latest/greatest enterprise features (deduplication and the ability to combat bit rot.)
Realistically, it would be nice to see the native (not FUSE based) code from OpenZFS be included as an alternative, but the CDDL/GPL conflicts likely will make this a no-go.
Unfortunately, the GPL-compatible ZFS ship has sai (Score:2)
What difference would the GPL make to ZFS? (Score:2)
It would be the biggest "fuck you" in the history of open source if ORACLE licensed ZFS as GPLv3 only, as the license would still be incompatible with the Linux Kernel.
The whole reason the CDDL was chosen by Sun was to be incompatible with GPLv2. Oddly enough, the GPLv3 is incompatible with GPLv2 as well.
From a license persepective, it makes no useful difference, as you'd taint the kernel with an incompatible license to run the code whether it's GPLv3 or CDDL.
Re:Unfortunately, the GPL-compatible ZFS ship has (Score:4, Informative)
Re:Trial by fire... (Score:5, Informative)
Realistically, it would be nice to see the native (not FUSE based) code from OpenZFS be included as an alternative, but the CDDL/GPL conflicts likely will make this a no-go.
Well, isn't this your lucky day, then? ZFS on Linux works now, today, without the use of FUSE. [zfsonlinux.org] Nothing about the license conflicts prohibits use or distribution, just distribution together. I have ZFS/Linux servers in production right now, and they are quite stable. Starting with a vanilla install of CentOS, the instructions are roughly:
1) Install the yum repo file.
2) yum Install kernel-devel zfs
3) Start the ZFS service.
4) Start creating ZFS volumes....
A reboot isn't typically necessary... (though not a bad idea)
Re: (Score:1)
Running zfsonlinux here for everything except / and /boot. The last officially supported Linux version is 3.11. I am guessing (hoping) that 0.6.3, the next version of zfsonlinux, will be released soon after Linux 3.14 (with support commensurate support). If you're OK waiting a few kernel versions, or you're OK running HEAD, you'll be happy with zfsonlinux. It has been very stable for me, though I throw a ton of ECC ram at it (like you're supposed to).
Re: (Score:2)
I appreciate that link. That is very useful. It would be nice if it was part of EPEL directly. I thought the ZFS development was dead on this, but apparently there was a release last August.
And an active development... (Score:2)
It also has an active development community; the git repo has regular and frequent commits (for a filesystem). ZFS on Linux seems to test more and release less often -- a fact I appreciate as I haven't lost a single bit of data on my ZFS filesystems, but have lost entire btrfs filesystems multiple times. (Yeah, sure, btrfs is "experimental" and will eat your data... so why is Facebook even thinking about using it?)
Re: (Score:2)
btrfs has been "experimental" for quite a long time (2009?) To compare, Linux went from MINIX's filesystem to ext2 in two years, which lasted quite a while. It takes time to get a filesystem going, but five years is almost an eternity in the computer world, and realistically, Linux should have chucked the LVM2/ext4 combination long ago for ZFS or something ZFS-like.
The good thing is that with FB's devs hammering on btrfs, that will do nothing but improve things and get btrfs ready for prime time use in th
Re: (Score:3, Informative)
Check out: http://open-zfs.org/wiki/Main_... [open-zfs.org]
http://dtrace.org/blogs/ahl/20... [dtrace.org]
ZoL is very active and very up-to-date. All the versions and compatibility is in sync with Illumos (the main source of OpenZFS) and FreeBSD. You can create and move zpools between these 3 platforms seamlessly.
2 of the main founders and creators of ZFS itself (who used to work for Oracle and wrote ZFS) who now work for Delphix and continue to improve OpenZFS (started with the last open release of Oracle ZFS) in Illumos and have ac
Re:Trial by fire... (Score:5, Informative)
> Personally I would be *EXTREMELY* wary of running ZFS on Linux.
So basically you are making an decision based on emotion instead of actual facts??
Try reading the FAQ next time:
http://zfsonlinux.org/faq.html... [zfsonlinux.org]
Re: (Score:1)
Seriously. I've been running ZFS on Linux for 2 years and it's as stable as it ever was on Solaris. I'm 100% confident in it for even Production use.
Re: (Score:3)
A reboot isn't typically necessary... (though not a bad idea)
What is this? Windows?
Re: (Score:2)
Re: (Score:2)
And facebook will be burnt (Score:3)
Not that anybody'll really notice, but I have a feeling that Facebook's backup and recovery system is queuing up for a stress test.
Having lost data with BTRFS multiple times on my disk array (as recently as last month), I have no confidence in it. The best thing I can say about btrfs is is that it was able to tell me that it had lost data. Not many filesystems do that; but ZFS on Linux [zfsonlinux.org] has been rock solid for years, and not only tells me if data has been lost, but actually preserves the data as well.
Re:And facebook will be burnt (Score:4, Interesting)
You are not the only person who have reported data loss on btrfs. Normally I wouldn't worry about Linux filesystems (even ext4 became rock solid after a while.) However, I worry about what I hear from people who use btrfs.
One concern is that a filesystem can't check for bit rot by itself. True bit rot checking requires at least some working with the LVM layer to check CRCs, find a damaged sector and fix it. I've read that btrfs can catch some bitrot issues, (and please correct me if wrong), but it can't catch/correct anywhere near as much as ZFS or Storage Spaces + ReFS can. btrfs also uses a 32 bit CRC, rather than a 64 bit one.
I'm hoping that Facebook's coders can find the issues with btrfs and squash them. There are not many companies with the sheer server use of FB, and if they can get it working solidly, btrfs should be more than ready for prime time for everyone else.
Re: (Score:2)
If BTRFS bit rot detection is anything like ZFS' then it most certainly can check for bit rot and correct it automatically.
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
I would think licensing wouldn't be much of an issue. Facebook probably maintain their own internal custom linux distro. GPL incompatibility between ZFS and the kernel presumably wouldn't be a problem as they wouldn't be distributing it to anyone else.
I could be wrong though :)
Re: (Score:2)
Re: (Score:2)
Well, since ZFS is available for Linux I had to wonder why there would be people making a fuss about btrfs. You bring up licensing which is an issue, and I'm guessing Oracle did not help the license issues, or possibly made the license issues worse.
Well, for starters btrfs plans to have features I need, like reshaping RAID, while ZFS has no plans (that I'm aware of) to add this feature. No, I'm not talking about adding/removing raid from a zpool - I'm talking about adding/removing drives from a RAID while maintaining redundancy while the filesystem is online. mdadm supports this, and so does btrfs (though doing anything with raid5/6 on btrfs is risky right now).
The main strength of ZFS is its maturity/stability. Feature-wise, I'm sure it does somet
Re: (Score:2)
IMHO, this is a very good thing. btrfs doesn't have as many capabilities that ZFS or Storage Spaces/ReFS possesses.
Beyond maturity, what is actually missing? When I look at the feature lists for both if anything it seems like btrfs has more features, like being able to reshape a raid. The last time I checked ZFS supported adding or removing a raid from a zpool, but not adding or removing individual drives from a raid (without degrading it). That is, you can't turn a 4-drive raid5 into a 5-drive raid5 without adding 5 drives and then removing 4.
I'm certainly willing to believe btrfs is missing something, but it has a
Re: (Score:1)
RAID 5/6 is exactly what is needed. The industry seems to swing from advanced hardware RAID to having the OS handle this task. There was a time back in the Ultra 450 days when one had to have Veritas's LVM software because Sun machines would ship as "dumb" JBOD without a hardware controller... then the world moved to SANs, from there to cloud storage, and from there (now that SAN prices are through the stratosphere) back to having the OS do the striping/RAID.
It is good for an OS/filesystem to have this ca
Re: (Score:2)
RAID 5/6 is exactly what is needed.
Btrfs supports RAID 5/6, with reshaping (ZFS does not support the latter).
Then there is the issue of bit rot. A ZFS scrub isn't just an online fsck... it goes through every single sector looking for corruption and either finding it... or if there is redundancy left, fixing it.
Fully supported on btrfs. I do it weekly. All reads are of course checked, but a scrub checks all the disks asynchronously.
Finally there is the issue of snapshots. With ZFS, I can mount a drive, snapshot the entire system, copy that snap onto the mounted drive, dismount it and be on my way, a backup done.
Snapshots are fully supported on btrfs. You can also use send/receive with them which would be more efficient in this use case than just copying the snapshot (which copies all data and not just changes since the last snapshot).
My question was what features does ZFS have which btrfs doesn't have? All of these a
Re: (Score:2)
I'm not sure if all of those are on the roadmap - a few are.
As I said, I think the main issue with btrfs is maturity. Btrfs "supports" raid5. Nobody sane would store anything important on it today. Etc...
I suspect that df and quota support fall into that general category of stuff that is half-done. You can add quotas after creating subvolumes, but the userspace tools don't automatically set them up for existing subvolumes today. There is no reason it couldn't do the job in the future.
Heck, just the out
Re: (Score:2)
btrfs lacks stability and reliability
Hence my original question, "Beyond maturity, what is actually missing?"
Re: (Score:2)
I have been evaluating both recently. In the end, btrfs is mostly to be more capable. I say to be because at the moment, it is not yet mature and some of those capabilities are either absent or not working properly, especially RAID >0. ZFS seems much more mature and it's capabilities work now.
Neither lost data in my brief tests where I abused them with hard resets and disappearing drives (in a VM). However, btrfs got to a point where anything touching it got stuck in the D state, so it might as well have
Re:btrfs? (Score:5, Interesting)
LOL, nice! Or ...
* Broken To Read Free Space
* Broken Treatment Reading Free Space
btrfs FAQ 4.4 - 4.8
* 4.4 Why does df show incorrect free space for my RAID volume?
* 4.5 Aaargh! My filesystem is full, and I've put almost nothing into it!
* 4.6 Why are there so many ways to check the amount of free space?
* 4.6.1 Raw disk usage
* 4.6.2 Actual data
* 4.7 Why is free space so complicated?
* 4.8 Why is there so much space overhead?
https://btrfs.wiki.kernel.org/... [kernel.org]
--
Microsoft Windows 8: A 64-bit compilation of 32 bit extensions and a graphical shell for a 16 bit patch to an 8 bit operating system originally coded for a 4 bit microprocessor written by a 2 bit company that can't stand 1 bit of competition with 0 bit of understanding good UI.
Re: (Score:1)
Microsoft Windows 8: A 64-bit compilation of 32 bit extensions and a graphical shell for a 16 bit patch to an 8 bit operating system originally coded for a 4 bit microprocessor written by a 2 bit company that can't stand 1 bit of competition with 0 bit of understanding good UI.
I am no windows fan but this hoary old cut-and-paste Microsoft/x86 slam is the dumbest fucking ahistorical bullshit and you should be ashamed of repeating it.
1. No part of the x86 family's lineage was ever 4-bit. Despite what you may have heard (from idiots), there is essentially zero compatibility on any level between the 4004 and 8086. There isn't even true compatibility between the 8080 and 8086: although the 8086 was designed to make it easy to port 8-bit 8080 software, Intel did this exclusively at t
Re: (Score:2)
1. Sarcasm.
2. Whoosh.
3. MS-DOS, aka, 86-DOS, shares some of the same design of 8-bit CP/M.
* http://en.wikipedia.org/wiki/M... [wikipedia.org]
Do you understand what a File Control Block is??
Big whoop (Score:2)
As far as I know, the web tier is basically read-only images of the services to be run. The updates and data are on the back end.
So what, precisely, does using Btrfs in such a deployment prove? It's the stability of modified disks that is in question.
Re: (Score:1)
As far as I know, the web tier is basically read-only images of the services to be run.
The web tier will serve cached copies of user pages generated as requests arrive, and then distribute these generated pages as load increases. Given that Facebook has about 18.5% of the species actively using the service every month you can assume there are tens of thousands of page gens per minute into the web tier caches. The storage elements used to achieve that will experience very frequent writes.
Re: (Score:2)
Thanks for the clarification. I hadn't considered caching.
I have an idea (Score:1)
Btrfs definition (Score:5, Informative)
Re: (Score:3)
Re: (Score:2)
btrfs brings really useful features like data integrity protection through the combination of checksums with either multiple copies or error correction codes*, snapshotting** and the ability to create a logical copy of a file without creating a physical copy. These are features that most other linux fileystems don't have. You can find out more at https://btrfs.wiki.kernel.org/... [kernel.org]
Unfortunately it's hard to take a filesystem, particularly a complex one like btrfs from "seems to work fine under our lab tests"
Re: (Score:2)
next-gen is just marketing propaganda for "new, untested, and hype because it is not old 'X' "
Re: (Score:2)
next-gen is just marketing propaganda for "new, untested, and hype because it is not old 'X' "
I understand that may apply to Wayland or Mir, but not filesystems, unless you refer to XFS :D
Also, I understand that ZFS is not just a filesystem, as it also covers the functionalities of volume management, kitchen sink, life, the universe, everything, and emacs. Btrfs is a filesystem in the unix philosophy (unless the name refers to 'butterflysystem', in which case it covers all aforementioned functionalities and then some).
Re: (Score:2)
Btrfs is fairly comparable to ZFS in terms of capabilities/architecture. Personally I tend to prefer the design (devices go directly into pools - you don't have to designate groupings of devices into RAID/etc). Each has some feature the other lacks, but ZFS is more mature.
Ultimately btrfs seems likely to replace ext4 some day, though that day could be quite a ways off. ZFS is unlikely to do so unless the license issue is overcome - sure, you can use it, but there will always be a drive to have the #1 gen
Re: (Score:1)
How is it a filesystem in the unix philsophy? It's monolithic in the worst possible way; a clumsy mess of layering violations. One analogy would be if someone said "http would be so much better if it wasn't for those pesky tcp, ip, and ethernet layers!"
I remain bitter that all of that work into advanced data protection, volume management, and efficiency features was wasted on a single filesystem instead of placed in device mapper where they belong. Then ext4 could have useful features such dedupe, load bala
Re: (Score:2)
--You sound like a Luddite. Sure, in a blue-sky world ALL filesystems could have ZFS capabilities. But the ZFS implementors decided to get past all the cruft and implement it from scratch, and for the most part they've done a fantastic job. Quit complaining and being bitter and try FUNDING CODE DEVELOPMENT on the dev mapper subsystem if you want to see it happen in the next 5-10 years, or it likely never will - it's easier right now to keep those features at the FS level. LVM(+RAID) on Linux is a *horrible*
....and nothing of value was lost (Score:2)