Making ZFS and DTrace Work On Ubuntu Linux 137
New submitter Liberum Vir writes "Many of the people that I talk with who use Solaris-like systems mention ZFS and DTrace as the reasons they simply cannot move to Linux. So, I set out to discover how to make these two technologies work on the latest LTS release of Ubuntu. It turned out to be much easier than I expected. The ports of these technologies have come a long way. If you or someone you know is addicted to a Solaris-like system because of ZFS and DTrace, please, inquire within."
ZFS on Linux (Score:3, Interesting)
So what am I supposed to do about all the kernel panics and absurdly slow IO and transfer speeds?
Re:ZFS on Linux (Score:5, Informative)
DTrace and ZFS are quite mature running under FreeBSD.
Re:ZFS on Linux (Score:5, Interesting)
If you've been playing in Linux land, and never bothered with any of the *BSD, do yourself a favor and install one of the BSDs in a VM. You'll not be disappointed.
Re:ZFS on Linux (Score:5, Informative)
I have to second this... Debian was always my preference but I tried FreeBSD to get ZFS. For dependencies, ports does some things... differently than APT, but they are similar enough that it won't completely shock your system.
And just like Debian, it is easy to start with an extremely minimal system and only add what you need, so stability and boot speeds are excellent.
I think that Debian is still faster at certain things, though that is subjective.
Re: (Score:2)
It will be interesting once Debian's kFreeBSD is out to see whether it supports ZFS & DTrace or not.
One thing I'm wondering - b/w OpenIndiana and FreeBSD, the main difference is that the former is based on SVR4 and the latter on 4.4BSD. Are there differences b/w them that would cause anybody who needs ZFS to prefer either one over the other?
Re: (Score:2)
I use Solaris at work, but as a workstation and not as an admin so while I have a "feel" for it, I can't directly compare them. I would say they are very similar but do have semantic differences... de-facto file locations, behavior of temp directories, etc. I feel that they are more similar than Linux is to either FreeBSD or Solaris.
I looked into both (at the time) OpenSolaris and FreeBSD and decided that FreeBSD's better hardware support was worth the tradeoff in lagging ZFS versions. And not that I antici
Re: (Score:2)
I do know kFreeBSD atleast supports ZFS, pf, pfsync and carp from OpenBSD which are all part of FreeBSD kernel.
Not sure about DTrace, haven't tried that.
Re: (Score:2)
Re: (Score:2)
Yeah, sorry I wasn't more clear. pkg_add still fits into the ports framework, though - so really you can use the two simultaneously and things generally work... most of the time. I've taken to just using ports unless I'm in a huge hurry. You can technically use apt-get to bulid from source as well (-b flag).
Re:ZFS on Linux (Score:4, Informative)
Re:ZFS on Linux (Score:5, Informative)
Re: (Score:2)
Fairy nuff. ZFS has a lot of benefits over the standard Linux filesystems for certain things (just as all the Linux filesystems have their own niches in which they are the supreme overlords). It's rare for me to create a system in which I use fewer than 4 filesystems and if I were to try for a fully-optimized system that would probably go to 5 or 6. ZFS running reliably under Linux would pretty much guarantee me moving to such a model.
Re: (Score:1)
But the last I checked there are at least 30 filesystems available (buildable and bundled) in a standard linux kernel. Some offer backward compatibility to very old systems, some offer unique network support (plan 9 is a different kind of beast, but not bad, just very interesting), and some run under fuse (like zfs) which makes them 'out of kernel', and thus, rather slow. BTRFS is (apparently) coming along. Someone will surely point this out on this forum. Last I tried it, it was still a work in progres
Re: (Score:2)
ReiserFS works extremely well on small files, so I like using it for /tmp. I would use it for /etc, /usr/share and /usr/man as well, but I've had problems where it has corrupted the FS. I can afford to lose /tmp, but /etc is another matter. If I felt safer with it, I'd consider ResierFS to be near-obligatory for any directory in which short files were the norm.
(More partitions is no big deal with gparted-style partition tables. More than 6 partitions was more of a bother under the MSDOS-style partition tabl
Re: (Score:2)
I am terribly disappointed with ext4. If you want a good and stable combination of volume management and a filesystem, I'd recommend LVM (which of all the software solutions I've worked with is by far the best), combined with XFS, which is a stable and reliable journalling filesystem. Where ext4 fails just as terribly as its predecessor, ext3, as far as disk performance goes (kjournald, big kernel lock and the likes), XFS just ploughs on and on.
As for Solaris and ZFS. At the company I work for we still use
Comment removed (Score:4, Insightful)
Re: (Score:2)
Well, you better get used to ZFS. In Solaris 11 not only is it the default, but booting from anything but ZFS is not supported. UFS is supported as a legacy filesystem so I guess you can keep using it for a while, but Oracle is definitely pushing 'ZFS everywhere.'
ZFS works fine on a SAN LUN. It's main drawback in my opinion is that it isn't a clustered filesystem, so sometimes Veritas is still required.
Re: (Score:1)
zfsonlinux is quite solid; I've been using it on all my systems including this laptop for over a year.
Re: (Score:2)
So what am I supposed to do about all the kernel panics and absurdly slow IO and transfer speeds?
I thought ZFS ran in userland on Linux - how does it cause kernel panics?
In any case, I've been running zfs (raid-z) on a home Ubuntu based fileserver for over 2 years without a single kernel panic (record uptime was 9 months before I rebooted to apply updates).
This fileserver is used to stream movies, as well as act as a DVR for 3 home security cameras, and is the backup target for several Windows computers so it gets a fair bit of use.
You're right about slow I/O though.. it's not nearly as fast as hardwa
Re: (Score:3)
This article uses the kernel module, not the userland FUSE stuff.
Re: (Score:2)
Re: (Score:1)
I actually run an OpenIndiana installation on a system with an Atom D510 and two 320GB 2.5" drives. The performance is actually not all that bad. Even so, I didn't really build it to perform very well, I built it to see if I could set up something that eats less than 100 Watts while still offering something remotely resembling a performance. With regard to that, the system is working perfectly ;-).
Re: (Score:3)
Dude, you're doing all that...on an Atom? Doesn't it drag ass? if it were me I'd replace that sucky Atom with a cheap Phenom X4e, those support ECC and can be had for $62. [starmicroinc.net] Figure in $30 for an AM2+ board and $20 for a 2Gb RAM stick and for less than $115 you'd have a machine that would be a HELL of a lot faster than an Atom at multitasking.
It runs surprisingly well, I get around 15MB/second write speeds (and over 30MB/second read) which is more than I need for what I use it for. About the only time I notice it being slow is after I've ripped a movie from DVD and am copying it over to the fileserver. Most of the time I access it via Wifi so the disk is faster than the network. It's used only as a headless fileserver, no windowing system is installed so I don't need to worry about interactive performance.
I thought adding the webcams and zonemin
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Speak for yourself, I've been using it for months without a single kernel panic, and the performance has actually been better than it was on OpenSolaris.
Re:ZFS on Linux (Score:4, Insightful)
I used to have very high expectation of OpenSolaris after Ian Murdock became the head of the project... But then Oracle came and destroyed all my hopes.
BTW, is ZFS SSD-aware?
Re: (Score:1)
....
BTW, is ZFS SSD-aware?
ZFS has had SSD support for the ZIL and L2ARC for years.
Re: (Score:1)
Using a small SSD for L2ARC and ZIL can bring near SSD performance from spinning disk... (near I said, not identical - but damned close)
Re: (Score:2)
IMO L2ARC is just a little too primitive for subset caching for permanent storage ... what is needed is a mix between MFU and Superfetch.
Re:ZFS on Linux (Score:5, Informative)
I used to have very high expectation of OpenSolaris after Ian Murdock became the head of the project... But then Oracle came and destroyed all my hopes.
Good news! Your high expectations and hopes are alive and well at the [open] crossroads of America [openindiana.org] . They're also welcome at freenode on #openindiana.
Re: (Score:3)
not a serious OS (Score:4, Insightful)
OpenIndiana has only made three "development releases" since 2010, it is not a production grade system. Just a hobbyist system.
Re: (Score:3)
Yes, no one would suggest ZFS as the filesystem for your phone. You might have to do things like disable checksums [solarisinternals.com] if you have an older or otherwise underpowered CPU, and it's tuned by default to use memory quite heavily. That anecdote isn't very relevant for today's desktop or server environments though.
Re: (Score:2)
Yes, no one would suggest ZFS as the filesystem for your phone
Actually, some guys at Samsung would. They trimmed a lot of the optional features (made them optional at compile time, rather than run time), reduced the size of the ARC to 4MB, and got ZFS running reasonably well on a machine with 16MB of RAM. We're investigating bringing some of their changes into FreeBSD for ARM systems, but it's starting to look as if it would be one of those things where by the time it's done the hardware has improved to the point that it's no longer worthwhile.
Re: (Score:3)
Interresting enough btrfs works really well on a phone.
Re: (Score:1)
If you think a SSD is the way to go you don't need zfs because you already show that you don't care about your data. I use a ssd for speed in applications where I don't care if the data is lost. How does your ssd drive with fat32 ensure the integrity of the files themselves. I'm not just talking about file system metadata. Hopefully people won't pay heed to ignorant hipsters like yourself. Then again if the goal is data integrity running the Linux port of zfs isn't the answer either.
One of the jewels of zfs
Re: (Score:1)
Re: (Score:2)
I don't know about "heavy"... I have it running on a 6-year-old HP xw4400. Sure it's a nice old machine with a 64-bit processor (required for stability) and ECC RAM (required to get the real data integrity benefits of ZFS), but you can get a HP Microserver new for under $400 that has both of those things.
Re: (Score:2)
Re: (Score:1)
So what am I supposed to do about all the kernel panics and absurdly slow IO and transfer speeds?
Learn to use and configure linux
Not quite that simple. A number of presumptions that zfs makes about the underlying OS are in conflict with the linux kernel approaches (zfs was written for Solaris after all).
Issue is not implementation (Score:2, Informative)
The issue with ZFS and Linux has always been more about copyright than implementation.
Re:Issue is not implementation (Score:5, Interesting)
Re: (Score:2)
Which is what makes me wonder - why are people trying this? If ZFS is what is needed, why not go w/ a perfectly good and compatible FOSS unix, such as either OpenIndiana or FreeBSD? The former has the same license as ZFS, while the latter's license is not incompatible w/ CDDL.
If the objective was to get some of the nice things from the Linux world, that might be possible w/ Debian kFreeBSD, which has some support for ZFS [debian.org]. Might be a better option than Linux.
Re: (Score:1)
Because a filesystem isn't an operating system. You are asking why people don't throw the baby out with the bathwater.
Re: (Score:2)
Because a filesystem isn't an operating system. You are asking why people don't throw the baby out with the bathwater.
Ah, but the implementation under the various operating systems can have a lot of effects that are relatively unknown until you hit them. Yes, the ZFS Kernel Module programmers are excellent about reponsiveness, but that doesn't help much when you've got a ZPOOL that's suddenly dropped offline and because the other ZPOOLS are still running you can't take an outage to reboot.
I've played with ZFS under Ubuntu 10.04, BSD and OpenSolaris (and recently OpenIndiana) and there are "quirks" to each implementation th
Wrong about license (Score:4, Informative)
The DTrace integration is via a kernel module, so the license on DTrace is irrelevant..
There are a couple of interfaces in Linux that should be externalized for getting stack tracebacks into user space in a standard manner without caring about binary architecture (they are currently static). I've personally used a modified Linux with DTrace mods and these functions externalized, and it's rather stable and usable. Specultive tracing is also a lot better for finding the origin of some random errno in the kernel, or who in user space is calling gettimeofday() a bazillion times in order to time stamp X events.
Obligatory disclosure: I was on the team that did the DTrace port to Mac OS X.
-- Terry
Re: (Score:2)
Re: (Score:2)
They're not distributing binary packages. The Ubuntu packages have the source code and the DKMS config to build the thing, and have the package pre-requisites such that everything "just works".
Re: (Score:2)
Sun doesn't exist anymore, it's all Oracle now.
OK Howto article, but missing key points (Score:2)
I just looked at this article as my employer uses Debian and Ubuntu heavily and I've been pushing for ZFS on our file servers. There is no mention of ZFS version, the feature set available, or even a link to the source material.
There isn't much mention of how to use ZFS. I happen to know most commands, but I think this article would be difficult for a beginner even though it seems to be targeted at that demographic.
Re:OK Howto article, but missing key points (Score:5, Informative)
I just looked at this article as my employer uses Debian and Ubuntu heavily and I've been pushing for ZFS on our file servers. There is no mention of ZFS version, the feature set available, or even a link to the source material.
ZoL is based on ZFS version 28 from the last open Solaris release, and currently integrating Illumos as its upstream.
There isn't much mention of how to use ZFS. I happen to know most commands, but I think this article would be difficult for a beginner even though it seems to be targeted at that demographic.
It looks like the Slashdot editors are doing this blogger a favor by linking to a mostly empty article.
At a minimum, this article should link to the ZoL home page [zfsonlinux.org], the ZoL Launchpad page [launchpad.net] for packages, and maybe the ZFS introduction [opensolaris.org] or another tutorial.
Re: (Score:1)
Re: (Score:2)
Re: (Score:1)
If a debian shop, kfreebsd might be the way to go.
Debian is kernel agnostic, several choices including freebsd (kfreebsd should be ready for when wheezy goes stable).
You get a fully baked distro with everything you expect from Debian (which is ummm... pretty much everything you could ask for already vetted and pre-packaged-- with ease of management fully thought out), and the stability of zfs on a freebsd kernel (which lately has become pretty solid).
I work in a Debian shop too, and really prefer Debian to
Re: (Score:2)
Huh? (Score:5, Interesting)
So there's a list of 10 steps to install zfs and that's it? Didn't do anything? zfs/zpool upgrade -v? zvols? zfs send/receive? snapshots? rollback? Scrub? Performance tests? Compression? Encryption? Can I export my pool from my Solaris 11 SPARC system and import it into linux, make some changes and then move it? L2ARC support? Separate ZIL support? Case sensitivity?
I know this isn't exactly a great comment, but is it at all possible that someone make a judgement as to the value and truth of a submission before putting it up?
Re:Huh? (Score:4, Interesting)
Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. It performs fairly well in my testing so far. Yes. Yes. Yes, if the pool version is below the currently supported Linux port's version (28). Yes. Yes.
Granted, we haven't been using it long, but so far it's been fairly stable and capable.
http://zfsonlinux.org/ [zfsonlinux.org]
Re: (Score:2)
Re: (Score:2)
Mostly all yes though performance is spotty in some use cases. The code is evolving rapidly - it's always been safe, but sometimes new releases have regressions. The developers usually fix these quickly.
I'm using a dm-crypt volume under ZFS where I need encryption as that hasn't made it out yet.
I've been using it in test since August, for in-hosue production since January, but I'm not yet recommending it to clients because of the rough bits.
Now that it's been Oracled... (Score:4, Interesting)
I've been running ZFS on FreeBSD for a few years and it's lived up to its promises, but I think I'll be migrating off of it. The problem is that I trusted Sun. They did some goofy things, but you knew where you stood with them. They release ZFS under an Open Source license? You could take them at face value and know that you were allowed to use it. But now that Oracle holds the reins, I have no desire to depend on any Sun-borne projects anymore. Yes, ZFS is Open Source. So was Java, and Google just spent roughly a bazillion dollars defending themselves for using something that looked like it. I can't afford to take on a case like that.
Other than the Oracle-owned btrfs, what ZFS alternatives are available and ready for use today?
Re: (Score:1)
Re:Now that it's been Oracled... (Score:5, Insightful)
So was Java, and Google just spent roughly a bazillion dollars defending themselves for using something that looked like it. I can't afford to take on a case like that.
So you take the Oracle vs. Google case as Oracle eventually going after individual users of legitimately licensed code?
Nonsense.
As much as I think Larry Ellison is a douchebag, he is motivated by profit. The results of this last case were less than optimum for him, going away from the case with bupkis and a bunch of fees from BS&F. Alsup also established he fact that independent implementation of APIs are not copyright violations, ever, under current law, which had not been proven until now, which is a big win for everyone including Google, and a stupendous loss for Oracle.
Larry Ellison learned an expensive (David Boies doesn't come cheap) lesson here, that even his bluster and hubris doesn't win court cases.
Google was not the loser here.
ZFS and btrfs have free licenses and it's tough to put the worms back in the can once something is under a free license. Forks happen. Look at what happened to OpenOffice and Libre Office. Sure, Oracle can close off future code, but Very Useful Stuff like this gets forked by the community. There are enough smart people poking around in the guts of ZFS and btrfs that *do not* work for Oracle and the projects will continue on in the community even if only to give Oracle the finger.
Your fears are overblown.
--
BMO
Re: (Score:2)
Suing users is completely plausible for Oracle. The case would be meritless, but I don't have the time and money to go up against Oracle in court.
Re: (Score:2)
1. You are not that important
2. More serious answer: Going after end users is a waste of resources and Larry isn't as dumb as Darl McBride.
--
BMO
Re: (Score:2)
1. I'm a disposable cog, but my employeer may make waves... ... and if those waves compete with Oracle's, they may start an IP pissing match.
2.
Re: (Score:2)
See, here's the thing...
If you, or your company, files lawsuits repeatedly for meritless reasons, you can rapidly find yourself being classified as a "vexatious litigant" where you need a court's approval before even filing a lawsuit.
Sure, Oracle can sue for anything. If they make an ass of themselves in the system, the system can slap them back. It happens. It happens with these "Sue 5,000 John Does" that various extortion-firms have filed against copyright infringers, like what happened to Righthaven,
Re: (Score:2)
David Boies has had a frustrating decade in tech work. He won the Microsoft antitrust case only to have it overturned. He lost representing SCO va unix, he lost representing napster and now he lost it for Oracle. He's a legal superman, but he should stay away from tech for a while. . .
Re: (Score:2)
David Boies signed the Contract From Hell vis-a-vis SCO.
The contract had a cap on fees, and BS&F was to represent SCO vs IBM until the heat-death of the Universe, because they stood to gain a portion of the FIVE BILLION DOLLARS IBM was supposed to cough up.
--
BMO
Re: (Score:2)
Wow, it's almost like the dude got 419d!
Re: (Score:2)
There is no scam without a mark.
--
BMO
Re: (Score:3, Informative)
Check out Dragonfly BSD's Hammer FS. It has offline dedup that doesn't require much mem at all (unlike ZFS's online dedup), some other features to ensure data integrity. It is not the life the universe and everything approach of zfs and btrfs, but it has much of what would be traditionally thought of as the fs part of the functionality.
Re: (Score:2)
My preference would be for the wide range of open source u*ix-style OS' to get together, hammer out a cross-os VFS layer and thus reduce the discussions of FS' to technical points on the FS itself, eliminating the OS from the equation. There is nothing inherent about mapping/remapping/versioning/distributing physical data in logical files to blocks of data that is the least-bit OS-specific.
Re: (Score:2)
That's a great idea - HAMMER FS. Comes under the BSD license, and is developed by DragonFlyBSD to start w/.
Question - is Veritas' file system under any open source license, or is it only closed source?
Re: (Score:2)
The Oracle/Solaris version of ZFS is dead to the open-source world. Almost all of the original ZFS developers left Oracle and are working at other companies developing OpenSolaris (now called Illumos). That means that the open source version of ZFS is getting better all the time, and the Linux ZFS code is pulling fixes and features from Illumos, so it isn't just sitting still either.
Oracle doesn't "hold the reigns" on ZFS anymore, though they may like to think that. That is the benefit of open source soft
Re: (Score:2)
Other than the Oracle-owned btrfs, what ZFS alternatives are available and ready for use today?
The only serious filesystem with similar features (B-Trees everywhere, hashing for integrity, etc...) that I know of is Microsoft ReFS [wikipedia.org]. It's still beta, and will be missing key features of even NTFS when released, so it won't exactly match up to a mature filesytem like ZFS.
Re: (Score:2)
If you want a BSD-licensed alternative, try DragonFly BSD. It comes with the HAMMER filesystem, that provides some (most?) of ZFS funcionality.
Re: (Score:2)
One more alternative - AdvFS. That was the file system of DEC's OSF/1 a.k.a. Digital Unix a.k.a. Tru64 Unix. When HP decided that they had nothing to gain by moving it to the Itanium, they made this file system GPLv2 in 2008. It is available on Sourceforge.
I have no idea about how it compares to Veritas, ZFS, XFS, HAMMER, BTRFS, ext4, or anything else. That's one more thing you could go to under Linux, or even BSD, since OSF/1 was originally a BSD based unix.
Re: (Score:1)
Not so fast.
While the btrfs engineers have not pursued patents, it is a ZFS knock-off, and Sun patented the hell out of ZFS (all those $2000 patent bonuses for the ZFS engineering team), and Oracle now owns those patents. btrfs is GPLv2, which does not cover patents. ZFS is CDDL, which does.
Liicensing? (Score:4, Informative)
I always thought the hold up on ZFS and DTrace on linux was the fact the CDDL and GPL didn't play nicely with each other. It was never a technical reason.
I've been running both on FreeBSD for a couple years now. Still don't have any production machines with ZFS yet, but I've found DTrace to be a life saver on more than a few occations.
Used both on Linux: ZFS is great, Dtrace unstable (Score:5, Interesting)
I use ZFS on Ubuntu 11.10 in "production" for my main workstation and fileserver with a 3x3TB raidz pool with an L2 ARC. I/O is blindingly fast, and it has been rock solid. It serves about 10 machines, and feels an order of magnitude faster than the md/lvm based xfs array it replaced.
I write 10GbE drivers for Linux, MacOSX, FreeBSD and Solaris. I make heavy use of Dtrace for both debugging and performance analysis. I feel naked without Dtrace, and I've used the linux dtrace a few times for debugging. Unfortunately, I've never had dtrace run on linux for more than a few minutes without crashing a machine. This is not necessarily bad, and often just a few seconds is all I need. But I would never run linux Dtrace on any production machine, whereas I use it all the time under Solaris / FreeBSD and MacOSX and often have customers run Dtrace probes on those OSes to diagnose issues.
Re: (Score:3)
I've been using ZFS on Linux also a while on my Ubuntu based backup/media box. No problems so far, and the average transfer rate of a 100 GB disk image has been 50 MB/s from internal drive A to internal drive B (non RAIDed, Asus E35M1-I DELUXE Mini ITX with 8GB of mem and 2*Western Digital Caviar Green 2TB SATAIII 64MB,). The CPU usage hits maximum while transferring, and ZFS also uses most of the RAM quite efficiently.
What about Dtrace replacements (Score:2)
Have you tried one of the following replacements?
http://sourceware.org/systemtap/wiki/SystemtapDtraceComparison [sourceware.org]
Low bar for entry (Score:5, Insightful)
So an article lacking knowledge of the technologies, any sort of testing, anything beyond "make install" or "apt-get install", will make it to the Slashdot homepage? This person openly admits that they didn't test ZFS beyond creating a zpool, and they don't know enough about DTrace to try... anything.
As an aside, why was Linux capitalized, but Solaris was not?
I don't see Solaris users migrating (Score:2, Informative)
3) Partition the new drives.
)9 .... “sudo zpool create zfs-blog raidz /dev/sdb1 /dev/sdc1”
Ha ha ha. You know part of the magic of ZFS is management of the entire disk drive. No partitioning
Look: The ZFS on Linux project is a noble effort, and I am sure many Linux users will eventually benefit, and it will maybe be good enough for them to not switch to Solaris.
But none of this stuff is production quality yet on Linux, and the performance VS Solaris is questionable. Linux doesn't h
Re: (Score:2)
Re: (Score:2)
> > Ha ha ha. You know part of the magic of ZFS is management of the entire disk drive. No partitioning /dev/sdb and /dev/sdc instead.
>
> You don't need to partition, you can use
Actually, i tried that, it give me an error.
Re: (Score:3)
You don't need to partition, you can use /dev/sdb and /dev/sdc instead. My question to you is, is Linux doing something that Solaris doesn't support? ie: Capable of running ZFS in a partition instead of using whole disk?
Solaris is capable of running ZFS on a disk slice, and it's something I will rarely use, but I will use it for the boot media, when running the ZFS boot volume off a CF card, I will mirror it to a small slice on a pair of hard drives, and have another slice on those two drives mirrored f
Re: (Score:2)
What you must never do is utilize a SSD slice for the ZIL.
I've been looking for a valid reference for this statement for a while. I've had a couple people tell me this and simply insist with no valid reasoning.
I understand that ZFS can't send a cache flush to a slice, but if I'm using a SSD with a supercap (say Intel 320) I have never heard any valid argument or reference. I have a LOT to gain from not allowing ZFS to 'use' the whole disk; My 300GB SSD can write something in the neighborhood of 600-800TB if I hand the whole thing to ZFS before I run out the m
Re: (Score:2)
I hand the whole thing to ZFS before I run out the media wear indicator. The same disk vastly under-provisioned (giving ZFS a 15GB slice) my media wear indicator will last about 4.2PB (yes, 4200TB).
There is no point in getting a 300GB SSD to use as ZIL, because the maximum amount of disk space that ZFS will use on the ZIL is one half of the system's RAM; you would literally need 600GB of RAM, before a 300GB could possibly be utilized as ZIL by ZFS.. It would not be necessary to "slice" or "partitio
“sudo make all”?! (Score:1)
Why “sudo make all” when “make all” will do?
Just one complaint (Score:3)
I've been using zfsonlinux for a few months now, ever since I migrated my file server from OpenSolaris to Ubuntu Server, and I've generally been pretty happy. It's been stable and fast (faster than osol was, anyhow). My only complaint is that mounting filesystems on boot seems eternally broken.
In previous options, there was a config file option for a workaround, and the filesystems usually (but not always) got mounted on boot. Then that solution was removed in favour of an updated mountall package; unfortunately, this new solution never works. I'll boot the system, no filesystems mounted, but running mountall from the command prompt gets everything mounted OK... Sigh.
Re: (Score:2)
My only complaint is that mounting filesystems on boot seems eternally broken.
yeah, there's udev work to be done yet. I think there's an open bug for that.
I currently use a shell script to interrogate and export/import my zvols to get everything up at boot. Messy.
Re: (Score:2)
It seems like I should probably be able to add mountall to a startup script, but I shouldn't have to.
So what about performance ? (Score:3)
I did a quick test with 2 identical VMs on my desktop with Intel SSD, I installed the ubuntu-zfs as from the article and I installed btrfs-tools.
The VMs have 4 CPUs and 4GB of memory, 3 virtual disks.
The btrfs has RAID1 data and meta data, the ZFS setup used RAIDZ as in the article:
mkfs.btrfs -m raid1 -d raid1 /dev/vdb1 /dev/vdc1
(I needed to create the partitions, for some reason the ZFS version didn't want to work without it)
My quick stupid test, create a large file:
ZFS:
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 16.8489 s, 31.1 MB/s
real 0m16.853s
user 0m0.000s
sys 0m0.480s
btrfs:
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 15.232 s, 34.4 MB/s
real 0m15.234s
user 0m0.000s
sys 0m0.640s
Re: (Score:2)
I should add that ZFS uses more CPU.
Good Article (Score:4)
Congrats... this is a good summary on getting these working under Ubuntu. I did the ZFS install "naked" (without a summary as good as this) with a 10.04 box about a year ago and it has run great guns. Now, having said that it's good for what I use it for which is a temporary location to dump my SQL backups to from a large email archive using dedupe prior to running it off to tape... and another zpool mounted as an archive VMFS volume through NFS to our VMware farm so we can archive decommissioned virtual machines for 30 days prior to deletion per our policy. I am not 100% convinced I would use it for anything production though; supportability is still an issue with this and as such I remain a little dubious whereas with most of our system I can call a vendor and have them fix it. As the storage admin I find this a great way to keep up with the demands for storage while having a relatively transparent way (for my admins) to put stuff into a place where it doesn't take up so much space.
Now having said that there are some caveats; as the zpool gets really large the ability to delete files becomes slower and slower when it's deduped. This is because a lot of database transactions take place to remove the files particularly when there's a lot of deduplicated blocks... and this problem is a lot worse under Ubuntu than it was under OpenSolaris (which is where I first played with ZFS). There are times also that when reading the SQL backups to dump them to tape it can make both storage pools unresponsive enough that VMware drops the NFS datastore and I have to manually remount them. Far less than perfect... but good enough for what we use it for.
I have recently taken a decommissioned physical server (a DL380 G5 with two processors and 16GB of RAM) and put OpenIndiana on it to play with ZFS some more and it is working fantastically well. In my tests though it still has the slowdown issues, high utilization in one pool won't cause the other pool to grind to a halt when both are deduped. Also, it's been nice to (at least in test) create a ZVOL on my ZFS and present it through fiber-channel to my VMware hosts as a potential replacement for the NFS volume on Linux (I have only Emulex cards, and I have yet to see a properly working Emulex target mode under Linux). So far my testing has gone marvelously and I have found dedupe rates to be about the same as the NFS mounted volume... though slightly lower. I suspect that's probably because the data isn't really block aligned all that well but it still saves me a bunch of storage when we have 30 almost identical virtual machines being archived! On the bright side there I have not yet seen utilization get so high on the OI box that it causes any significant issues or dropouts that cause VMware to complain; so far it's been rock solid. I may migrate my ZFS stuff to the OI box and get it off my Ubuntu box... but at the moment they're both working great and I have no complaints.
Re: (Score:3)
Oh yeah, and for bonus points if you have an OCD admin who is allergic to the command line, you can easily put Napp-It [napp-it.org] on your OpenIndiana server to allow them some visibility into the stats or even create their own filesystems. Simplifies my job as I can give them a zpool to play with and let them build ZFS shares or ZVOLs to their heart's content. Now whether they actually understand the statistics... well that's a different matter but at least you can tell your boss you're giving them the tools they need
Re: (Score:1)
ps.: long time Solaris user here. don't see the appeal anymore. the illumos/openindiana project suffer form the same short slightness that caused opensolaris to fail (couldn't expect anything different if the same sun people are major forces behind it).
Re: (Score:2)
There's things Solaris has that Linux doesn't, there's things Linux has that Solaris doesn't, there's things Inferno has that neither Solaris nor Linux have, there's things Hao Ya tea has that no OS will ever contain. Interoperability is best when essentially external components are portable, tea is best when hot and not Earl Grey.