XFS Merged into Linux 2.4

Alphix writes "As noted on KernelTrap Marcelo has merged XFS into 2.4 after a code review by Christoph Hellwig. The mail from Marcelo on LKML is here. Apparently it touched very little VFS code so people not using XFS shouldn't see any ill effects from this (it's even supposed to fix some VFS bugs). XFS is described by SGI as '...a journalling filesystem developed by SGI and used in SGI's IRIX operating system. It is now also available under GPL for linux. It is extremely scalable, using btrees extensively to support large and/or sparse files, and extremely large directories. The journalling capability means no more waiting for fsck's or worrying about meta-data corruption.' Let the stability vs. new-features flamewar begin."

ext3vs XFS?

[dummkopf]

back in the days when ext3 was still in our dreams i downloaded the SGI XFS kernel from their site and installed it on my wife's laptop. it was extremely stable and had the advantage, that her "oops, i have to run off and just close the lid"-atacks would not corrupt the filesystem (which i would have to clean up...).

nowadays i use ext3 on my machines because it comes default with RH (by the way EL is now available for academia, woohoo!). hence my question:

can someone offer a nice comparison of ext3 versus XFS?

Re:ext3vs XFS?

[Trigun]

Here [zemris.fer.hr] ya go.

Re:ext3vs XFS?

[Three Letter Acronym]

from the article:

"Of course ext2 isn't a journaling filesystem and therefore it isn't a journaling filesystem and therefore has less advantages then xfs, jfs, reiserfs, ext3."

you see, i just find stupid stuff like this funny and therefore i just find stupid stuff like this funny. So sue me and sue me!

Re:ext3vs XFS?

[Merlin42]

in response to:
http://aurora.zemris.fer.hr/filesystems/
Th i s seems like a pretty poorly designed benchmark. One of the major tests was copying b/w two partitions (which is a valid test), but they put both partitions on the same disk! Whichever partition hapened to be allocated near the outside edge of the disk would have a clear advantage. Also it is not clear if the read, write, and delete portions of the test were done using the exact same partition or if some filesystems were handicapped by being on the

Re:ext3vs XFS?

[Gudlyf]

You can always look back at this old Slashdot article [slashdot.org].

Re:ext3vs XFS?

[HidingMyName]

Does linux have an XFS dump/restore ported to it? That makes a difference for our installation. Currently we use ext3fs so that we can dump to tape (in spite of Linus's hate for dump, the admin features of dump are very useful).

Re:ext3vs XFS?

[_|()|\|]

Does linux have an XFS dump/restore ported to it?

Yes [sgi.com].

Re:ext3vs XFS?

[Nothinman]

I hope you verify those backups because Linus' hate for dump has a reason behind it.

Re:ext3vs XFS?

[mcbridematt]

yeah, just hold the poweroff button :)

For me, ext3 doesn't offer much data redundancy over plain old ext2. It still suffers from ext2's dataloss problems. Infact, with ext3, I had the horror of i/o errors. Once I had a bad powerdown, and I came close to reformatting just because it wouldn't let me into php.conf.

Personally, anyone looking for a data-redundancy fix should use ReiserFS (which we have had for a looong time), JFS or XFS.

Re:ext3vs XFS?

[Nothinman]

Personally, anyone looking for a data-redundancy fix should use ReiserFS (which we have had for a looong time), JFS or XFS.

I would definatley not recommend reiserfs if you care about data. I've had atleast 2 occiasions where it lost data for no good reason. I would rather use ext2 than reiserfs any more, but when available I use XFS.

Re:ext3vs XFS?

[_|()|\|]

can someone offer a nice comparison of ext3 versus XFS?

Ext3 can grow or shrink an unmounted file system. XFS can grow a mounted file system.

Ext3 and XFS both have dump utilities, which many sys admins prefer for backup.

Ext3 supports three modes of journaling: writeback (risky metadata only), ordered (metadata only), and journal (all data). I believe XFS is comparable to ordered ext3.

Ext3 has been widely deployed on Linux, and it trivially reverts to ext2. The XFS design is mature, but its implementation on Linux is less proven.

Re:ext3vs XFS?

[Fruit]

ext2dump is unsupported; in particular I recall a quote from Linus to the extent that anyone who uses ext2dump might just as well not make backups at all.

xfsdump on the other hand will work correctly.

Re:ext3vs XFS?

[Nothinman]

dump is not recommended with ext2 or ext3 because it opens the block device directly which bypasses the page cache and can give you corrupt data if there are dirty pages that havn't been flushed to disk.

I'm not sure if xfsdump is any smarter about it because of the DMAPI stuff available, but I'd be carefull.

Re:ext3vs XFS?

[_|()|\|]

dump is not recommended with ext2 or ext3

Dumping a live file system can result in bad backups. Dump is best used on an unmounted file system. It can also work well with snapshots or split mirrors.

Re:ext3vs XFS?

[MSG]

xfsdump is definitely smarter because of DMAPI, and is safe to use on live filesystems.

Can't XFS also dynamically create more inodes?

[emil]

AFAIK, ext3 can't, so when you run out, you get to backup, recreate the filesystem with more, then restore.

Quotas!

[mattbee]

From what I could find out XFS is the only Linux filesystem which stores quota information as meta-data-- there's no risk of an XFS filesystem getting its quotas "out of sync" with the contents of the disc and having to run a tedious quotacheck. We recently deployed it as a backup server and it's working very well!

No quotacheck

[Booker]

XFS journals quota info, so you won't have to
wait for a quotacheck on a huge filesystem.

Benchmarks

[Space cowboy]

Any up-to-date comparisons between the 3 main journalling filesystems (ext3, xfs, reiserfs), for both speed and reliability ?

I like xfs on the SGI - it's never let me down yet. I have to admit I'll be sorely tempted to try out xfs now that it's passed the 'seal-of-approval' and made it into the kernel - surely the best benchmark of all :-)

Simon

Re:Benchmarks

[martinde]

Don't forget IBM's JFS, it's in 2.4 AFAIK, and the last time that there were benchmarks linked from slashdot [slashdot.org], it actually seemed the best overall, even over the highly anticipated reiser4.

Re:Benchmarks

[Thaddeus]

Unfortunately, JFS requires you to fsck an unclean filesystem and the process is quite slow, whereas XFS replays the journal automatically. Also JFS has no quota support.

Re:Benchmarks

[noselasd]

No it doesn't. replaying the jfs journal(initiated by fsck) is just as fast as xfs.

Re:Benchmarks

[Thaddeus]

Maybe on a small filesystem... on LBD's, the difference is quite noticable.

Re:Benchmarks

[AlxRogan]

http://www.newsforge.com/os/03/10/07/196222.shtml? tid=2

Re:Benchmarks

[Space cowboy]

Cheers guys :-)

Simon

Stability has been there for a long time.

[Anonymous Coward]

Let the stability vs. new-features flamewar begin.

It's already been stable for years, since VERY early in the 2.4.x cycle. It's just a detail in the naming that makes it merged as part of 2.4.x itself.

Careful with LILO

[slashnik]

Be careful those of you who still use lilo

Q: Does LILO work with XFS?
This depens on where you install LILO. For MBR installation: Yes. For root partitions: No, because the XFS superblock goes where LILO would be installed. This is to maintain compatibility with the Irix on-disk format. This will not be changed. Putting the Superblock on the swap partition is reported to work but not guaranteed.

Re:Careful with LILO

[mark_lybarger]

LILO? dude, that's like, so 199^N^N^N^ er, debianish. that bacon's done moved over ages ago.

Re:Careful with LILO

[mst76]

LILO? dude, that's like, so 199^N^N^N^ er, debianish. that bacon's done moved over ages ago.

What else would you use? According to their website, GRUB [gnu.org] is not quite ready for public consumption yet.

Re:Careful with LILO

[Dan-DAFC]

And Mandrake I believe, though it does give you a choice.

Re:Careful with LILO

[Jeffrey Baker]

I guess you've never run into LILO's "timestamp mismatch" error, which is undocumented and has nothing to do with timestamps. It prevents machines with large numbers of SCSI devices from booting. This is also precisely the market XFS serves.

GRUB is good. Boots anything. Wish we had OF.

Re:Careful with LILO

[Anonym0us Cow Herd]

I was not aware that the alternatives were bug ridden. I use Grub because it was what SuSE 8.2 wanted to run by default. My first reaction was "Oh God, why did they have to change what I was already familiar with?". But after reading up on Grub, I would never want to go back to LILO. Grub is extremely convenient. What you can do at boot time is more powerful. Changing kernels or initrd images is much easier. You can just rename kernel files so that the new kernel has the name of your old kernel. Gru

Re:Careful with LILO

[Anonym0us Cow Herd]

Initrd is something for the good new days, not good old days. The idea is that during boot, the boot loader load two things: (1) kernel, and (2) initrd. The initrd is a ramdisk image of a smallish filesystem. The kernel is started with the smallish filesystem. The kernel runs a program from this filesystem prior to completing the boot process and running /sbin/init. The program that gets run prior to boot completion could do anything you needed to do prior to starting /sbin/init. Some useful ideas...

Re:Careful with LILO

[Vlad_the_Inhaler]

try: man initrd
for a start. SuSE (for example) have the support for all filesystems except ext2 in as modules. They use the initrd mechanism to load the rest of the stuff. This is not an approach I like - it means one more thing that can go horribly wrong if you compile your own kernel - but that is another matter.

Re:Careful with LILO

[big_groo]

Doesn't everyone keep a boot disk or 2 lying around?

That simple exercise has saved me a couple of times.

Re:Careful with LILO

[Blob Pet]

does the same apply with grub?

Re:Careful with LILO

[slashnik]

Grub's OK

Q: Does GRUB work with XFS?
Yes there is native XFS filesystem support for GRUB starting with version 0.91 and up. There is a GRUB rpm that supports XFS in the download section for the 1.0.2 installer on the FTP sites.

Re:Careful with LILO

[Tux2000]

Since the old days when there was a 1024 cylinder limit for the ROM BIOS and so for LILO, I usually have a very small partition (10 to 50 MBytes) somewhere in the first 1024 cylinders mounted at /boot. LILO is installed in that partition's super block, and the partition is marked active.

So if I would use XFS for / and other partitions, I still would keep that trusted little helper /boot partition with an ext2/ext3/minix/whatever filesystem that works with LILO.

Minor drawback: You have to uncomment export

An Overview

[Gudlyf]

SGI has an overview on the XFS filesystem [sgi.com], just briefly pointing out some highlights. I also recall reading somewhere that it was possible (moreso than ext* filesystems) to undelete files on an XFS filesystem, although I'm skeptical.

Re:An Overview

[Fruit]

No. That is wrong. It's usually *harder* to retrieve a deleted file from an XFS filesystem than from ext2/ext3, not in the first place because the on-disk structures are more complicated.

Comparison

[Alphix]

For all those that are looking for a filesystem comparison, I found this story [kerneltrap.org] to be quite interesting...or go here [netnation.com] for the test details and results.

XFS Rocks

[fmlug.org]

I use XFS on serveral different servers, mainly because I belive it performs better then ext3, or any other fs. Also because Alot of the servers I run are samba servers and the ACL support is built native into XFS. And last I looked ACL support was still not quite stable in ext2/3 it has been awhile so it could be stable by now.

Re:XFS Rocks

[oracleofbargth]

And last I looked ACL support was still not quite stable in ext2/3 it has been awhile so it could be stable by now.

As of the patch for kernels 2.4.19+, acl support is very stable for ext[23]. In fact, I've been using it in production for over 2 years now. (I did help write some of the ext3-xattr+acl code, though, so maybe that means I'm a little bit more trusting of the code.)

The only big issues I've ever had is when using them in conjunction with quotas, but even when stress testing the filesystem, I

I love XFS

[Apreche]

Back in the day I would always just use ext2. Then I realized, hey there are other filesystems to choose from, so I decided to shoot in the dark and try reiser and xfs and ext3. xfs has always been the most awesome.

Just one thing. Now that we've got the source code for dealing with xfs, can someone write a driver so I can mount my xfs partitions from windows xp? It would really help out a lot of us dual booting types. I would do it myself, but I don't know jack about how filesystems work. I just know which ones do what.

I hope they put the xfs into 2.6 also. Maybe it wont be necessary to have seperate xfs-sources in gentoo anymore and xfs will finally be included in the gentoo-sources.

Re:I love XFS

[kill-1]

XFS has been in 2.6 for a long time. It was merged early during the 2.5 development cycle.

Re:I love XFS

[glwtta]

Maybe it wont be necessary to have seperate xfs-sources in gentoo anymore and xfs will finally be included in the gentoo-sources.

Well, yeah, that's pretty much the gist of merging features into the kernel - distro maintainers don't have to patch them separately.

Re:I love XFS

[pantherace]

Obviously haven't seen the patchset for gentoo-sources (if you don't want patches, in gentoo look at vanilla-sources (2.4) or development-sources(2.6))

I was going to paste a selection, but given that it seems to be ~250patches, I will just note that it's a LOT of patches :)

Re:I love XFS

[smcv]

And yes, I've tried to write an ext3 driver for W2K. Now I'm not knocking MS specifically, but it clearly shows that they never thought anybody would like to do somethign this.

If you were knocking MS specifically, depending on your level of conspiracy theory, you might even think it shows the opposite :-)

Re:I love XFS

[Vlad_the_Inhaler]

?? - I thought that there was a product available for windows which allowed you to access ext2 filesystems, although it may not be free. If an ext3 partition was correctly unmounted, it can be treated behave exactly like ext2 - just leave the journal alone.

Re:I love XFS

[red floyd]

The reason we see so few file system drivers for windows is that it's blody hard :(

Not to mention fscking expensive. The regular DDK is free (for NT/2K) or cost of media (for the XP/2K3 DDKs), but the IFS Kit is $1000 US. And not well documented, from what I understand.

If anyone is going to try to write these for Windows, may I recommmend you check out OSR [osr.com]?

Disclaimer: No relation with OSR except as a satisfied client of their driver class.

its about time

[Buckwheatz_tm]

I have been using it for for almost 3 years now. It has never let me down unlike some of the other journaling filesystems. No corrupt superblock like jfs. Kinda slow for small files when the parttions reach around 80% full. Plus its sure to piss SCO off :) but for video editing or files over 100kb it can't be beat. Reiserfs is great for directories like /var since 2.4.18 but I have lost too much data in the past to the bugger.

SCO

[einhverfr]

Plus its sure to piss SCO off :)

That is not the half of it. You see-- Hellwig is a former SCO employee who when he worked there, worked with IBM closely on their port of JFS to Linux. He was also heavily involved in the SMP development process too. Just do a search for his name and SCO and Caldera on your favorite search engine. I think it will be hard for him to avoid a deposition ;-)

Now he works for SGI.

Why so much fuss over JFS?

[Anonymous Coward]

Just wondering, why does everyone get so excited about journaling filesystems? Many distros default to ext3/reiserfs now for even home boxes, but it's like a big band-aid.

If your box is crashing enough to make fscking a chore, you already have bigger problems. Sure, I can see where JFSs are sometimes useful, but on dekstops and most other machines the better-performing ext2 is a much more appropriate choice.

Re: Why so much fuss over JFS?

[Pike65]

I never find fscking a chore!

Quite the opposite in fact . . .

. . . we're talking about different things here, aren't we?

Re:Why so much fuss over JFS?

[larien]

It's not always crashing that's the problem, power failures are another (UPSs can cover that, but...). Also, component failures can often result in a crash.

In any case, when you have 500GB of filesystems, you really don't want to fsck that if a CPU fails.

Re:Why so much fuss over JFS?

[virtual_mps]

Well, one of your mistakes is assuming that the non-journalling fs will be faster. XFS will wipe the floor with ext2 on certain workloads. The other is assuming that it takes a number of crashes to make fscking a problem. A single fsck on a large filesystem could take upwards of an hour.

Re:Why so much fuss over JFS?

[GooberToo]

I remember about 7 years ago, I was working ona project that had a VAX cluster for our Sybase DB. It crashed. He had to wait almost 4 hours for it to finish fscking the disks.

Re:Why so much fuss over JFS?

[Lumpy]

It's based on the value of your data.

if you are running a single IDE drive, then your data is not worth much. SCSI = worth a bit more... SCSI raid 5 with 4 drives and a journaling filesystem = worth alot more.

then we get to the level of hotplug scsi drives with tons of redundancy hardware raid 5 (works awesome with linux) and a real tape backup system (DLT) means your data is actually valuable....

and we can go up from there to extreme redundancy and data safety.

JFS = cheap security for valuable data...

Re:Why so much fuss over JFS?

[Nothinman]

In this case journaling is only one of the main features of the filesystem. XFS is much faster than ext3, supports logging to a seperate device to speed it up even more, nicely supports huge filesystems and file sizes and has much nicer userland tools.

Re:Why so much fuss over JFS?

[drinkypoo]

I agree wholeheartedly. It seems like every other time my power went out I lost data on ext2. I've had many bizarre failures (including some fun kernel panics, and plenty of cord-pulled-out-of-wall debacles) while using XFS and never lost anything. Journaling is a must, EVERYONE should be using journaling.

Want another anecdotal data point about journaling filesystems? Windows NT from 5.x has a Journaling NTFS. (I've heard people say that NTFS has ALWAYS had some form of journaling, and that it wasn't turn

We've just begun

[fserb]

I agree with Marcelo's action on this. We're still in a very early stage of the 2.6 (stable) branch to feature freeze 2.4.

I know we need the maximum user base for 2.6 testing, debugging and to recieve those "My TV stopped working when I installed kernel 2.6" messages. But we have to take it easy.

2.6 rocks. And a lot of distros have plans to release 2.6 based releases in the first quarter of 2004, which will greatly improve the user base.

IMHO, a good feature freeze, as Marcelo said somewhere in LKML, is 2.4.24 or even 2.4.25.

It's no time for a flamewar to begin. The Beaver is in the building. :)

I wonder

[AndyFewt]

.. what SCO will have to say about this...

Patch size.

[rf0]

Fun for the whole family with guess where the patch was applied From the snapshot directory [kernel.org]

bk6 - 424K

bk7 - 964k

bk8 - 1.2M

Well thats increased the kernel by about another 5-10%. However I would say I do like xfs and its proven quite stable now.

Rus

Christoph Hellwig, former SCO (Caldera)!

[infolib]

after a code review by Christoph Hellwig

Incidentally, this is the Christoph Hellwig who contributed code to the kernel on Calderas behalf. [groklaw.net] His contributions may become an important point in the SCO-IBM-RedHat battle.

YEAH, WOO HOOO - ALRIGHT!

[cluge]

After patching every single kernel thats come out since the early 2.4s, I now have a kernel that I don't need to patch. WOW, about darn time!! Perhaps I'll even get lucky enough that RedHat and others that do not support XFS yet will build it into their kernels. That will make MY life easier, and updates go faster.

We chose XFS after lots of serious testing. It beat all comers at the time and we've been using it ever since. The only downside to XFS is file deletion times are a bit long, especially compared to Reiser, but when you have a server that is uner HEAVY load (Databses, mail servers) and with LARGE files (log server) nothing beats XFS.

Thanks guys, this is one of those merges that has made me estatic!

Angry People Rule [angrypeoplerule.com]

Re: FINALLY!

[jbeamon]

I've been patching XFS support into my distro for a few months now. Let me get you caught up briefly. We had ext3 patched with acl support. That patch tends to lag behind kernel versions a little, which is not a problem if you're running a distro's standard back-ported kernel. However, I just undertook a migration on a live production box from ext3+acl to XFS on a second attached disk array, so I've been in Patch Hell for the last few weeks. I will say up front that I am not a k3rn31 h4ck0rz, but I get

Re:YEAH, WOO HOOO - ALRIGHT!

[Malc]

How does it compare with JFS?

Re:YEAH, WOO HOOO - ALRIGHT!

[GooberToo]

-XFS
+JFS

Looks like they are pretty close. I think the only difference is the X and J. ;)

No Complaints

[LightForce3]

Mandrake has offered XFS since at least 9.0, my first Linux distro. I've been using XFS (at the suggestion of my friend who helped with the install) for at least 6 months now, with only instance of a problem (not sure if it was a fault in the filesystem itself): lost or corrupted an inode or two, and fixed very easily once I knew what to do.

It works with both GRUB and LILO, is reasonably speedy, and has enormous partition and file size limits.

Count me a happy customer.

~~LF

Oddly Enough

[thesolo]

This was just mentioned here on /. the other day, but according to this article on Groklaw [groklaw.net], Christoph Hellwig is (was?) a Caldera (SCO) employee.

SCO is going after SGI for XFS [linuxworld.com], when one of their own employees was working on it.

No.

[Booker]

Christoph was not working on XFS while he was employed by Caldera.

Re:Oddly Enough

[Vlad_the_Inhaler]

was.

Groklaw said in the article that he 'also' had 2 e-mail addresses at SGI but not that he has been working for them for several months.

Finally

[LynXmaN]

I've been using XFS in production servers for more than two years already without any problem, it was time that it was merged into 2.4 kernel...

Maybe this way RedHat begins to support it for their installations

Journals are for girls?

[Dynastar454]

from the journals-are-for-girls dept.

Huh? I always thought it was diaries that were for girls... at least, that's what I told my friends when they made fun of my journal. :-)

Bechmarks

[kompiluj]

You can find the benchmarks on:
http://epoxy.mrs.umn.edu/~minerg/fstests/results.h tml [umn.edu], or a copy at: ReiserFS homepage [namesys.com].
Of course your mileage may vary but I generally got results consistent with those cited.
My own experiences (I have used both reiserfs and xfs with 2.4.20 kernel:

• reiserfs is a little bit faster than xfs
• xfs gives you 2 times bigger CPU usage than reiserfs
• both are still much better than jfs
• the reliability of both xfs and reiserfs is satisfactory
• the results are still order of magnitude worse than those I get with UFS2 with softupdates on FreeBSD 5.1

Re:Bechmarks

[Pow.R Toc.H]

These benchmarks have been run with Reiser 4 which, AFAIK, is not shipped by default with no Linux distribution. Even Mandrake guys, who are fond of experimental software on their distributions hasn't included Reiser 4 on Mdk9.2. Most distributions include 3.6.28, IINM.

OTOH, I've been using XFS to store and edit 36-bit film scans (40+ MB file sizes) and XFS has been serving me extremely well, without data corruption of any kind - differently from Reiser 3, which needs a reiserfsck every time I boot Win2k

A few other nice XFS features

[isoga]

...no one has mentioned yet:
(from http://www.sgi.com/software/xfs/overview.html)

Guaranteed Rate I/O
XFS is the only file system available that provides a guaranteed rate I/O system, which allows applications to reserve specific bandwidth to or from the file system. The file system can determine the available bandwidth and guarantee that a requested level of performance is met for a given time. This functionality is critical for media delivery systems such as video-on-demand or data acquisition.

Expanded Dump Capabilities
Unlike traditional file systems, which must be dismounted to guarantee a consistent dump image, you can dump an XFS file system while it is being used. The XFS dump utility, XFSdump, can dump an entire filesystem, a directory tree, or specific files. XFSdump is restartable, which allows a large dump to be spread over an extended period of time or to be resumed after a system restart.

-->tech stuff [homelinux.net]

no GRIO on Linux

[Booker]

GRIO is not available on Linux, because it requires a lot of other support in the kernel proper, in the various I/O subsystems etc.

however, the realtime subvolume, which is a component of GRIO, is available for use on Linux.

Hmm..

[NegativeK]

I happened to stumble on the thread while browsing the lkml a while ago.. I noticed that someone from Fermilab tossed in their support for merging it, as they run a 300TB or so setup. You think, maybe, this has something to do with it?

On a different note, I've been running XFS on my 2.6.0-test box for a while.. Now that it's going to be in a stable kernel, I can't wait to back up everything and switch. =D

NTFS not GPL, FAT not free

[James McP]

SGI released XFS into "the wild" and has ensured its longevity with little to no support on their part and increased the number of "out of box" coders they can hire to work on FS projects.

Microsoft....hasn't. Heck, MS is preparing to charge media makers (CF, SM, MMC, etc) to use FAT.

I say media makers switch to using XFS or another GPL'd journaling file systems. Won't take long for other platforms to support it in bulk (make/ config.....) and for stuff like flash where corruptions can occur often, I'd like a bit of journaling to minimize the impact.

Re:NTFS not GPL, FAT not free

[AllUsernamesAreGone]

The problem with using jouranling filesystems on removable media like CF is that it takes more processor power and space than normal filesystems (for example, resiserfs partitions allocate 30Mb for journal space by default). That journal always takes some space, and the less space you allocate, the less effective the journal is for handling problems.

ext2 on its own though....

Re:NTFS not GPL, FAT not free

[Merlin42]

Actually IMHO journalling on flash would be a bad idea. Most flash memories give you only about 100k write cycles before giving up the ghost. For mp3 players or digicams this is just fine. But, the point of the journal is that it is flushed to disk immediately on a write operation, so depending on usage you could wear out the memory cells that contain the journal file an order of magnitude faster, killing your flash memory REAL FAST.

Re:NTFS not GPL, FAT not free

[Anonymous Coward]

Actually, this is not quite true. Most modern flash file systems are built upon a wear-leveling structure, so that rewrites to a particular sector are remapped uniformly over the remaining freespace. This prevents a single location in the flash from receiving too many rewrites. In practice, this makes the device last virtually forever. (Though knowing the wear-leveling pattern, you could probably force the worst case, in practice, this will not occur.)

Re:NTFS not GPL, FAT not free

[ozzee]

JFFS [qslinux.org] addresses the flash write frequency concern. It would be good if someone was to create a tiny JFFS auto loader that would load off the flash filesystem into windows automagically. That way you can make it seamless.

This is not an endorsement of JFFS, it's just an example of a flash friendly journalling filesystem. (I have not used it - it may be the best filesystem ever, I don't know).

Re:NTFS not GPL, FAT not free

[swillden]

Bad web site. It's almost like they don't want anyone to try out their work. Another example of why developers should stick with developing and get their little nephews to put up their web sites for them.

Right, because a web site without a lot of nice colors and pictures might scare away my Grandma when she's looking for a new file system for the embedded Linux distro she's building. I mean, she really sees value in a file system that is compressed and offers near-optimal wear leveling while minimizing

Re:

[account_deleted]

Comment removed based on user account deletion

Re:but NTFS

[JohnFluxx]

That's certaintly very interesting, but to run it requires a copy of windows NT. Makes it a bit hard for a rescue disk.

Re:I say what....?

[irc.goatse.cx troll]

Faster, More trusted (SGI's been using it for how many years now?), not sure how it compares cpu-wise.
The main thing that keeps me on ext3 is ext2 backwards compatability. You dont have to worry about having custom repair/bootdisks to recognize your install, and its easier to do stuff like mount under windows (great for dual booting)

Re:I say what....?

[Cthefuture]

... not sure how it compares cpu-wise.

In my testing, not good. But it makes a good server filesystem.

XFS seems to steal cycles from deep in the kernel itself therefore making it fairly fast for file operations but at the expense of everything else.

I didn't like it for a desktop system. It uses too much CPU and user responsiveness suffers.

Re:I say what....?

[fishnuts]

Extended ACLs, btree filesystem structures to facilitate huge files, fast sparse files, large directories, fast deletes, and a couple other niceties that would have required huge functional changes to ext2/ext3 to implement. It's also completely 64-bit clean, as it has from its conception.

The btree-based storage structure is already employed by reiserfs in a similar manner, but XFS' implementation has been stable (used in IRIX) for quite a bit longer.

Re:I say what....?

[glwtta]

So what's the big advantage of xfs over ext3?

Performance; scaleability (something around 9 million terabytes, if memory serves); stability - in the sense of a longer proven track-record - while ext3 is quite stable, XFS is simply a lot more mature; features, like ACLs and other small things.

Re:Vendor pressure

[mark_lybarger]

yeah, they probably sent Vito down to visit Marcelo with a suite case full of jacksons and a simple message to deliver. "Marc... look we've got this fine filesystem here. you've seen the code. its nice code. but, you've got our competitor filesystem code in your kernel. now before we get our brawls all in an up roar, all we're asking is you HIGHLY consider putting this here XFS filesystem into that there kernel you're maintinging. Now, be a good boy, Marc, I know you'll do the right thing"


me i always h

Re:Vendor pressure

[fishnuts]

This "big merge" has nothing to do with vendor pressure. The XFS patches have been available and well-tested throughout most of the 2.4 kernel's life cycle and since its (XFS') stability has already been proven to play nicely with the rest of the kernel, it's quite appropriate to do a merge so late in the 2.4 tree's live cycle. The team at SGI that handles merging the XFS code into the kernel have done a very good job of keeping up with bug reports and changes in the kernel vfs code.

Marcelo probably shar

Re:Vendor pressure

[knuffie]

That is not true. The biggest hold back during the past 3 years has been the fact that the VFS layer needed a number of alterations and so far Marcello did not merge XFS because of this.

It wasn't untill Cristoph OK'd the VFS changes that Marcello merged the XFS core.

SGI as a vendor has had nothing to do with it. Buy a altix 3000 and they would happily maintain any special patch you would need for that (IA64) machine.

I think I know what I'm talking about since it's my name on the XFS FAQ. And no I don't w

Hellwig's role in all this

[einhverfr]

Iirc, isn't he a former SCO/Caldera employee who was heavily involved in developing SMP and the Linux port of JFS? Iirc, Groklaw has a thing on this.

Now he works for SGI. The question I have is this-- it seems as if he has a conflict of interest to give his employer a beneficial review of the driver in order to ensure that it is included. I wonder how independent he is in his review.

That being said, he has been an important contributor to the kernel, and so I will give him the benefit of the doubt. I

Re:Wonder...

[The Almighty Dave]

Not a whole lot, really. I haven't used XFS and haven't read the article, so I don't have anything useful to say on the subject.

Re:XFS for Win32?

[TheRaven64]

Your best bet would be FAT32, since that can be mounted rw on both Windows and Linux (it's even possible to boot Linux on a FAT partition, although far from sensible).

The second option would be ext2/3, since there is an installable FS driver that lets Windows NT mount ext2 partitions, although I haven't looked at the progress for a while.

The final option (the one I use) is to put your personal data on a cheap *NIX box (mine's FreeBSD box with a UFS disk) and SMB mount it for remote access under Windows

Re:XFS for Win32?

[drinkypoo]

The final option (the one I use) is to put your personal data on a cheap *NIX box (mine's FreeBSD box with a UFS disk) and SMB mount it for remote access under Windows and NFS or SMB mount it under *NIX.

I guess you missed the article on Using the Real ntfs.sys Driver Under Linux [slashdot.org], eh?

Re:XFS for Win32?

[lpq]

One of the XFS authors said anyone who wants to undertake such a port -- "go for it".

Considering the difficulty in ensuring data integrity and support for B-tree arranged data, Microsoft would not look kindly upon XFS being ported to NT, since their next generation OS is supposed to include database like features to speed up indexing and accessing data like XFS already has built-in. It would really rain on their parade. Also, benchmarking shows NTFS is considerably slower than XFS (or FAT32 for that matter) for large files and NTFS has no support for Real-time I/O partitions or journals being located on separate disks.

NTFS also requires (according to ad-copy) constant defragmentation due to their primitive block allocation scheme while XFS does quite well even without the XFS FSR (File System Reorganizer). XFS's FSR was created for 1 specific customer who had a particular application that generated excessively fragmented disks. Before that, an FSR (/defragmenter) wasn't considered necessary because XFS is intelligent about how it lays out files when they are written and how it stores free space (with free space also stored in ordered B-tree's by powers-of-two size of the free space blocks.

The only benchmark I've seen XFS run noticeably slower on linux, on is deleting large numbers of small files -- something one doesn't notice on IRIX, since the space deallocation happens in background on IRIX, and only the inodes need be marked deleted before the user prompt returned. I seem to remember on Linux the space had to be deallocated synchronously for some reason or another.

Makes sense given the way free space is managed -- when files are deleted, free blocks are recursively combined with adjacent free blocks to create the largest possible 'free block size' (I think up to 128k blocks, default=4k block size) (my numbers may be a bit rusty). Free space blocks were combined asynchronously, under IRIX (as I understand it), in a system thread after the last reference to an inode was released. Linux, if I remember correctly, didn't support the facilities for such a background thread -- thus the block combining happens synchronously, explaining the performance hit for file tests that delete lots of small files: there are many small free blocks that are candidates for being merged with adjacent free space.

I'm not entirely sure why a special "XFS_del" process couldn't be started at system run time who's sole purpose was taking unreferenced inodes and doing the space combining in background, allowing foreground programs to continue asynchronously after simply marking the inode as unusable and enqueing it to the XFS "free space" combining process. It is quite possible some of this has been implemented and my information is dated. But free space combining on cleanup is one of the main reasons why, historically, XFS file systems, didn't need to have _continually_ running programs like Executive Software's, _DiskKeeper_, running, full time in background: because XFS had it's own built-in defragmentation every time a user did a file-delete.

For the degenerate case -- *one* customer was not getting sufficient speed for real-time, uncompressed video recording to disk (back in the early to mid 1990's when disks were much slower). The swat team, assigned to the problem, found that the customer's particular use kept many small files around while deleting some files in a way that prevented automatic space consolidation. This odd usage was just enough to slow down direct-to-disk video recording (something quite difficult on systems in the early to mid 90's when disks were not so fast and SCSI-2 was still state of the art). To solve this problem for *one* customer, the "xfs_fsr" util was written.

To make the most of the efforts spent on the one customer, SGI incorporated xfs_fsr into the general OS to be run occasionally to stave multi-month/year buildup of possible, similar degenerate cases. I.e. XFS customers considered fragmentation such an unlikely / non-issue, that the X

Re:Changes to the stable kernel?

[fishnuts]

Now that the VFS layer has been stabilized and supportive of such FS drivers as XFS, very little needs to be changed to add XFS support. It's almost completely "additive", rather than modifying the existing code.