Open Source ExFAT File System Reaches 1.0 Status 151
Titus Andronicus writes "fuse-exfat, a GPLv3 implementation of the exFAT file system for Linux, FreeBSD, and OS X, has reached 1.0 status, according to an announcement from Andrew Nayenko, the primary developer. exFAT is a file system designed for sneaker-netting terabyte-scale files and groups of files on flash drives and memory cards between and among Windows, OS X, and consumer electronics devices. It was introduced by Microsoft in late 2006. Will fuse-exfat cut into Microsoft's juicy exFAT licensing revenue? Will Microsoft litigate fuse-exfat's developers and users into patent oblivion? Will there be a DKMS dynamic kernel module version of the software, similar to the ZFS on Linux project? All that remains to be seen. ReadWrite, The H, and Phoronix cover the story."
This doesn't make sense to me (Score:2)
A file system is normally designed for one's own usages. A file system is entirely contained within your computer system (or in the event of a distributed file system, within computers under your control). What use then is "sneaker-netting" files between Windows, OSX, and Linux? Isn't this a network concept?
Re:This doesn't make sense to me (Score:5, Informative)
You do know what a "Sneaker Net" is dont you ? I guess not. It is using media such as SD Card or USB stick or hard drive to move files from one location to another by walking ie on your feet that are wearing sneakers, also very similar to using a "V8" net as in "Never underestimate the bandwidth of a V8 station wagon loaded with tapes / drives hurtling across the country"
Re: (Score:2)
Thanks for the chuckle. I knew what Sneaker Net was, but I'd never heard of a V8 Net.
Re: (Score:3, Insightful)
ahh sneakernet... god I love wifi.
Wifi is famously good when transferring terabyte sized files like exFAT is intended for.
Re: (Score:2)
Wifi is famously good when transferring terabyte sized files like exFAT is intended for.
Actually it compares quite well with the flash media that exFAT is intended for. Most flash drives are pretty slow, but even if you get 20MB/sec write and then 20MB/sec read on the other end once you factor in transport time an 802.11N connection pushing 10MB/sec would still be faster. Plus 1TB flash drives are pretty expensive, where as most people already have wifi.
Re: (Score:2)
I bet (literally, wanna?) we're within ten years.
I sometimes carry around an 8GB one, and it recently occurred to me, "Hey, this isn't really all that much bigger than the one I had in 2004. Huh. That can't be right. Wasn't that a 4GB one? Something doesn't make sense." Then I figured out my mistake. Can you guess what it was? I had the digit right, but not the unit prefix. 9 years ago, my "cool" new flash drive was 4 megabytes. This one (which
Re: (Score:2)
Re: (Score:2)
Re:This doesn't make sense to me (Score:4, Informative)
Ahem. I believe Kingston demo'd one at CES this year, and you can buy a 512G flash drive today. Cheap? No, but I'll put money on being able to purchase a 1TB thumbdrive-style flash drive in 18 months, max.
I spent the last few days re-doing my home backup system. With an equal number of OSX and Linux devices, and no windows devices, the best option for a drive that could go back and forth with minimal custom/flaky driver installs -- but still handle files over 4gb was, of all things, NTFS. I was ... well, frankly, more pissed off about that fact than a normal person should be about disk formats.
Finally (and what I dug into this thread to say) is that Station Wagons have craptastic lag.
Re: (Score:2)
Will they make one with an esata interface?
Re: (Score:2)
Dude, 512 GB flash drives have been announced by Kingston already. Sure, they're expensive, but we'll have them this year. Double that is 2 or 3 years ahead at most.
Intel's got a 600GB drive out already.
Re: (Score:2)
Re: (Score:2)
Already do. Mind you, it was announced 15 days ago.
1TB - http://www.usatoday.com/story/technologylive/2013/01/09/kingston-terabyte-flash-drive-ces/1820159/ [usatoday.com]
Re: (Score:3)
Because NTFS isn't designed for removable media.
Re:This doesn't make sense to me (Score:5, Informative)
Not that we're anywhere close to terabyte flash drives.
You sure about that?
http://www.theverge.com/2013/1/7/3847628/kingston-announces-1tb-flash-drive [theverge.com]
They're not cheap yet, but they're here.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
"Never underestimate the bandwidth of a station wagon full of tapes." It's still faster to drive terabytes of data across town than it is to "network" it unless you have an unusually fast internet connection on both ends and a reliable tube between the locations.
Re: (Score:3)
Re: (Score:2)
Let it work to your advantage. There is no reason the tape can't be read while the writer is still writing.
Re: (Score:2)
It really depends on what you mean by a "good network connection", how much data you have to move and how many items of storage media (not nessacerally tape) you can fill in paralell.
Lets assume
You have 3TB of data
You fill/empty your drive at 1 gigabit per second (number plucked from wikipedia and rounded)
Your network between sites is 100 megabit per second.
It takes 24000 seconds to put the data on the drive and another 24000 seconds to copy it off again at the destination. In comparision it would take 2400
Re: (Score:2)
FAT filesystems are traditionally used on USB flash drives, SD cards and other removable storage to copy files between computers, cameras, printers and other devices which may not be attached to each other on a network. Sneakernet is a term for when you move files via removable storage between computers instead of using a network. For example, if you want to copy several gigabytes of data from one location to another and it would take several hours to complete via the Internet or only take 15 minutes to dri
Not if the network is cost prohibitive (Score:2)
What use then is "sneaker-netting" files between Windows, OSX, and Linux? Isn't this a network concept?
Not if the network is cost prohibitive. Good luck transferring 10-gigabyte files over cellular.
Re: (Score:2)
It would be nice to live in a world where I had a server on a gigabit network connection with unlimited traffic and every machine I touched also had a gigabit network connection with unlimited traffic back to that server.
Back in the real world I don't have that luxury, end user ports on the university network are only 100 megabit and my cable connection at home is even slower so when I need to use large files on multiple computers it's faster to carry an external hard drive arround than to try and use the n
UDF, MTP, NAS (Score:2)
so when I need to use large files on multiple computers it's faster to carry an external hard drive arround than to try and use the network.
And that external hard drive can be formatted UDF, as long as you don't need to write to the drive from a Windows XP machine [superuser.com].
I guess I could carry a laptop arround and setup a network connection directly with the machine I wanted to use the file on but that is a lot bulikier and more hassle than just plugging in an external hard drive.
That or an external hard drive in a NAS enclosure, so that any computer that speaks FTP or SMB can read and write its files. Or an external hard drive in a USB enclosure that speaks MTP, so that any computer that speaks MTP (such as any Windows PC or any Mac with the MTP class driver [google.com]) can read and write its files. The difference is that both NAS and MTP work on the level of files, unl
Re: (Score:2)
Re: (Score:2)
Because you would never need to transfer a file to something you can't hook up to your network, like the USB port on your car stereo, would you?
You would always want to waste days transferring a terabyte or two across the Internet, instead of 30 minutes to drive it across town, right?
Never underestimate the bandwidth of an SUV filled with hard drives hurtling down the freeway. The latency sucks, though.
Re: (Score:2)
ExFAT is a 64-bit filesystem.
Thumb drives are not the only thing that can be used with ExFAT.
2.5" laptop drives can be powered from USB, fit into a bubble-wrap envelope nicely, and scale to terabyte levels.
NTFS is not useable read / write without a whole lot of fucking around on any OS not named Windows, where ExFAT may be built in (it is on Mac OS X 10.6.something and above.)
ExFAT can be licensed by embedded electronic manufacturers that don't need the full weight of something like NTFS (Blu-ray manufactur
Re: (Score:2)
Back in my day, we used punch cards.
exFAT is already on OS X (Score:5, Informative)
As far as I know it's part of OS X since Snow Leopard. But I could totally use the Linux support.
They won't sue yet. (Score:3)
First they'll give enough time for it to get established to the point of being considered an essential for any functional desktop.
*Then* they'll start suing.
Re:They won't sue yet. (Score:5, Informative)
Based on their previous actions, they will allow the use of this project in distros but will sue any commercial implementation that uses it. So they haven't sued Ubuntu or Mint, but have sued TomTom.
Re: (Score:2)
*Then* they'll start suing.
Suing who?
Re: (Score:2)
Obviously anyone who makes money off their shitty file system.
Re: (Score:2)
Re: (Score:3)
Me too, because I don't plan on touching any FAT file systems with a ten foot pole if I can help it. It's bad enough portable systems like cameras and phones often require it; I get my data off of those systems ASAP and on a more sane file system, first chance I get. I have already tried externally formatting my Android phone's SD card as ext2 with no success... it would be nice if ext2 and UFS were supported by these things.
Android... the Linux that can't even support its own native file system.
If SD causes you to stumble, cut it off. (Score:2)
You want to support SD cards larger than 32 GB?
No, not at least until you can prove it's absolutely necessary. Why can't a future device design just drop support for SD cards and use UDF or Ext formatted USB flash drives instead? As a widely respected first-century teacher might have put it: "If [SD support] causes you to stumble, cut it off."--Mark 9:43 [pineight.com].
Re: (Score:2)
Size. MicroSD is really the only option on phones and some other portable devices. Many tablets now are so thin there isn't even room for a USB port.
Re: (Score:2)
MicroSD is really the only option on phones and some other portable devices.
Most of these portable devices have a receptacle that supports acting as a USB host (USB OTG Micro-AB, iPod dock, or Lightning), and many have already dropped the microSD slot. Connect your USB storage device to the device through an appropriate adapter cable.
Many tablets now are so thin there isn't even room for a USB port.
Most of these are also too thin for microSD, such as every iTrinket since the first iPhone. If they don't have a USB port, then how do they charge?
DKMS? (Score:4, Insightful)
As the name clearly states, this is a FUSE implementation of exFAT, i.e. userspace. In which case DKMS is as useful as a fork for soup.
So not only we get the news two days after Phoronix [1], but the poster has no idea on what he's talking about.
[1] http://www.phoronix.com/scan.php?page=news_item&px=MTI3OTQ [phoronix.com]
Re: (Score:2)
Unless the copyright owners re-license it.
Re: (Score:2)
no need to litigate (Score:2)
The fact that this isn't in the kernel and that device manufacturers can't ship it remains a serious problem for Linux.
Re: (Score:2)
"The only way to win is not to play". If you use MS filesystem they're winning. Big operators like Google/Samsung/Sony (okay, maybe not sony) should be able to agree around some free and gratis FS for devices, and then use the courts if necessary to make sure it's easily available on Windows (I hear they have a store now).
What's wrong with ext2/3/4?
Re: (Score:2, Insightful)
One word: Windows
Re: (Score:2)
One word: Windows
ext is open, you can implement it on Windows (in fact it's already been done).
Re: (Score:2)
Though there are several solutions, none of them are ideal. When you're running Windows 7 x64, it gets even more hairy. Here's one discussion thread about the problem and options. [techpowerup.com] So, I'd be unwilling to call it "already been done." The problem is that the Windows file system "API" is less than functional and a pain in the ass under the best of circumstances. From what I have read, the universal "best solution" is to boot from a Linux (any) LiveCD if you want to work with ext# partitions.
Re: (Score:2)
Re: (Score:2)
Explore2fs supports read-only and does not have Windows Vista/7/8 support. My statement still stands.
I can write code but, I'm not a programmer so any attempt I make to work on it will just make things worse.
Re: (Score:2)
Explore2fs supports read-only and does not have Windows Vista/7/8 support.
Since you won't bother to read your own link I'll quote it for you here:
Works great for me in Windows 7 x64. Provides decent transfer speed and apparently supports both read and write ability. Ignores permissions so you can get access to every file on the partition.
In any case it's not that it can't be done, it's that nobody wants it, people just write utilities that are just enough for what they need, if you cobble the functionality together you have a workable solution. It's all open so either learn to c
Re: (Score:2)
Sorry, I went one link deeper and didn't specify. I linked to the site I did as it had more options if people wanted to investigate.
According to the Explore2fs official website, the current version only supports read-only and only up to Windows XP. There is a beta of version 2 (v0.7) that has "Supported by all versions of Windows (Vista is still Work In Progress)" listed. I'm not saying you can't make it work or that it won't work in many scenarios. But, it's not ready for prime time in a production
Chicken and egg (Score:2)
ext is open, you can implement it on Windows (in fact it's already been done).
So if I have a USB flash drive formatted in ext, how do I load the ext driver onto an Internet-disconnected PC running Windows so that I can use the flash drive with that computer?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I've read that if a drive is "removable", Windows refuses to read partitions past the first.
So flip the removable bit. Or use FAT32 instead.
Flipping the removable bit (Score:2)
So flip the removable bit.
I thought removability was part of the USB device controller, not something stored in the file system, and there was no standard way to "flip the removable bit" that works across brands of flash drive. This answer confirms my suspicion [superuser.com]. Or were you recommending that people investigate which USB flash drive brands support end-user control of the removable bit before buying the drive in the first place?
Or use FAT32 instead.
Windows won't format FAT32 bigger than 32 GB, and surpassing the 32 GB limit is SDXC's reason for existence.
Re: (Score:2)
Or were you recommending that people investigate which USB flash drive brands support end-user control of the removable bit before buying the drive in the first place?
If you're in the ridiculously bizarre and quite frankly obsolete situation you're describing above, yes.
Windows won't format FAT32 bigger than 32 GB
So use the DOS format utility...christ do you need to be hand-held through everything, google isn't that difficult to use if you don't already know these things.
Re: (Score:2)
If you're in the ridiculously bizarre and quite frankly obsolete situation you're describing above
I don't see how it's obsolete to end up using a computer that has no easy access to the Internet, or a computer on which one is not permitted to install software that requires administrative privileges. Or do you mean USB mass storage itself is obsolete, that people should be carrying devices that implement MTP instead of mass storage and storing data on those?
christ do you need to be hand-held through everything
I don't, but the median end user does, and mass-produced consumer products are made for the median end user, not Slashdot users.
google isn't that difficult to use if you don't already know these things.
Google is difficult t
Re: (Score:2)
I don't see how it's obsolete to end up using a computer that has no easy access to the Internet, or a computer on which one is not permitted to install software that requires administrative privileges.
And has no networking, and you don't want to use FAT32 and you're using ext and ext drivers aren't installed and an ext-supporting userland application isn't installed and you don't have a usb stick formatted with a compatible filesystem with which to use such a utility...etc...etc...yes obsolete.
I don't, but the median end user does
The median end user isn't transferring between non-exFAT supporting systems like Linux and non-internet connected Windows machines that they need to print/scan from either. If you're in that bizarre situation (I ca
Re: (Score:2)
I've partitioned removable HDDs before (attached via USB), and Windows (Vista and up, at least; never tested XP) can access all the partitions. It's possible on flashdrives as well, though they might not all automatically mount... I'll have to test some more to see what happens there.
Re: (Score:2)
Re: (Score:2)
Sadly, you'd have to adjust the thermostat in hell first.
Re: (Score:2)
The courts do not have that power. Maybe - maybe - in a full antitrust case, but the last time MS was involved in one of those... well, United States v Microsoft was filed in 1998, and concluded in 2002. Four years, and it only finished because MS settled it on terms quite favorable to themselves. Legislative action could do it, but good luck out-lobbying microsoft, not to mention all the economic conservatives screaming about how the commies are trying to steal the hard work of a good American company.
Re: (Score:2)
Re: (Score:2)
Those are not supported by Microsoft Windows right out of the box so that are not readily suitable for use in flash drives and SD cards.
A lot of devices are not supported by Microsoft Windows right out of the box, that's hardly causing problems, that's just an excuse.
Re: (Score:2)
Re: (Score:2)
You mean the 4GB file size limit of FAT32 never causes problems?
No, i mean A lot of devices are not supported by Microsoft Windows right out of the box, that's hardly causing problems [slashdot.org]. You really think people only use things that are supported by Windows out of the box?
Re: (Score:2)
A lot of devices are not supported by Microsoft Windows right out of the box, that's hardly causing problems, that's just an excuse.
This, unlike the case you're thinking of, does cause problems. People normally use a USB flash drive to copy the driver for one of these "devices [that] are not supported by Microsoft Windows right out of the box" from the computer with the fast Internet connection to the computer in another building with the printer, scanner, or other similar peripheral. But when the flash drive itself is the peripheral, it creates a chicken-and-egg situation [fashionablygeek.com]. Or do you believe that anybody who doesn't carry two flash driv
Re: (Score:2)
Re: (Score:3, Interesting)
Ext2/3/4 sucks as an interchange format. In short, it does too much. Any filesystem sufficiently complex to support real workloads is going to impose an excessive implementation burden for sneakernet. The bizarre thing is that we have a minimalist filesystem that can represent the file model with fidelity (large files, unicode names, etc) that is implemented in every modern OS: UDF. If it can read DVDs, it can read UDF and every general purpose OS released in the last decade can write to the appropriate ve
Re: (Score:2)
Ext2/3/4 sucks as an interchange format. In short, it does too much. Any filesystem sufficiently complex to support real workloads is going to impose an excessive implementation burden for sneakernet.
Format disk, use as normal. Hardly an excessive burden. But yes UDF works just as easily.
Re: (Score:3)
It's annoying using a file system with file ownership on a flash drive, because chances are the computer I plug the flash drive into has an entirely different set of user IDs that don't match up to the flash drive's files' ownerships. I wish there was an easy way to mount an ext filesystem with all of the files owned by a specific user id (such as the id of the active desktop user when I plug in the flash drive). I wouldn't be surprised if there already is a way, but it should be do-able via the UI and not
Re: (Score:2)
+1
if you plug a drive into another computer you should have no expectation that the computer will respect the permissions, so what is the point of having them. it should be possible to make a udev rule that 777s any external disk when you mount it. or hack ext4 to ignore permissions on external drives.
Re: (Score:2)
Crappy xttrs support (4kbs if enabled)
Re:For once it's true. (Score:5, Interesting)
Microsoft has already won by having ExtFAT part of the SDXC spec, so every big SD card comes with it. The only thing the Open Source world can do is damage control by implementing it and thus staying useful.
Re: (Score:2)
Microsoft will never allow one of its modern file system to have an "official" implementation on free operating systems.
What do you mean by 'official'? You mean from Microsoft? If so yes they probably won't create a Linux implementation, but that's cool because there's this project, there's Paragon and there's Tuxera if you really want exFAT on Linux, just like NTFS. Alternatively you could use the ext file systems.
There will be the threat of litigation or looming incompatibility and data loss through lack of public documentation.
We've had the NTFS driver for a long time.
Instead we should develop a simple and robust filesystem that's suitable for embedded systems and have it standardized.
Go for it, ext3/4 is probably a good start.
Re: (Score:3)
NTFS on linux was created through many years of hard work reverse engineering the filesystem from no documentation - what little MS had published was only available under licenses that would render it useless for open-source development. That it works at all is impressive, that it works so well is a small miracle. Even now, NTFS support in linux has to be via the NTFS-3G userspace filesystem - full support was never included in the kernel itsself, only read-only access. That may well be the future of linux
Re: (Score:2)
The kernel has ( or had, I haven't built one in a while ) experimental write support since at least the 2.6.18 branch or prior... I don't remember exactly when it was introduced. It is completely useless[1], but it is there.
That said NTFS-3G really is the way to go. It also, as an added bonus, fits the UNIX philosophy: "Do one thing and do it right".
[1] The in-kernel write support for NTFS only allows you to write to an existing file, and only allows you to write the same amount of data as the exact files
Re: (Score:3)
fuse-exfat is also an userspace driver like ntfs-3g. If US-based distros like Fedora ar able to ship with ntfs-3g installed by default, they might be able to do the same with fuse-exfat, unless Microsoft closed the legal loophole used for ntfs-3g.
Re: (Score:2)
NTFS support in linux has to be via the NTFS-3G userspace filesystem - full support was never included in the kernel itsself, only read-only access.
It doesn't have to be, Linux maintainers don't want patent-encumbered specs implemented in the kernel, that's their choice, distro vendors may feel differently.
NTFS on ReactOS? (Score:2)
Re: (Score:2)
FAT is in the kernel (presumably reverse engineered). That is supposedly covered by patents. It should not really be any more dangerous to put ntfs, exfat or zfs in the kernel.
(ZFS is slightly odd, in that there is already code that could easily be dropped in if it were not for licensing. It would need some to reimplement it under a GPL licence, but i guess while there is the faint possibility of the original code being relicensed no one will want to put in the work)
Re: (Score:3)
Jelly Bean's Play Store introduces DRM and that's incompatible with GPLv3.
I wasn't aware that the APK encryption in JellyBean was mandatory.
Re: (Score:3, Funny)
Obligatory XKCD [xkcd.com]
(Seriously though, patents on file system are bullshit.)
Re: (Score:2)
Re:The wrong way around (Score:5, Insightful)
Standardise all you want. You should know what'll happen. Windows will not support it out the box, and if Windows doesn't support it, that filesystem is effectively dead. Who is going to want a USB stick formatted so it won't work on the operating system running on upwards of ninety percent of desktops and laptops?
Re: (Score:2)
Put the driver for the standardized/interop filesystem in a (comparitive small 1%) fat16 partition* on the same media. Include nice installers and use the autorun features of the OS. Problem solved.
*: I had an older 512Gb USB drive that included the drivers for retarded windows versions on a seperate device, it emulated a cd with iso9960 of UDF.
Re: (Score:2, Insightful)
Re: (Score:2)
That sort of thing worked in the win9x era when everyone had autorun enabled and there was little lockdown of systems. Nowadays it's going to mean your device can't be used with all the locked down desktops found in corporations and educational institutions.
Re:The wrong way around (Score:4, Insightful)
You can call FAT and its variants a lot of things, but "modern" isn't one of them.
Re: (Score:2)
Re: (Score:2)
It works on Windows out of the box
Write support only works out of the box on Vista and later. Not a huge problem to work around, but it's there
UDF certainly looks like the most appropriate candidate for a truly universal file system, though
Does patent beat copyright? (Score:5, Interesting)
Am I allowed to use this implementation?
Depends on what you want to use for: as a form of expression, you should be able to. Use the binary form to read/write, all depends on the MS patents and whether or not MS grants you a license.
Will Microsoft litigate fuse-exfat's developers and users into patent oblivion?
Regarding developers: the software is posted as source [google.com] code with instructions on how to [google.com] install them from source. Being source code, is a form of expression, protected by copyright. As such, can a commercial entity try to block the dissemination of the "speech" that the source code constitutes?
Mind you, any existing patents should not play any role into it: after all a patent is a public disclosure of methods/constructs that constitute the invention (the text of the patent is not copyrighted), so the source code should not be anything but an alternative form of expression of the same.
Regarding users: yes, using the compiled binaries would violate the temporary monopoly granted by any existing patents. However, I can't imagine any corporations starting to track which hobbyist home users:
1. downloaded the source code - should not be, per se, illegal - the copyleft license allows you to do it and the patent should not trump the copyright.
2. for each of them, ask for a discovery to see if that source code has been compiled - again, compilation should not be illegal, I'm obtaining a derivative form of expression and the GPL copyright license allows me to do it
3. use the binary - this is the only step that would violate the patent
Re: (Score:2)
So by your logic, that copyright trumps patents and the use of the system is the violation I can sell/import anything I want that is covered by a patent. It's not me who is violating anything. Its the individuals who purchase and use it.
Oh wait, that's completely wrong.
I can however, do what ever I want with anything covered by a software patent in a country that doesn't enforce them. As long I don't get Kim DotCom'd by the FBI.
Re: (Score:2)
So by your logic, that copyright trumps patents and the use of the system is the violation I can sell/import anything I want that is covered by a patent.
Not anything, but anything that is a form of expression.
Chances are you are too young to remember/know about "encryption as a weapon" [wikipedia.org] brouhaha approx 18-22 years ago (or too old and already forgot about it).
Anyway, as a memory refresh: the Ninth Circuit Court of Appeals ruled that software source code was speech protected by the First Amendment and that the government's regulations preventing its publication were unconstitutional [wikipedia.org]
Re: (Score:2)
So if I happen to write machine code myself, that machine code is protected free speech too.
Re: (Score:2)
So if I happen to write machine code myself, that machine code is protected free speech too.
As long as it's source code, yes. Shouldn't matter if the source code is binary.
If you think it's crazy, don't blame me: IP validly stands for "Internet Protocol" and "Imaginary property".
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Yes... because this has never happend before *rolls eyes*
Was this as a result a law or a court decision?