Ext3cow Versioning File System Released For 2.6

Zachary Peterson writes "Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system as it appeared at any point in time through a natural, time-shifting interface. This is can be very useful for revision control, intrusion detection, preventing data loss, and meeting the requirements of data retention legislation. See the link for kernel patches and details."

Re:So which is it?

[Bob54321]

From the example screenshot [www.ext3cow.com] it appears it is a file system. You take a snapshot of your system at some point in time and it stores this data even when files change. Of course, with any file system it is important to have functionality that allows you to view the files as well...

Hooray for repeating history!

[Anonymous Coward]

VMS had a versioning filesystem 20+ years ago. You'd create a file and any time you'd edit it a number would be appended as the version:

eg foo.txt;1 foo.txt;2

The C in CVS.

[SharpFang]

Concurrent...

Sure you can "go back in time", but two users working on the same file at the same time would be a pain. Networking would require additional layers - even plain SAMBA/NFS, but still. Plus a bunch of userspace utilities as UI to access it easily.

It's not bad as a backend for such a system, just like MySQL is good as a backend for a website, but by itself it's pretty much worthless.

Interesting sponsor for ext3cow

[Anonymous Coward]

The ext3cow project sponsor SecurityEvaluators is a rather interesting company in terms of some of their funding arrangements (sorry, cannot publish details here).
(2006) FBI Head Wants Strong Data Retention Rules [slashdot.org]
(2005) EU Approves Data Retention [slashdot.org]

Re:Overhead?

[JoeD]

Check the "Publications" link. The first one is an article in "ACM Transactions on Storage".

It's a bit dry, but there is an explanation of how it stores the versions, plus some performance benchmarks.

Re:So

[TodMinuit]

It's more like Plan 9's Fossil [bell-labs.com], only without the extremely cool Venti [bell-labs.com].

Re:Overhead?

[DaveCar]

Couldn't read TFA (slashdotted), but I would *imagine* that 'cow' is copy on write and that it just uses new blocks for the changes - so only the differences, but not minimal differences.

some background

[pikine]

I'm answering questions that people posted so far altogether.

Is it a file system or a file manager?

It is a file system. You access old snapshot by appending '@timestamp' to your file name. You have to first instruct ext3cow to take a snapshot first before you can retrieve old copies, otherwise it simply behaves like ext3. It appears that snapshot is always performed on a directory and applies to all inodes (files and subdirectories) under it.

My complaint is its use of '@' to access snapshot. Why not use '?' and make it look like a url query? Better yet, use a special prefix '.snapshot/' like NetApp file servers.

Does it store many copies of each file? or only the differences between the old and the new version?

How far off is it to use these filesystems as a revision control system replacement?

ext3cow takes it's name from "copy on write," and it does this on the block level. When you modify a file, it appears to the file system that you're modifying a block of e.g. 4096 bytes. COW preserves the old block while constructing a new file using the blocks you modified plus the blocks you didn't modify.

You can think about it as block-level version control. However, when you save a file, most programs simply write a whole new file (I'm only aware of mailbox programs that try to append or modify in-place). Block-level copy on write is unlikely to buy you anything in practical use.

Does it provide undelete?

Only when you remember to make a snapshot of your whole directory. An hourly cron-job would do, maybe. There is always the possibility you delete a file before a snapshot is made.

Re:Overhead?

[anilg]

COW has been present for a long time in ZFS [opensolaris.org] since Solaris 10. The overhead there is negligible and its quite stable. Last I heard, it was being ported to FUSE on linux. Upcoming in the next releases of FreeBSD and OSX. Wiki has a pretty good article [wikipedia.org].

Re:True undelete

[xenocide2]

There's a couple reasons for it not being in the kernel. First, it misleads users who expect some degree of data security. The good news is that sort of person likely follows kernel patches to the FS and would likely be aware of the problem, possibly even writing a script that replaces rm with a real-rm.

The second argument is that it's better handled in user space, so the OS doesn't have to make that sort of policy. There's no reason you can't just alias rm to some .Trash, or configure your Desktop Environment to do so (GNOME does, for example). There's all sorts of things you have to decide that might not suit everyone. For example, if I delete a file on a USB drive, does it go in a .Trash storage in the USB drive, or do we copy it over to a main .Trash folder? Many people don't realize they have to empty the trash to reclaim space on their thumbdrive in GNOME.

The final argument I can come up with is security problems. We can't have one global .Trash bin in a multiuser system. And quotas. And permissions.

Reading historic archives of the LKML [iu.edu] suggests it's at least come up once. I guess Torvald's opinion is that anything that CAN go in the userspace SHOULD. Can't explain the webserver in kernel though. Perhaps that opinion has changed some time in the last 10 years?

Re:Can No One Else INNOVATE?

[siride]

Because it wasn't REVEALED until 2006, so even if Apple was working on it in 2002 (not likely, since Open Source projects generally have longer cycles than proprietary ones due to manpower issues), the ext3cow people would not have been aware of it. Why do you think people are stealing this from Apple? It's a good idea that follows logically from ideas found in revision control software such as Subversion and its predecessors. And as others have pointed out, VMS had this 20 years ago. The idea certainly has been in existence for longer than Apple has. The wikipedia article indicates that the TENEX operating system in the 60s first had versioning filesystems. In any case, Apple hardly invented it, Apple was hardly the first to use it, and Linux implementations have been released before Apple even demoed Time Machine. So, basically, you are 100% wrong.

Can't tell, its slashdotted

[tinkertim]

Well, is it the file system, or the file system manager?

I can't tell, the site is experiencing the /. effect.

Mirror of the patch (I grabbed it when I saw this in the firehose) can be grabbed here [echoreply.us] until my server gets sluggish too.

in /usr/src type : patch -p1 linux-2.6.20.3-ext3cow.patch

The site said its not been tested with other kernel versions, but if you feel brave just s/linux-2\.6\.20\.3/your-version/g. Haven't tried it, but should work.

It wen't dark just around the time I was getting the docs and utilities.

Did anyone happen to grab the utilities? Got a link?

Re:So

[samkass]

Apple's Time Machine isn't just a *file* backup system. It's a *record* recovery system. Neither MS Shadow Copies nor this provides an API for software to search records back through time and pull a single record back to the present (ie. a single address book entry or photo). It's frustrating having people equate them so closely when it misses half the point of Time Machine.

Re:Excellent work but...

[cyclop]

If someone writes kernel drivers correctly, those drivers will end in the kernel mainline. Linux supports out of the box more hardware than every other OS, no matter how obsolete and obscure. If you don't have your drivers accepted, AFAIK it's a problem with your code not being of enough good quality, nothing else.

Re:Can No One Else INNOVATE?

[sofla]

Given that Unix has had the concept of file "versioning" since I don't know when (but a long-azz time!)

*scratches head* Unix? Versioning? Never seen it myself. Not to say it isn't there, but over the years I've used several *ix flavors and fs versioning isn't something I've come across. I suppose next you'll tell us Unix has file locking (afaik it doesn't, unless you count advisory locks. I don't).

This is a reverse-engineering of Apple's Time Machine, through and through.

I hate to be one to point this out, but, er... Time Machine is a BACKUP tool. Don't believe me? Go to http://www.apple.com/macosx/leopard/timemachine.ht ml [apple.com] and read the copy yourself, being sure to pay special attention to use of phrases like "the drive you're backing up to". How on earth you could possibly confuse a backup tool with a versioned file system is beyond me.