Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Linux Software

Reiserfs Released 107

Posted by justin++ from the btree'd-fs dept.
Kewlname writes "Namesys released Reiserfs for GNU/Linux. The specs and the press release " To think - we get this and beta journaling code in the same week (I hope!). Anyway, I don't know how applicable it is to Reiserfs, but Theodore Ts'o has a paper in the 1998 Linux Expo proceedings about how one might go about introducing B-trees into ext2. It might be interesting to compare this to the design of Reiserfs.
This discussion has been archived. No new comments can be posted.

Reiserfs Released More

Reiserfs Released

Comments Filter:

  • Re:What to make out of this? (Score:1)

    by Anonymous Coward
    Stephen Tweedie spoke at length about the new filesystems
    at the UK Linux Dev conference last weekend.

    The filesystems coming out now are pretty well tested
    and it is unlikely that they will cause filesystem
    corruption. There will certainly be bugs to be ironed out
    but these code releases come AFTER an enormous amount
    of debugging.

    It is likely that a fully stable filesystem will be ready within 6 months.

  • Xia Filesytem (Score:1)

    by Anonymous Coward
    The Xia filesystem was named after its author. Before ext2, there was Minix FS and Ext FS. Both had severe limitations. Frank Xia, a grad student in maths at Columbia, wrote the Xia filesystem for Linux which was much faster and had bigger limits. It was the hep filesystem until ext2 stablized. Many Linux old timers have fond memories of Xia filesystem. But ironically one of the first controversies in the Linux world resulted from Frank Xia initially calling his file system the Linux FS. That created a hullaballo until he threw up his hands and then decided to name it after himself.

    Frank Xia remains an unsung hero of Linux developement. His filesystem overcame the size and field limitations of the Minix FS, and it also overcame the severe speed limitations of the Ext filesystem. He deserves a place in the Linux Hall of Fame. He bought time for Linux while a more sophisticated Ext2 filesystem was under development. Linux luminaries such as H.J. Lu swore by the nimble Xia system, and others might have abandonded Linux altogether if they would have had to use the problematic alternatives.

  • no. (Score:1)

    by Anonymous Coward
    The files being served in the MindCraft benchmark are significantly larger than block size IIRC. Reiserfs shows its most dramatic performance advantage on sub-block-size files.

  • Re:It's simple. (Score:1)

    by Anonymous Coward
    damm it.....could you people please read an intro book to IP. Simple question, what would the point of a license be if the copyright holder automatically kept all changes??


    A license is specific allowance to do something, it extends the rights of the licensee. If a license gives the ability to alter someones code then this can be done. Those changes are under the copyright of the write, not the original. If the license specifically said any changes you make fall within my rights that would be different - this is the case with the NPL.

    You cannot extend copyright in the way you are suggesting. You *can* do what you are suggeting wiht a license. If you want to comment on this very *important* area for Free Software (and you should want to) please, please, please read an intro book on IP/licenses

  • Concurrency control (Score:1)

    by Anonymous Coward
    Obviously, you need some kind of concurrency control. This
    can take the form of:

    1. Lock the whole system during your transaction. Safe but
    inefficient!
    2. Lock each BTREE node as you use it. Got to watch out for
    "deadly embraces", where two processes are waiting for what
    the other has locked.
    3. Optimistic concurrency control, something I read about
    at least 15 years ago. You "check out" nodes as you do
    the transaction, then "verify" when you're done. The
    "verify" fails if another process modified one of the nodes
    during your transaction, in which case you start over. I
    don't know if this has ever been proven useful.

  • B-Tree definitely not a binary tree. (Score:2)

    by Anonymous Coward
    Not even close. It's a balanced tree that tries to minimize the block reads to find a specified code, specifically designed for filesystems. The balancing is accomplished by splitting and joining nodes to maintain the average number of children per node in certain limits. One node at a time is usually read from the disk and then the next is chosen on the basis of the that node, so having multiple keys in one node is faster than just 1 key per node as in a binary tree.

    like this (increasing order to the right, cant get less than to work...)

    k1 k2 k3 k4 .. kN

    There's a child node between every key (and before the first and last keys) and the value of every key in that child node (suppose the node is between keys K1 and K2) is between K1 and K2. In a binary tree we can cut out 1/2 of the unsearched keys per node, in a B-tree we can cut out N/(N+1) keys. The rationale is that searching a node with multiple keys linearly is a lot faster than reading another node from disk.

    In addition this fs uses a B*-tree which has some additional constraints (I can't remember exactly, was it that every node has to be 75% full at all times)...

  • Re:Stupid name? (Score:2)

    by Anonymous Coward
    With so many responses to your post, I assumed someone was bound to point out that he explains his reason for naming it after himself. I stand corrected(everyone missed it or didn't even bother read the brief).

    First paragraph from acknowledgments: "Hans Reiser was the project initiator, primary architect, supplier of funding, and one of the programmers. Some folks at times remark that naming the filesystem Reiserfs was egotistic. It was so named after a potential investor hired all of my employees away from me, then tried to negotiate better terms for his possible investment, and suggested that he could arrange for 100 researchers to swear in Russian Court that I had had nothing to do with this project. That business partnership did not work out.
    "
  • Posted by Justin:

    I personally could care less...
  • BTW - if you had gone to Mr.Reiser's web page and actually read it you would have seen that he addressed this at the bottom of the paper.

    Dear Mr AC (no, not you Natedog!):
    It's a long paper, so just skip all that boring file system stuff, and go here [devlinux.com].

    --

  • yeah....of course XFS is not an option. HOw could I expect to use a filesystem based entirely on the X Font Server?!
    ;-)

    (I'M KIDDING....don't flame me)

    On a serious note....should we rename the 'xfs' that runs in the cron on my Red Hat box? I think newbies are going to get very confused when there are two completedly unrelated LInux terms "xfs" in their manuals.

  • Yeah...I realized that after I hit submit. Oh well.
  • >introducing B-trees into ext2

    Geez, I can barely remember where everything is
    now. cd /usr/local/bin is a whole lot easier than
    cd /right/right/left/right/left/right/right/right.

    ;)
  • Not really as the whole working set was only 75 MB and thus fitted completely in the cache. For an idea what happens when the working set is larger see the C't benchmarks (it could be though that RAID5 is extremely bad for NT but that's a pretty usual setup).
  • Actually he named it after his father.
  • 1) You can dsirtibute your _own_ code under any number of licenses you want. So you can distribute it under both the GPL and under other licenses for use in proprietary code.

    2) Namesys wants money from distributers of proprietary Unixen for whom the GPL is unacceptable, I very much doubt they also expect money from distributers of free operating systems.
  • The main author has to get permission from everyone who submits patches in order to use them in the proprietary versions. Or he can rewrite them himself. neither are particularily hard. Peter L. Deutch has been doing this for years with Ghostscript.

  • but you got it wrong! its up up down down left right left right b a b a select start (omit the select on some games)

    ahh nintendo...my ONLY memory of childhood
  • One *could* say the same about Linux, mind you..
  • That's odd, I've been using it for over a month! At the moment I only use it for fairly small loopback filesystems (it's still awesomely fast). Once I get a better feeling for how stable and corruption-resistant it is, I'd like to use it for all my filesystems. /usr/doc (on Debian systems) would rock using reiserfs (it brings out the worst in ext2). I'd say the transition from ext2 -> reiserfs is almost as awesome as the transition from FAT16 -> ext2. Especially in terms of space efficiency.
  • Read linux-kernel mailing list, you'll find a big discussion on the subject.

    Last time it was discussed (that is, just some months ago), the answer was : in (almost ?) all cases where someone could wish for user-defined attributes, they can be emulated with directories, and directories have the advantage of not breaking existing applications.
    That's another reason to use reiserfs btw.

  • I don't think either belong to the filesystem layer, what's wrong with encryption (loop) layer and raid ?
  • Ok, so if I count in the "ext2 with b-trees" as one future option for a file system, I now have XFS, Reiserfs and b-ext2 as an option.

    Could someone clearly give a nonbiased look at what the good and bad points of these are?

    I know I'm using a machine with a b-tree based file system right now and occasionally when it crashes, things can get very hairy. I'd take redundancy over speed any day, so I guess I'm for XFS?
  • B-tree - as in Balanced Tree [ic.ac.uk] (although some people also say B-tree for Binary Tree [ic.ac.uk]).

  • The code may be good, but the PR sucks. Even to a frenchman like myself, the language used in the press release sounds clumsy and amateurish.
  • useless post..
  • Does it have compression? Compression built into the filesystem is a cool thing. What other features does it have?
  • ext2 with b-trees has been promised for some time, and when it arrives it will not be using modern tree technology. XFS for Linux is a press release, and they will be making it an XFS lite, and they aren't going to do the port themselves, they are going to let some university do the port for them, and they have legal issues with SVR4, and it just goes on. They say it is coming out soon, but they need changes in the buffer management code, and....

    It is vaporware, which is not to say it won't ever be real, but.... we are shipping now, and we aren't using obsolete forms of B-trees. When they ship, we will be even farther ahead by then....

    All that said, XFS is a very nice, respectable, mature technology.
  • You can't really use ext2 for this application, it becomes unusable for 10,000 file directories.

    I am afraid you have little alternative to us for this particular application.
  • We do quite well at the create, delete, write, operations that dbench has a lot of.

    We hope to do a lot more extensive benchmarking, but at the moment it seems that once they fix the networking code enough that the FS is the bottleneck, we will be able to help.

  • B*Trees are really cool because instead of splitting a page in half when it overflows, it gives some nodes to it's neighboring sibling. If the neighbor is also full, then the two pages split into three. When it underflows, it's combined with it's two siblings and the three become two... thus the pages are ALWAYS at least 2/3rds full!

    This concept could of course be extended to do a 3-for-4 page split and merge, but the routines become painfully complex at this point...

  • Actually, Andrew is named after both of CMU's founders, Andrew Carnegie and Andrew Mellon. (CMU = Carnegie-Mellon University)
  • I think the general model will be to use ext2 by default for a while, but ultimately reiserfs looks very interesting for most 'desktop' style usage of Linux (including servers for desktop systems), e.g. mail, news, reasonable size databases, etc.)

    XFS looks more appropriate for large databases with huge disk sizes and file sizes (and stringent time-to-recover requirements), as well as for huge multimedia files (which SGI of course is traditionally focused on).

    I don't think a merger of reiserfs and XFS is practical or desirable. What might be very useful is a tool that takes performance logs (e.g. sar(1) output on disk I/O) and scans your filesystems, then recommends which files should be on which filesystems.

    One more blue-sky idea is based on the observation that you have to move the file to a different part of the filesystem tree to change it from (say) ext2 to reiserfs. It would be very handy if some user-level daemons could automagically migrate files to different filesystems based on sizes and usage patterns - just as hierarchical storage systems move files from disk to tape etc, this would change the underlying implementation without the user having to worry about this.

    However, this would require you to separate the filesystem namespace entirely from the actual underlying filesystem, on a per file basis perhaps, which introduces all sorts of complications. On a much coarser level of granularity, this is somewhat analogous to having a logical volume implemented on top of various physical disks. But this is probably a feature for Linux 5.1 :)
  • cd /usr/local/bin is a whole lot easier than cd /right/right/left/right/left/right/right/right.

    It dramatically decreases different technical details, doesn't it? Must be more user friendly then. Like mice with less buttons and parameters hidden behind multitudes of dialogs.

    :P

    [dicklamer: i like macs too]
  • Quoth Namesys' technical introduction [devlinux.com]:

    This project is GPL'd, but I sell exceptions to the GPL to commercial OS vendors and file server vendors.

    Seems like you'd have to derive your own version of GPL in order to not break the original by yourself - unless it already makes this possible. Well, I guess I should read it myself before making comments like this :P

    An interesting situation will arise if reiserfs or major parts of it will be applied in Linus' kernel releases or optionally patched into distributions. Namesys' effort to fund itself with its GPL'd code might fail, if their product gets provided implicitly. Practically and hopefully this might be solved by RedHat et al striking a deal with Namesys. However it's a stretch of the spirit of GPL to assume a reward although the code is free. This kind of self-funding free development should be supported, but is it really? Or am I so mislead that I should just bury my nose to the dear old source of GPL?

    Anyone shed some light on this fundability vs. GPL issue?
  • My god, am I really already too lazy to read even my own writings?

    On a side note, I found Namesys' technical introduction well written.

  • 1 - Well, bringing community changes from under GPL to the proprietary version sounds still like breaking the rules :P

    2 - This sounds very reasonable. So they expect 'real' unices as paying customers. In a world somewhat fair, I can figure them getting funds from several big-time distributors, opensores or not.
  • Answer to an implicit question:

    Distributions can include the GPL version. No deals are needed at all. I have no idea where you got that idea from, since GPL is spelled all over it.

    Question was not about deals or no deals, but where to make money from. That problem arises when they try to make a (/some) living out of their fs. But actually it's just that 'ages old' opensores business model: A good piece of code will be supported by those who find it useful. Even with cash, where that's appropriate.

    Misunderstanding is natural considered the hazy nature of my wonderings above.

    Code forking between the different licenses might become a problem. Then again, problems are just something to solve.

  • I have about 1 million files, averaging about 5k apeice, on an IDE drive and being served by a web server. Some directories hold tens of thousands of files. The files are served from a low traffic web server; well under a gig per day. Is reiserfs better than ext2 for me?
  • Where might I potentially notice an improvement? I'd hate to go through the hassle of switching to a new filesystem just for a faster 'rm -rf'
  • I found the technical introduction to be extremely well written. Of course, most people would probably prefer to just read the press release and not have to wade through lengthy technical details.
  • :)
  • Could you recommend such a book?
  • Is AFS (Andrew File System) named after its creator? Personally I would name any of my creations after what I call my penis: Papa Smurf.
  • like many before him, he could hire a marketing firm, pay them to pick a slick and buzzword complient name, and then charge for it.

    In OSS, you may not get as much $$ as those that charge for their work, but one of the main motivations is exposure. Just as Linux was named after Linus giving him credit for what he started, let Mr.Reiser have credit for what he has done. sheesh!

    BTW - if you had gone to Mr.Reiser's web page and actually read it you would have seen that he addressed this at the bottom of the paper.

  • Is Mr. Reiser on some sort of ego-trip so that he named the fs after himself?

    Not without precedent -- look into some older kernels for xiafs. IIRC, named after its creator, Francis(?) Xia.

    Besides, it's not like anybody ever refused to submit filesystem patches to old Bob Ext or his son (Bob Ext II)

  • Build a better mousetrap, Release it under two licenses: One open but non-profit, one closed but for profit. Just like qt does. You can make alot of friends, and you can afford to pay for lunch too.

    Only problem is that you depending on the legal system to defend your interests, which could be an expensive and difficult task.

    Anyone got any ideas about how to solve this?
  • There is nothing wrong with this. The copyright holder can release the code under multiple licenses separately... they can release it under the GPL (which is then good for everyone) and they can also release it under a different license to a specific party. What this means is that the specific party is free to make changes that they do not need to release under the GPL. However, only the copyright holder can do this.

    There are other packages that are released under multiple licenses. Ghostscript is one... in fact, ghostscript is released under a more restrictive license first, and under the GPL later (with a delay of about a year, I think).

  • Yes. This is one of the things reiserfs is good at. Give it a try.

  • Users should not have any problem. One is called xfs and the other is called XFS. Its mainly windows and mac users who are (case) insensitive.
  • It's turtles all the way down, man.
  • My understanding is that the Linux name was *not* picked by Linus himself... I forget what name he had originally for it. One of the ftp maintainers of the system early on suggested Linux, and it stuck.

    Anybody else can confirm this?
    ----------

  • >> introducing B-trees into ext2
    > cd /right/right/left/right/left/right/right/right.

    You are confusing binary trees with B-trees (which are quite different from each other). It was funny, though...

    Lars FJ

  • Cato wrote:
    XFS looks more appropriate for large databases with huge disk sizes and file sizes (and stringent time-to-recover requirements), as well as for huge multimedia files (which SGI of course is traditionally focused on).

    Disclaimer: I work for SGI. SGI created XFS.

    I've been running XFS on my IRIX desktop since 1995. There is one fairly good reason to use XFS on the desktop, which I haven't seen mentioned: you never ever have to fsck.

    K<bob>
  • Papa Smurf aside, I always figured that AFS was developed at CMU, and named after Andrew Carnegie.

    Just a guess, though.
  • I doubt it.. but would using this fs
    have any dramatic impact on the 'MindShaft'
    Benchmarking results?

    No.. I"m not quite THAT hip to exactly
    what this would accomplish.
    I am curious though.
  • No, it's much WORSE with B-trees than binary trees. "Damnit, did I remember to split the /usr/src node on the downward traversal last time? Or should I be splitting when I back up out of the recursion? I don't even know any more. . ."
  • Ok, so we've all noted that peformance is awesome for 100 byte files. But check out his section on larger files too. The benchmarks of files all greater than 512k are also quite impressive:
    Look at the rm -rf * comparison--
    Ext2: 0:02.63
    Reiserfs: 0:00.16
    Ratio:16
  • Ok, so we've all noted that peformance is awesome for 100 byte files. But check out his section on larger files too. The benchmarks of files all greater than 512k are also quite impressive:
    Look at the rm -rf * comparison--
    Ext2: 0:02.63
    Reiserfs: 0:00.16
    Ratio:16
    Wow, now a new user logged in as root can wipe out his/her system 16 times faster than ever!
    Seriously, though, a .16 second rm -rf * time for 80+ megs in a series of subdirectories is pretty damn impressive.
    --JZ
  • Check out the results for the 80/20 banded file set. A key result is that Reiserfs performs MUCH faster when large numbers of files are grouped in one directory. Copying all the files from one directory under Reiserfs was 12 times faster than the same operation under ext2. Scanning all files (if located in only one directory) was improved by a factor of 6. These results drop to ratios of 2 or 3 to 1 if the files are in many subdirectories. This uses a mixed set of both large an small files (about 50 megs' worth), but emphasizes the very smallest files (12,000 of the files are under 100 bytes). Those really small files provide reiserfs with the big edge.
    However, its results are still quite good for medium-large files and the author admits that he's done virtually no optimizing for larger files. I suspect that, if Reiserfs catches on, we'll quickly see these optimizations in the next few months. Thus, I'd wait a little while and see how things pan out before putting it into use.
  • I don't know about combining the results but I think this will be a good thing for fitting Linux into the various markets where it get's used. Some filesystems will be better at somethings than others.
  • Does it address the other ext2 limitations? In particular, does it address the lack of metadata for new features such as:

    1. access control lists
    2. capabilities (keep them out of ELF -- yuck)
    3. compression
    4. undeletability
  • > Linus himself changed his original
    > pronounciation to a more popular one!

    No, he just got an american accent. The "leen-oox" finnish pronounciation that Linus gives on that sound file is different in accent only. That pronouciation uses the finnish short "i", just as "lynn-uhcks" uses the american short "i".
  • Reiserfs is made by Namesys therefore they are free to release it under whatever license they wan't. They choose to release under two very different licenses - this is their right.

    BTW perl is under two licenses also, the GPL and the rather less restrictive Artistic License. The GPL is somewhat viral in nature (which is the point of the GPL - so no surprise there), but that is not acceptable to everybody.

  • "Freax" it was, I believe.... I prefer Linux, myself.

  • c't/heise Linux/NT benchmarks [heise.de]

    It's good. Read it!

    Summary:
    - NT is faster in one unrealistic test.
    - Linux is much faster in one somewhat unrealistic test (CGI on NT).
    - Linux is much faster in a more realistic test (many files).
    - Linux is slightly faster in all other tests.

    I still wonder why the Linux guys accepted the unrealistic test for the PC Week benchmark ("Mindcraft 3") and didn't insist on additional realistic benchmarks.

    Did I miss this one on the Slashdot front page?
    Or weren't they posted?
    If they weren't, please someone kick Rob...

  • Well, for most existing applications you'll probably be better off in ext2 (its simpler and less likely to be corrupted, IMO). I think the interesting thing about this file system will be when it is used by people who do a lot of work in "glue" languages like perl and python get their hands on it. It provides the kind of fast access of a dbm database with the flexible hiearchical structure of a unix file system, and data can be directly manipulated by normal utilities like wc, awk, sed, etc. Combine something like this with Python's "pickle" and you have a poor man's object database.

    I'd really like to see transaction logging on a file system like this, but its an interesting step.
  • Naturally, you wouldn't use something like this for multimedia streams, but I think the developer's point is that for a lot of applications *should* use lots of little files. Programmers often clump the data together into large files to avoid performance issues or to provide indexed retrieval (e.g. dbm).

    I think there is a lot of application for something like this in applications that track lots of text files (document management, source code control, etc). Just think of how much simpler a lot of your tasks become. For one thing you get all the great UNIX file and text utilities like "find" and "grep" prebuilt and ready to use with your data. For security, you get creation and modification information, and can specify permissions on individual objects within a database like rows or fields, as opposed to entire collections like database tables.

    I think a filesystem like this could become very popular in web applications, both for efficiency and because of ease of programming. I could easily see something like slashdot implemented on it.
  • cd/up/down/left/right/b/a/select/start
  • Let's say Sun decides to include reiserfs as part of Solaris. They will most likely not want to GPL Solaris, so they'll pay to get reiserfs under a different license.

    And of course people or organizations might pay for support, or to have Reiser and his team implement certain features.

    I don't know how easy or difficult it will be to make a living out of reiserfs.
  • Actually, I think it's more like Kaffe. Qt is released under two different licenses, but it's just one source tree. Kaffe has a free GPL'd version, and a commercial version, and it's not legally possible to keep both trees the same. (Unless you only accept patches from people willing to transfer the copyright.)

    At the moment there's just one reiserfs. If people start using it, this will change. The community only has to deal with the GPL'd version, but Reiser must maintain both and save his legal butt.

  • It's simple. (Score:3)

    by Trojan ( 37530 ) on Wednesday June 30, 1999 @02:43AM (#1825670)
    If you write a piece of code, you (or your boss) owns the copyright. If you own the copyright you can release it under whatever license you like. For example to the community under a GPL license, and to paying customers under a different license.

    If reiserfs gets included in the kernel, and the kernel developers modify it, these modifications are applied to the GPL'd release, so fall under the GPL again. Reiser won't be able to include these patches in his commercial version (unless he explicitly asks for and gets permission).

    Distributions can include the GPL version. No deals are needed at all. I have no idea where you got that idea from, since GPL is spelled all over it.

    Anyway, just to reiterate, since you're not the only one who doesn't understand (I'm thinking of a certain BSD advocate who recently tried to crush the GPL using very false arguments): if you release your code under the GPL, the GPL only applies to that release. You are still the owner of your code, and you can do with it whatever you want. (But of course you can't retract the release itself.)
  • From the press release:

    Reiserfs is GPL'd, with exceptions to the GPL available for sale.

    I didn't quite get that bit. Is Namesys selling derivatives of GPL'd software with a commercial license? Isn't this supposed to be the big no-no in the GPL? I checked the website linked to in the press release but couldn't find any other mention of this. Does anyone know any details?
  • Exactly! The first time I saw it, I read it as rye-serfs. I had no idea it was an FS until I read more about it. In fact, that's why I kept reading--to find out what a Reiserf was.

    -awc
  • I do, when I have finished using (playing) with a program and its source, I do remove the source from my system. If you never use your system or don't clean up after yourself, than no, it wont be useful for you, but for me, anything to improve dismal unlink times is a godsend. BTW if anyone thinks ext2 is slow unlinking, try deltreeing a large directory without a disk cache loaded!

    Also, some important programs need to do lots of unlinks occasionly. Take squid (web server cache) for example. So don't crap on about something you know nothing about.

    Hopefully, unpacking tar files will be faster too, helping me when i download programs too.
  • Some of the admited architectural weak points worry me, especially with large files. I'm worried Resierfs is a niche filesystem, great for systems comprised of small files, but not really industrial strength. However, I'm not acquainted with some of the finer points and numerous "common" optimizations he refers to, so I could be wrong.

    Reiserfs and XFS both being released so close together might divide the rather small pool of Linux filesystem hackers who are qualified to merge all these ideas into something workable. Plus, I haven't grasped either to the extent that I can tell if they're able to be merged.

    To some up, I don't know what I'm talking about, but I'm nervous nonetheless.

Slashdot Top Deals

"What man has done, man can aspire to do." -- Jerry Pournelle, about space flight

Close