Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Linux Books Media Software Book Reviews

Understanding the Linux Kernel 96

Posted by timothy from the gee-look-at-all-the-little-black-dots dept.
Reader John Regehr contributed this review of O'Reilly's Understanding the Linux Kernel, which goes into greater depth than most people have ever seen of the kernel source itself. (I wonder what it costs to look at the Windows source.)

Understanding the Linux Kernel
author Author: Bovet, Daniel P. / Cesati, Marco
pages 684
publisher O'Reilly & Associates
rating 8.5
reviewer John Regehr
ISBN 0596000022
summary The guts of the kernel, labeled and explained.

*

Although isolated pieces of operating system internals are usually not difficult to understand, learning how a significant portion of a real OS works is a daunting task: there's a lot of code, some of it is complicated, and some of it operates under obscure assumptions that can be difficult to figure out by reading the sources. Two of the best existing books about OS internals have explained either a simplified but working OS (Tanenbaum's Minix book) or a real, but very small OS (Lions' book on Unix v6). Although these systems have the advantage of being easier to understand, there's an important reason why one might want to study Linux internals instead: Linux is currently relevant, it's likely to be around for a while, and any code you write can potentially be used by thousands of people the day after tomorrow. So, taking it as a given the a book about Linux internals is a good thing, how good is this one? Happily, it's very good - better than any previous such book that I've seen (Rubini's Linux Device Driver book is also excellent, but it has a limited scope).

Understanding the Linux Kernel is good for several reasons. First, the authors have included quite a bit of explanatory material that isn't specifically about Linux - it's the kind of thing one would find in a good undergraduate OS textbook. This helps the reader link explanations of pieces of code to the abstract OS functions that they implement. Second, the authors have chosen a good level of abstraction: core kernel algorithms are explained in text, supplemented with short code sequences (simplified to remove optimizations) for important routines. Flowcharts are used to explain components with complex control flow, and tables and other diagrams are used when appropriate. Finally, the book is well arranged and well written, and there's an auxiliary index at the end that maps symbols mentioned in the book to source code files.

There are a few things I don't like about this book. Most importantly, there is no discussion of the network stack. As the authors say, this is a subject for another book, but by leaving out one of the most interesting and relevant parts of the kernel they are limiting their audience. A second drawback of this book (and of any Linux kernel book) is that since it seems to take about as long to write a good book as it does to write a major version of the Linux kernel, as I write this review it's about to become obsolete - it describes Linux version 2.2. However, at the end of each chapter there's a short note about things that are done differently in version 2.4. This will help preserve the relevance of the book after 2.4 comes out and, maybe more importantly, it gives the reader a sense of what parts of the kernel are under active development and what parts have become mature and stable.

Although Linux is very much in the Unix tradition, many details have changed. For example, early Unix kernels used simple algorithms (such as linear searches) and fixed table sizes. Modern Linux kernels, on the other hand, avoid arbitrary limits on the numbers of many kinds of internal OS objects, do not use linear searches when the number of objects to be searched is potentially large, and use amortized algorithms in many places. In all parts of the kernel, any special knowledge about the way that OS services will be used is exploited in order to improve average-case performance. For example, the slab memory allocator makes use of the fact that kernels often allocate many objects of the same size in order to reduce memory fragmentation and to avoid creating hot spots in the data cache. These algorithmic optimizations are much more pervasive (and much more effective) than micro-optimizations such as tuning register allocation or packing flags into the bits of a memory word - they're what make Linux useful in large-scale server environments where high throughput is critical. However, they also make the kernel code quite a bit more difficult to understand.

Given this complexity, it seems reasonable to ask who needs to read this book and how well does it suit their needs. Three groups of people come to mind. First, potential kernel hackers will find this book to be a good overview of different parts of the kernel. Of course, for people like this a book is no substitute for lots of code reading, but it's a good start. Another potential audience is the group of people who need to understand the kernel in order to extract high performance from it; for example, authors of databases or network servers. This group's needs are well served by this book: the authors often point out why certain heuristics were chosen - this may help people whose applications have run afoul of a resource allocation policy that was designed to serve a different class of applications. Finally, computer science students interested in the internals of a real OS would do well to read this book. It would make a good supplement to a standard OS textbook in an introductory class on operating systems. However, Linux appears to be far too large to understand in its entirety in a single semester: classes that attempt to do this should use a teaching OS like Minix. To benefit from this book, readers should have knowledge equivalent to a couple of semesters of computer science: a basic understanding of programming, of the services an OS provides to user-level programs, and of the hardware mechanisms used by an OS.

This is a good book. The authors have cracked open a large collection of code that's currently very relevant. If they are in for the long haul and release revised books in a timely way, then this will likely become and remain the definitive explanation of Linux internals.

The web site for the book is here.

You can purchase this book at Fatbrain.
This discussion has been archived. No new comments can be posted.

Understanding the Linux Kernel More

Understanding the Linux Kernel

Comments Filter:
  • First, I wasn't really trying for a troll, but I appreciate the sentiment. Second, I'm sorry you got modded down; moderators aren't very clueful when it comes to rating anonymous posts. Or maybe informational content is considered Trolling on Slashdot, these days.

    Third, I thought I was clear on this: I wasn't talking about the kernel. I was talking about the dangers of breaking an implementation by working around the interface, something that can easily happen when the programmer in question knows too much about implementation-specific details. This is a trap that yospelld not standardized. This is an example of something that I would have liked to see standardized, and something that a programmer might end up having to work around.

    I do like your malloc() wrappers; those could potentially be pretty handy. But I can't necessarily expect all memory allocated in a program and passed to a function to use those.

    Oh, and I wouldn't call your moderator "an cueless asshole", just in case they still have some mod points, and happen to be a spelling nazi as well. ;)

    Cheers,

    Peter

    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • Hey, I'm not advocating this approach. You'll never see code like this in a portable program. I'm just explaining the dangers of tinkering under the hood; maybe I wasn't really clear on that.

    This stuff *does* change from one version of libc to the next, and I've *seen* demons get created from code like this! (didn't I say it was unspecified?) That's why I wish there was a function call that did this for me, in the C library. (there's sizeof(), but that doesn't work! :)

    And yes, as many have pointed out, what you can also do is pass a region of memory, and the size that you think it is. That's all fine and dandy until (a) someone lies to you or (b) all you have is a pointer to some unknown amount of memory. It's easy to do, and it will happen to you someday.

    Also, I stopped posting on USENET once it started getting spammed to hell and back, although I'm sure those groups are still moderated. I used to post to comp.lang.pascal.misc, until about when they reorganized the whole thing and created all the stupid delphi groups. But maybe I'll check it all out again sometime, since Slashdot sucks so much. :)
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].

  • Yeah. (Score:1)

    by pb ( 1020 )
    I like realloc() a lot; I wrote a replacement for fgets() that uses it, and allocates memory dynamically.

    But what if you want to know how large a memory structure is, and don't want to truncate it? It can happen.
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • Hey, thanks; that's really cool!

    If I ever need it, I'll try implementing it for different C libraries on different platforms. :)
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • Well, first, this is just an example; I've tested this out before to see if I could do it, and to see how it would be implemented. I also wish this was standardized into the language already, because I can see how useful it would be. (it would make implementing something like vectors in C trivial)

    But yes, the best theoretical use for something like this would be for memory coming from outside your program, or into your function when you do not already have knowledge about it. I know that C can do it, because functions like realloc() make use of this information; I just wish I could as a programmer.
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • Hey, this was just an example; I didn't intend for it to be a *serious* example. The memory I wanted to test was allocated by no one, or possibly malloc(); I just wanted to see how it worked, and hope that I'll never have to use a trick like that, because it is dangerous... which was my point.

    So, of course you can have a pointer handed to you that wasn't directly from malloc(); I'm sure the function in question would have to have that in huge letters on the documentation, somewhat like fgets(). ;)

    I'll have to lurk in trolltalk some more; I haven't posted there in a while. How time flies; I started the one on kuro5hin, and that's already dead...

    And yes, I've obviously been karma capped for way too long; it's at 252 now, from 350 originally... But who cares, right?
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].

  • I think you mean snprintf().


    --
  • I don't agree with this, you can't get documentation like this about Windows. It isn't suited for an end user that would like to know how to change their background colors. There is a pretty limited number of people that would even need a book like this, such as OS students, aspiring kernel hackers, etc. Even a VB/VC++ application developer wouldn't need information of this detail.
  • In fact, all the good OSes make testing malloc() for 0 effectively redundant. You simply have all the memory you want or something is horribly wrong and the machine needs to be reset.

    That would only be true if you had unlimited swap space.
  • there's a better answer out on the net [sinfest.net]....

  • IMHO, Vahalia's "Unix Internals" is not worth 61$. It's dated in the sense that it doesn't really talk about the "modern" PC Unixes (e.g. FreeBSD and Linux) at all. If you want to know how the current system that you're running right now works, this book is just too old.

    If you really want to know about performance tradeoffs, get a more CS-oriented book (e.g. Silberschatz, Tanenbaum, etc), not a Unix-only book from 1996.


    ---
  • Is "Linux Core Kernel Commentary", by Coriolis Press, I believe. Line by line comments...

    As for windows source code, all it'll cost ya to look is an NDA from hell, and possibly your first-born male child...
  • Heck, if you're into the stationary bike, stairmaster, etc, take it with you! O'Reilly's bindings are the best, the book can lay fairly flat with breaking its back. Get one of those copy holder attachments that most of that type of excercise equipment comes with, and let the info soak in while you sweat.

    --
  • If this is the kind of thing your interested in, definately spin for a copy.

    I spun around on my chair until I fell off. When can I expect you to send me a copy?

    --
  • I'd say 'your soul.'

    Eric
  • Hmmm... the Windows source code must be like some ancient and unspeakable Lovecraftian tome if it does that to one's mind. :)
  • All you have to do is
    1. Sell your sould to Sat*coughbillgatescough*an
    2. sacrifice your firstborn to the almighty BSOD
    3. kill a linux programmer and bring his brain to M$ headquarters.

    you will then be allowed to see the windows source. However, they will kill you if you laugh at the fact that windows has a whole lot of this(uhh, i don't know C so here it is in a sort of messed up pseudo language form:

    while foo > 0
    if condition.unknown=true
    then goto BSOD
    next foo

    or something like that
    ----------------------
  • If you must make snide comments about the Windows Kernel source, read "Inside Windows 2000 - 3rd edition". Has plenty of information from people who have seen the source and is probably a lot more useful than the source code would be anyhow as a guide to how and why Win2000 works the way it does.

    Oh, I forgot. You don't care about facts when bashing Windows - this is Slashdot.
  • Yeah that book sucked. The author even had notes to himself, which I thought was ridiculous (check out line 9 of page 3, for an example). I'm saving that flame for another time, though. The information was for the most part accurate in that book, but it is not the ideal source for "linux internals" information. A better source, is from the author's columns for Byte.com from which the book developed. Cheers, Nate
  • I'm looking on fatbrain right now and it looks like there might be a new edition of the book coming out, but fb says that it's not until 2002. I wonder if I should wait or if it's worth the 61$ now (ouch, PTR/PH is spendy).

  • Has anyone besides me ever noticed that O'Rielly books have the paper with the best smell and texture.

    They have a nice texture, I've never noticed the smell though.

    There's nothing like coming home from a nice workout all sweaty and settling down for a nice night of paging through O'Rielly techincal manuals

    If you're all sweaty, you've probably gotten sweat all over the pages which is what is giving them that smell you like.
  • I remember taking my first Introduction to Operating Systems class in College, and there was a lot of reference to Tannenbaums book. The actual OS book we had was good, but vague and at a high level.

    I wonder whether this would be a good "textbook" to teach the practical aspects of an OS, and then relate it to a "standard" OS (rather than the Older Unices, or VAX systems).

    Any comments or thoughts from that aspect?

  • I have this book and I find it too be less technical and much more abstract then I had hoped. If your intent is to make kernel modules et al, buy Oreilly's "Device Drivers" book and not this. If you wish to have a general understanding of the kernel, but be abstracted away from the code buy this.

    One warning, it is very, very dull.

  • "IMHO, Vahalia's "Unix Internals" is not worth 61$. It's dated in the sense that it doesn't really talk about the "modern" PC Unixes (e.g. FreeBSD and Linux) at all."
    That's funny. If you read the Vahalia book and the Linux internals book reviewed here, you would notice that a lot of the stuff that Vahalia talks about (e.g slab allocator, SysV memory management integrating buffer cache and page cache etc), is just now being integrated into the Linux kernel. In fact I even remember seeing some parts of the kernel referring to the Vahalia book as the primary source of information... (just grep through the source for vahalia). In fact I would go so far as to say that for a lot of Linux kernel subsytems, as they exist in version 2.4, Vahalia book is an excellent reference. The two books I mentioned above nicely complement each other.
    Ciao
  • In VC++ you can use the _msize function.

    size_t _msize( void *memblock );

    Not ANSI tho.
  • >(I wonder what it costs to look at the Windows source.)
    No, you just have to live in Seattle. With so many contractors working for the velvet sweatshop you can't walk down the street with out tripping over some former contractor/slave that is holding up a sign that say, "Will trade W2K source for food." ;-)
    But seriously, several years ago we found a deep problem that micro$oft had no interest in fixing with NT, nor could we wait for their next release cycle even if they would, so we had everybody in our little company ask around and sure enough somebody was in the church choir with a micro$oft employee who knew the person who worded on that part of the code. We got the source under the table, fixed the bug, returned the fix and the fix now lives in the NT source tree. Not what I would call open source, but it saved our butts.
  • That's why I wish there was a function call that did this for me, in the C library. (there's sizeof(), but that doesn't work! :)

    Not to be picky, but part of the reason sizeof doesn't do what you want is that it isn't a function--it's an operator.
    /buma
  • I was under the impression that under Linux, if you set the appropiate flag in /proc, Linux will give you a pointer to all the memory you ask for; only when you actually page in all of the memory is there the possibility of a problem. (Or alternatively, turn off the flag in /proc, and then Linux will not over-commit memory resources. The first approach is nice when you're dealing with a lot of programs which may malloc a lot of memory but then never use the memory; the second is nice for situations requiring greater stability; programs can recover from malloc returning NULL, but programs discovering they don't have the memory they were expecting is a bit harder.)

  • This would probably be a great resource for anyone interested in writing drivers, as this would show how the kernel itself looks at things, and probably how other drivers work. This is the kind of information that really needs to get out there if the Linux community is to benefit from users writing their own drivers or even from companies writing drivers for their own products in an environment that they are unfamiliar with.
  • Also i wish they would make a book that was like a "stroll through OS design" cover the differences between OS's, what choices they made, how it effects performance, scaleability, etc..

    The book you want is "Unix Internals" by Uresh Vahalia. It is fantastic. It walks you through the evolution of Unix and talks in detail about the distinguishing features of Solaris 2.x, 4.4 BSD, Digital Unix, SVR 4.2 and Mach among others.

  • Tannenbaums "Operating Systems Design And Implentation" pretty much covers all you need to know for the practical aspects of kernel implementation. It does not cover the aspects of operating systems such as dealing with libc, the POSIX standard etc, but it does show you how to implement a schedular, memory managment aspects, IPC, device drivers etc.

    Yes, it does refer to an "Older" Unix (MINIX, natch), but then that means things can be kept as simple as they need to be. The information in the book should be transferable to almost any operating system though.

    I should imagine that if you combined this book along with OS Design & Implentation, then yes, you would have plenty of relevent information that can be applied to Linux.
  • (I wonder what it costs to look at the Windows source.)
    Only one soul. A small price to pay. :)
    Mark

    RMIT IT Test Lab Engineer
  • Does this make you somehow smarter than everyone else? Smarter than me? Forget it "dumb fuckwits". I randomly take my crack at writing wise-ass replies to people who sound like idiots. I guess I got lucky this time.

    Also, Macs don't suck. The PowerPC architecture is revolutionary and (I think) aesthetically pleasing. It is the MacOS (=9) that sucks.
  • I run it [mandrake tooled to 2.2.18] on a P90 and it loads from BIOS to Gnome in under 30 seconds. Win98 takes 3 minutes, and the hard drive isn't finished thrashing yet. Yessir, progress.

    BTW if you read this book you'd know that Linux doesn't load the "legacy junk" unless you have the hardware in your box. Can you say "pH33r d4 kernel m0dule5?"
  • You will want to know some basic CPU architecture information to get the most of this book

    Can anyone recommend another book that would be a good bridge to this one (Understanding...Kernel) for someone who knows only basic high level programming but is interested in surveying the scene "down there"?

    -Erik
  • is the smell of hypocrisy.

    While Tim O'Reilly goes around pretending to be a champion of open source, their outlandishly priced (and inferior) software (WebBoard - 1799$/$2999, WebSite Professional - $995) is anything but. Until they practice what they preach, I refuse to buy anything from O'Reilly. My copy center copy of this book is really good, though - highly recommended introduction to the kernel. Kudos to the authors!
  • For networking try "Linux IP Stacks Commentary"
  • (I wonder what it costs to look at the Windows source.)
    Just your soul [microsoft.com].

    ;)

    question: is control controlled by its need to control?
    answer: yes
  • Something like this, you mean?

    loop:
    if(bConditionUnknown == TRUE)
    goto BSOD;
    if(foo == 0)
    goto out;
    goto loop;
    out:

    (Sorry 'bout the lack of indentation ... )
    I don't know what the code for boolean is in that LPRSZ stuff Windows API is using, so I made a guess ...
  • Sorry, I was exaggerating a bit. I'm referring to the non-compete agreements they are enforcing rather vigorously. Granted, they expire after a while, but many people forced to wait that long will find another line of work.
  • From what we've been hearing lately, it would seem to be the ability to ever work in the tech industry again.

    Besides, if you're knowledgeable enough to be understanding OS source, you shouldn't be stuck with windows anyway, unless you're one of those poor schmucks who is forced to develop with visual studio.

  • Cost (Score:1)

    by Mike_K ( 138858 )

    (I wonder what it costs to look at the Windows source.)

    Your soul

    m

  • let's say you have a function that gets passed a region of malloc()'ed memory. You want to know exactly how much memory you have to play with. That number is stored somewhere before the beginning of that pointer. malloc() allocates a little extra memory, writes some status info at the beginning, sticks the pointer in front of that, and returns the address to you. However, exactly where and what it writes are somewhat unspecified.

    Um, you really don't ever want to do that. The right (portable) way is to pass both a pointer to the malloc'ed area and its size to your function. What you discribe above can break from one version of libc to the next. Not to mention cause undefined behaivor (according to the ISO standard) and additionly may cause demons to fly out of your nose!

    If this was posted on comp.lang.c you would quickly be flamed until you saw the error of your ways ;^)

  • windows source" ... Just post the review and stop it with the low blows. It's because of shit like this that people stop reading the headlines here.
  • Dude that Sea Otter is HOT... She wants me. Oh yeah.


    ---
  • If you want to know more about the Linux kernel, but are hard up for cash to buy a book on it, check these links out. They might help...

    Concrete Architecture of the Linux Kernel [uwaterloo.ca]
    http://plg.uwaterloo.ca/~itbowman/CS746G/a2/

    The Linux Kernel Hacker's Guide [redhat.com]
    http://khg.redhat.com/HyperNews/get/khg.html

    The Linux Kernel [linuxdoc.org]
    http://www.linuxdoc.org/LDP/tlk/tlk.html

    Analysis of the Extfs Filesystem [polymtl.ca]
    Analysis of the Extfs Filesystem http://step.polymtl.ca/~ldd/ext2fs/ext2fs_toc.html


    ________

  • Has anyone besides me ever noticed that O'Rielly books have the paper with the best smell and texture. It's light, almost trancelucent, but stiff and sturdy. There's nothing like coming home from a nice workout all sweaty and settling down for a nice night of paging through O'Rielly techincal manuals. C'mon people, I am not alone in feeling this way, am I?

  • But if you're looking for a good commentary on the TCP/IP stack, check out "Linux IP Stacks Commentary", published by CoriolisOpen Press and written by Satchell & Clifford. 42k lines of code plus explanation.

    And, *if* you live in Vancouver, BC, check out London Drugs: they're selling copies of this book for ~$20 inc. tax. Just picked up mine...

  • You're telling me that the source code for Windows is more intelligible? Prove it. Come on, cough it up. Let's see the source code.
  • I recently purchased this book (about a month and a half ago) and am delighted in its breadth and clarity. The authors layout the information they wish to present clearly, and every chapter is a refinement of these main areas of functionality. The book is also sprinkled with a lot of code, so that you can see the concepts in action. There are also plenty of diagrams, which gives the book a feel similar to something R. Stevens would write. (like TCP/IP illustrated / Unix Network Programmin) If this is the kind of thing your interested in, definately spin for a copy. It can be a bit hard of a read given its size and density, however worth the effort.

    Seems like a sweet book, I own the BSD devil book, while ocasionally i can decypher something out of it, it ususlly makes my head spin. Does anyone know if this book is someone more geared towards a laymon, maybe a guide rather then a reference? Also i wish they would make a book that was like a "stroll through OS design" cover the differences between OS's, what choices they made, how it effects performance, scaleability, etc.. Linux 2.2 is pretty 0ld sk00l compared to Solaris and FreeBSD.

    Once you start tinkering under the hood, there's no turning back. There are a few things that I'd love to be able to do in C, but the way to do it isn't standard across platforms or compilers. I can figure it out, but I'd have to test it on *everything*, or only support a few platforms where I know enough about the internals. For example: let's say you have a function that gets passed a region of malloc()'ed memory. You want to know exactly how much memory you have to play with. That number is stored somewhere before the beginning of that pointer. malloc() allocates a little extra memory, writes some status info at the beginning, sticks the pointer in front of that, and returns the address to you. However, exactly where and what it writes are somewhat unspecified. I'd love to have a function in C that did this for me, but alas, there isn't one. So what I have to do is seek back through before the pointer, hope everything is allocated, and look for something like status info. I've done this on Solaris and Linux, and they don't do it the same way. :) I think stuff like this is fascinating, but using internal knowledge to write your programs can be dangerous; it must be done carefully to avoid breakage.

    I wonder why there hasn't been a good book explaining at a high-level how the kernel works with the rest of the files in a particular distro. Of course, there would need to be diferent books for each distro, but for those of use who are still learning how it's all wired together it would be illuminating. Even a decent flowchart would be a nice tool.

  • I read "Linux Internals".

    There were clear publishing and printing errors in that book, which made me question the validity of the information therein.

    I forwarded my concerns about this to the publisher, who said nothing.

  • (I wonder what it costs to look at the Windows source.)

    Just your soul [microsoft.com].

    ;)

    Actually, that's still better than being a Windows software developer [fourmilab.ch] ;)

  • I'm am taking a OS course now and the professor made a point in class of telling us the reason why our class won't delve to much into Windows is because they would have to buy rights to the source for every single student in the class. Microsoft is out to make a buck off of everyhting they can. Most likely if they were going over the NT source, you either had to pay extra money to take that class, or they went over OpenVMS which is what the NT kernel is based on. Microsoft is gonna lose in a big way when all these CS majors graduate and know all about UNIX, and don't wanna take jobs with them and relearn everything they know. I know wherever I get a job I am gonna try my hardest to either find a UNIX/Linux office, or try to convert whomever I go to to UNIX/Linux.
  • moz25 writes:
    About the windows source... hhm.. there must be some internal documents explaining things, I'm guessing.
    I think you guessed wrong. In 1997, the Usenix Association had an NT workshop. One of the discussion groups was called "Do You Need Source ?" - you can find a write-up at http://www.usenix.org/publications/library/proceed ings/usenix-nt97/summaries/ntsums.html . One interesting quote from a researcher who had access to NT source:
    Vogels cautioned that NT consists of millions of lines of code and only six pages of documentation, which barely tells you how to get it off the CDs and how to build a kernel from scratch at the top level.
    I seriously doubt the situation has improved, given how much more code the Windows 2000 kernel includes and what we outsiders know about Microsoft's internal corporate culture.
  • I wonder what it costs to look at the Windows source.

    The unofficial cost for doing so is this:

    the right arm of the user in question, severed at the shoulder (heretofore referred to as "an arm"),

    the left leg of the user in question, severed below the hip (heretofore referred to as "a leg"),

    both visual information collection organs (VICOs, heretofore referred to as "both eyes"), extracted by means of suction,

    and one membership to MSDN.

  • After seeing what Larry Wall said about himself (hint, it rhymes with Cupid), I don't think I'll be buying any O'Reilly's books.
  • Your Career... Once you look at the windows source code, you will be ashamed to be a programmer and never be able to write a line of code again. If not, they might offer you a job, because you probably graduated from DeVry or ITT Tech.

    Lets not forget the other costs too:
    First Born
    Sell soul to satan
    Etc...


  • I would be interested in a book about the lifecycle of the Linux kernel source; that is, how bugs are reported and fixed, distribution approaches, discussion forums, fix approval procedures, debugging procedures, etc. Is there such a book? This one appears to be completely technical in its approach.

    Tim

  • Your sig says that we are a compilation of about ~640MB of genetic code.. does that mean that you could fit all of me on an 80M CD-R with a little bit of compression? Hmm.. The possibilities are perplexing.
  • Theres not enough bloat.
  • (I wonder what it costs to look at the Windows source.)

    According to posts on /.

    -Your soul

    -Your reputation as a REAL techie

    -Unspeakable bedroom actions with Bill and friends

  • Do you know what UK organisation did the evaluation of Win2K source? Was it the Defense Evaluation Research Agency (DERA)? (There also known as "dearer".... .... because they are) (b.t.w. "dear" = "expensive") If it was them, I'd throw their evaluations out the window and run away.... Or perhaps it's a secret way of the UK regaining control of the US military ;) Heh heh....
  • It is a wise man who knows what he does not know.
  • If you are looking for a book that compares various OSes, something by Silberschatz or Stallings might be more your speed. I don't really find those authors as helpful in my work, though. Tanenbaum (who influenced Linus' initial design) wrote one called "Modern Operating Systems" which is pretty interesting; it may be the best of that genre. The BSD book and the Linux book are not comparable to Tanenbaum's book (or for that matter, to each other, really).

    I own both books (and Tanenbaum's) and I have to admit that I find the BSD book most useful. I think that's because BSD was more "designed" and Linux is more a "big ball of mud" that just got to be how it is by evolution. The types of bugs I've seen over the years in Linux seem to reflect that, while the kernel and VFS design is quite clever (in an engineering sense), there are aspects pertaining to networking and filesystems (most notably) that only get addressed when they become a problem to someone who is willing to solve them (eg. Direct Server Return in 2.2.14). I'm not saying there is no design to Linux, but it is less pronounced than in *BSD. I do *NOT* think that ESR's Cathedral and Bazaar analogy is as apt as many other people seem to believe. Maybe it's just because BSD is easier to understand for me. And I work with it a lot. (and Linux... and Solaris... and NT/2K...) On the other hand, there is a much greater level of detail in Bovet and Cesati's book, line-by-line analysis of much of the kernel's data structures, which I am finding useful in hacking on it.

    My bias:

    I am an applications programmer, sysadmin, and network administrator, rather than a kernel developer or embedded systems programmer. So I am more interested in network and filesystem details than how certain atomic operations are implemented. I occasionally change little things here and there in the Linux kernel (almost never changing actual code semantics in any of the BSDs), but nothing earth shattering in either case. On the other hand, it is my job to *tune* kernels.

    My comparison:

    The Linux book seems to concentrate on internal data structures, how pieces of code are implemented, and why; more a tinkering than design perspective. The BSD book is a design-motivating-an-implementation book (as per the title). They're both useful books, but for very different reasons. At only $40, it's worth buying the Bovet and Cesati book for guidance if you hack on Linux AT ALL, especially on a laptop or a disconnected workstation (hopefully with all the HOWTOs and source installed on it!).
  • The Win2K kernel (or NT for that matter) is hardly less complex than Linux. If anything, it is a great deal more complex. This book addresses a need which simply isn't present for NT/W2K installations, because 99.9% of them don't have source and can't change anything internal to their OS. Linux is very different, and you don't seem to understand why. This isn't a book for end users.
  • Once you start tinkering under the hood, there's no turning back. There are a few things that I'd love to be able to do in C, but the way to do it isn't standard across platforms or compilers. I can figure it out, but I'd have to test it on *everything*, or only support a few platforms where I know enough about the internals.

    For example: let's say you have a function that gets passed a region of malloc()'ed memory. You want to know exactly how much memory you have to play with. That number is stored somewhere before the beginning of that pointer. malloc() allocates a little extra memory, writes some status info at the beginning, sticks the pointer in front of that, and returns the address to you. However, exactly where and what it writes are somewhat unspecified.

    I'd love to have a function in C that did this for me, but alas, there isn't one. So what I have to do is seek back through before the pointer, hope everything is allocated, and look for something like status info. I've done this on Solaris and Linux, and they don't do it the same way. :)

    I think stuff like this is fascinating, but using internal knowledge to write your programs can be dangerous; it must be done carefully to avoid breakage.
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • Kernel Hacking for Dummies.
  • Actually, I found the kernel to be quite simple. I bought "Linux Core Kernel Commentary", and looking at the code, it was very simple. Other people have also mentioned that the kernel itself has no real impact on user-friendliness. Actually, good user-friendliness requires _much_ more complexity. Let me give you an example:

    If you have a form that asks for a users country, then afterwards asks for their state/province, the user-unfriendly version would simply show the same form to everyone. The user-friendly version would look at the country, and then decide (a) whether the field is needed or not, and then (b) what that field is called in that location. So, the user-friendly version is actually much, much more complex. With simple code, the user does the work, with complex code, the user gets it easy. In Linux 2.4, auto PnP support was added. That increased user-friendliness, and also greatly increased the code complexity (they had to add a whole resource-management subsystem).

  • Can be free, since Microsoft gives the source to some academic institutions.

    Free as in beer, maybe, but the restrictions placed on you if you take such a course can be rather onerous.

    It can be rather difficult to legally find work with a Microsoft competitor, as you're "contaminated" with Microsoft IP.

  • Microsoft programmers are not allowed to read this book!

    It is well known that this book contains fragments of code coverade under the insidious GNU Public License. If any of this code made it into our products it could force us to make the rest of the source for that product available.

    We advise coders wanting to replicate the stability of LInux to look elsewhere. Reading this book can only harm your career at Microsoft!

  • > One or two more options, and we have the next /. poll!

    But is it worth CowboyNeal?

    Okay, I'll just be grabbing my coat . . .

    Geoff

  • Whoa there. When you talk about Linux becoming "less sophisticated and more user friendly" that really has almost nothing to do with the internals of the Linux kernel. Also, to become more user friendly, the kernel will probably continue to become more sophisticated, not less - things like hot swapping PCMCIA cards and USB devices require a very sophisticated kernel to be user friendly.

    But in general, user friendliness is more about the graphical desktop and applications: Gnome, KDE, KOffice, StarOffice, Evolution, etc. There's a lot of differences at that level, so it would be impossible to have a single user reference for the notional "Linux User Interface". But the GUI is not strongly tied to the kernel.

    I think that it should be possible to have good referenece documentation for the kernel, probably as a two volume set - one for the networking stack, and one for everything else. Just because you think the kernel is too complex to understand doesn't mean that everyone else agrees... And if we don't try to document it and understand it, we will end up in a situation where a very small number of people have the technical skills to work on the kernel, and that would be bad for the Linux community. One of the things that keeps Linux on course is that people have the option to fork it - either because they want to experiment with something, or because they have a different problem to solve (RTLinux), or they think they really can do it better than Linus. If the kernel is too hard for anyone but Linus, Alan Cox, and 30 other people in the world to understand, then that option to fork isn't really there anymore, and that would be bad for the future of Linux.

    User documentation, on how to use Linux, is also important but is a completely separate issue and is probably best done by documenting distributions. Like "How To Use Red Hat 7". Of course that book doesn't need to say anything about how the kernel works - users don't care, and they shouldn't have to.

    Torrey Hoffman (Azog)
  • One or two more options, and we have the next /. poll!

    --
  • This book should have gotten a 10. The reviewer's two complaints were:
    1. the network stack isn't covered
    2. it'll be out of date soon
    The first complaint can be answered by pointing out that two other linux books out there, the core kernel commentary [fatbrain.com] and linux internals [fatbrain.com], don't cover the network stack either; in fact there is another book that does just that and that alone [fatbrain.com]. Many topics can't be covered in a book and like the reviewer admits, the authors say networking deserves a book of its own. They have no reason to worry about limiting "their audience". They did such a thorough job with this 700 page book that if they were to include networking - and do the same thorough job with that subject too - the book would bloat to 1400 pages and not get out the door until kernel 3.0.

    The reviewer's second complaint is even dumber. Since the kernel is a construction based in time (rather than something more eternal), descriptions of it are going to be outdated eventually. As will any other linux book out there, when a new version gets out there. Even the New York Times will be outdated tomorrow. Big fucking deal. The authors are tracking the 2.4 changes for the next version of the book - and they plan to keep a website for the book - so everything should be okay. Plus their notes of what's new in 2.4 should be all you need for the time being.

    Sorry to vent, but I hate it when reviewers feel obligated to come up with flaws, just to make it seem like they did a super-penetrating read of the text. If you want to come up with flaws, try reading the book past the introduction. I'm sure you'll find a few legitimate ones at least.

  • If the API is any indication, I can only imagine what the implementation is like. Actually, I was looking at the bootloader code for Win95 the other day. Some guy had dissasembled it, and he kept refering to the code's author as "some crazy mother f*cker." Take a look here. [nondot.org] A couple of the comments are hilarious (as is the code!)
  • I doubt it. Microsoft is obsessive about documentation. The DirectX docs, for example, are around 800 pages just for the API. Anything and everything about that API is in those pages. I have a hard time believing that the same level of documentation doesn't exist for the source itself.
  • Even a 70 minute CD will store you without compression. 80 min is nearly 700 meg.
  • Umm, the kernel code has nothing whatsoever to do with the system code. If you've ever looked at Win32, you'll know that it is nowhere near as simple as POSIX (with one Xception.) The fact that the userspace is convultated is due to crappy design of the userspace, not crappy code in the kernel. (The kernel code is a bit convultated, but most people say it is quite elegant. But IANAKE)
  • > You want to know exactly how much memory you have to play with.

    I'm curious as to why you don't allocate a whole hunk of memory and do your own memory management? Yes, it is a little bit of extra work, but that way you know *exactly* how many bytes are being used. You can do your own tagging - as little as or as much as you went/need. The modern paradigm is a "memory pool" which is quite usefull in some circumstances.

    Of course if the memory is coming from outside your program, then I can see your point. Yes, I wish it was standardised too.

    Data structures will NEVER be standardized across the OS's, but we CAN standardize on the API ! *wishes the proc filesystem was standard on windows, *nix, *bsd, Be, etc.

    --
    "Sex is like math. Divide the legs, add it in, and out comes the result." - Anonymous

  • NDA (Score:2)

    by maraist ( 68387 )
    I wonder what it costs to look at the Windows source.)

    For startes, the inability to ever code another OS again. (NDA's)

    -Michael
  • and I was thouroughly pleased. I agree with Timothy, I was dissapointed that the network stack was ignored... especially when so much time is devoted to various types of memory allocation. Unless they have a book cooking to cover this topic, I feel mystified by the networking code. However, it was very beneficial to see how the VFS works, and the chapter on the Second Extended Filesystem was very insightful and informative. Furthermore, I liked the treatment of the bootstrapping process in the Appendix; this was most helpful in understanding it so you can exact finer boot control. Now I want Writing Device Driver! I give it an 8.
  • Can be free, since Microsoft gives the source to some academic institutions.
    At my school I've seen a notice a lab course, in which the NT sources are examined and extended to support experimental services and algorithms.
  • I'm in process of reading "Understanding Linux Kernel" and have easily made parallels to a book that I read just months ago -- "Inside Windows 2000" by Russinovich. It is quite a revelation to read about Linux kernel design after spending some time acquiring Windows 2000 design. I do really recomend reading both books. You are correct that there is more details on data structures and less on design patterns of the kernel code. Russinovich's book on other hand is suggesting more of general design ideas instead of details of kernel code, he himself has never seen the Windows 2000 kernel sources.

    Peter
  • I wonder why there hasn't been a good book explaining at a high-level how the kernel works with the rest of the files in a particular distro. Of course, there would need to be diferent books for each distro, but for those of use who are still learning how it's all wired together it would be illuminating. Even a decent flowchart would be a nice tool.

    Of course, if there is such a thing and I've overlooked it, please flame me.

  • Cheap shot...
    {Body bgcolor="blue" text="white"} {center} {strong} A fatal exception error has occured in...

  • which goes into greater depth than most people have ever seen of the kernel source itself

    I think I know what you were trying to say (maybe) - but Timothy, do you ever think to proof posts you read?

  • ...and have been incredibly pleased. I haven't done much spelunking in OS code before, and while I've been coding for a quarter century, my experience with UNIX is largely as an application developer. This book provides a solid foundation for understand the millions of lines of code that constitute the Linux kernel.

    I highly recommend Understanding the Linux Kernel.


    --
    Scott Robert Ladd
    Master of Complexity
    Destroyer of Order and Chaos

  • For example: let's say you have a function that gets passed a region of malloc()'ed memory. You want to know exactly how much memory you have to play with.

    Pass the length as another parameter. That's what fread(), sprintf(), etc mandate.

    What you're talking about is home rolled resource tracking. It's not too hard to do it portably, but there are already loads of leak tracking, etc malloc() replacements out there.
  • I hear that there are some new courses on Whistler being offered in Russia now, too...
  • Well, this seems interesting for most people enthusiastic about Linux, etc. Learning more about how things work internally can make it somewhat easier to diagnose/fix problems IMO.

    About the windows source... hhm.. there must be some internal documents explaining things, I'm guessing. Or maybe not...

    Moz.

  • Re:1st read (Score:3)

    by PureFiction ( 10256 ) on Tuesday January 23, 2001 @06:49AM (#488547)
    Does anyone know if this book is someone more geared towards a laymon, maybe a guide rather then a reference?

    This book is definately more of a guide, however, it is detailed enough to serve as a general reference as well. (Detailed reference == see the code ;)

    If you mean TCP/IP illustrated V.2 concerning the BSD stack implementation, then yes, this book might be slightly better from a technical prerequisite standpoint.

    You will want to know some basic CPU architecture information to get the most of this book. I.e. how CPU caches work, virtual/physical address mapping, etc.

  • 1st read (Score:3)

    by jon_c ( 100593 ) on Tuesday January 23, 2001 @06:39AM (#488548) Homepage
    i'm sure i'm not by now, i actually read the review.

    Seems like a sweet book, I own the BSD devil book, while ocasionally i can decypher something out of it, it ususlly makes my head spin. Does anyone know if this book is someone more geared towards a laymon, maybe a guide rather then a reference?

    Also i wish they would make a book that was like a "stroll through OS design" cover the differences between OS's, what choices they made, how it effects performance, scaleability, etc.. Linux 2.2 is pretty 0ld sk00l compared to Solaris and FreeBSD.

    -Jon

    Streamripper [sourceforge.net]

  • Windows source (Score:3)

    by stilwebm ( 129567 ) on Tuesday January 23, 2001 @06:50AM (#488549)
    I wonder what it costs to look at the Windows source.

    It probably costs you your sanity.

  • Writing Linux Device Drivers (Score:4)

    by Jerky McNaughty ( 1391 ) on Tuesday January 23, 2001 @06:41AM (#488550)
    Actually, if you're interested in writing Linux device drivers, I'd highly recommend O'Reilly's other Linux hacking book, Linux Device Drivers (more info here [oreilly.com]). I used this book to write a relatively simple device driver for a device about two years ago. It was incredibly helpful. I assume most of the information is still relevant with today's kernels. This book, combined with the existing drivers for other devices, provided me everything I needed to know.

  • This is a very good book (Score:5)

    by PureFiction ( 10256 ) on Tuesday January 23, 2001 @06:45AM (#488551)
    I recently purchased this book (about a month and a half ago) and am delighted in its breadth and clarity.

    The authors layout the information they wish to present clearly, and every chapter is a refinement of these main areas of functionality.

    The book is also sprinkled with a lot of code, so that you can see the concepts in action. There are also plenty of diagrams, which gives the book a feel similar to something R. Stevens would write. (like TCP/IP illustrated / Unix Network Programmin)

    If this is the kind of thing your interested in, definately spin for a copy. It can be a bit hard of a read given its size and density, however worth the effort.

Slashdot Top Deals

On the eighth day, God created FORTRAN.

Close