Cox on Torvalds and Linux Kernel Development

sebFlyte writes "Alan Cox' speech at FOSDEM sounds like it was interesting... according to this ZDNet report on it he has some interesting views. For one, he says: 'Linus is a good developer, but is a terrible engineer.' He also has a few digs at Torvald's methods surrounding security fixes, and some other interesting insights in the kernel development process: 'Sometimes you see a fix and think "this is perfect, move my fix into the kernel tree." Later you think, "I must have been drunk. Don't apply that patch."'"

Also for your perusal

[by Anonymous Coward]

An Interview with Linus himself [linuxmagazine.com]

Re:Also for your perusal

[TrekCycling (468080)]

That's interesting. Thanks for the link!

The most interesting part to me is that he uses a Mac running Linux now. For a long time the story was that he ran SuSE (I'm assuming on x86). I'm a SuSE user, and some of the enthusiasts in SuSE-land used to use this as proof of why SuSE was "the best". I just laughed. I wonder what distro he runs now.

Re:Also for your perusal

[Linker3000 (626634)]

I wonder what distro he runs now.

Hurd?

Re:Also for your perusal

[antime (739998)]

Heh, wonder why they cut out part of Linus' post quoted at the beginning? (The original post read "just a hobby, won't be big and professional like gnu".)

Obligatory Peanuts reference...

[tktk (540564)]

Linus probably keeps all the secret fixes under his security blanket.

Interesting take on Linus

[Gopal.V (532678)]

"Linus has this bad habit of fixing security holes quietly," said Cox. "This is a bad idea as some people read all the kernel patches to find the security holes."

I wouldn't advertise my mistakes either ... neither do the OpenBSD folks or any ego driven engineer :)

The article paints Linus as the typical Flawed Hero of contemporary literature. He's good and yet he's not perfect - at least that's what comes out of it for me. (and no digs on BitKeeper .. hmmm..)

Linus' Security Practice

[alanlke (685520)]

Cox may have had a good point on Linus' methods for security patches, but fortunately the community has spawned sites such as this http://www.securityfocus.com/ [securityfocus.com] to publicly announce when people find security flaws from poking through the patch code.

Even if Linus tries to keep these things secret, they'll get out quite quickly.

ZD states..

[Creepy Crawler (680178)]

That Linus is a person, and not a GOD as some people worship him as.

Its actually pretty damned nice to see a bunch of people get together and make something as big as the Linux Kernel. Linus started it, but we all will finish it.

Still, I fail to see how some bugs would be super-bad, as the article seems to say. Id rather have a crash bug, rather than a SUID change bug.. STill, not all security comes from the Kernel. Some security comes from network filter drivers, some com from the application, which many hackers target, and whatnot. Though, the kernel is a great place to attack if you have that guest acct and "want" root ;P

Re:ZD states..

[Stanistani (808333)]

Calling Linus an engineer is like calling Gandhi a politician... you need to look up just a little to judge how badly you underestimated the impact of the man and his followers.
Gandhi had his moments of pettiness and just plain tom-foolishness, but the sum of his efforts changed the way people gain power back from those who would usurp it for their own.
In a different, yet no less trivial way, so did Linus (although I would not call him Mahatma).

nothing new

[sewagemaster (466124)]

the /. headline makes it looks like there's quite a bit of fued between cox and torvalds, which isnt really the case if you RTFM.

different people have different working styles, no matter whether it's kernel coding, software apps, or ASIC designs. if either group/individuals are too giving to the other group, there can never be enough feedback/ constructive critisisms between them. having yes-men surrounding you isnt the best thing. and it's not like that they're arguing so much they've halted any soft of development progress.

[offtopic]
gives me an idea though, maybe when job interviewers start asking me those behavioural questions about "a time when you've had disagreements and a way of resolving them", there's no need to bring up something too dramatic.
[/offtopic]

Who cares about Linus anymore?

[mnmn (145599)]

Linux has become much bigger than Linus now. The kernel alone has its parts maintained by other people, many of whose patches are applied without much checking to the main tree because they're 'responsible' for it, like certain architectures, driver trees etc.

Apart from the name, Linus currently has the final say of what goes in. Thats just officially. In real life it seems far more is delegated to others for different parts of the kernel, and Linus is one of the developers, far from the most active, and not really exercising his right to block patches against the majority's will.

Re:Who cares about Linus anymore?

[bonch (38532)]

"Who cares about Linus anymore?"

Slashdot, apparently, since it usually posts an article on just about every mention of Linus, every minor activity, every little comment, and every little speculation. People glorify him as the guy who single-handedly wrote Linux, when Linux is really the work of thousands of developers who just send patches to him (and now other delegates too).

Re:Who cares about Linus anymore?

[nysus (162232)]

I'm interested in Linus for purely selfish reasons: I can learn from him. I learn from him even though I've barely looked at any Linux source code or do any programming in C or do any kind of low-level programming for that matter. But as someone who plays a small role on a small open source development project, it's fascinating for me to hear from a person who leads such a hugely successful project and the problems and obstacles he and other developers have to overcome to be successful. It helps me put perspective on my own work.

That's not glorification, that's taking advantage of somebody else's insights and experience.

Sign of a Strong Working Team

[kmactane (18359)]

I'm sure everyone who doesn't bother to RTFA will now think, "Oh, no, Linus and Alan are bitching each other out in public." That's nothing like what's going on here. For one, the submitter quotes only half of one particular line from the article:

"Linus is a good developer, but is a terrible engineer," said Cox. "I'm sure he would agree with that." [emphasis added]

So it sounds like Alan and Linus have discussed this particular difference in their talents before, either over beers at a pub, or over email or something.

Second, the article makes clear that part of what's going on is that Alan and Linus each have very different responsibilities in keeping Linux going, and so they necessarily focus on different things. Alan points out that as the dev tree maintainer, Linus is trying to keep the code maintainable, while Alan's trying to keep it stable.

And both of these things are necessary. It sounds to me like rather than being "at loggerheads", or "ready to call off the working relationship", instead Linus and Alan are a very well-matched and complementary team, both of whom contribute enormously to Linux's success and quality.

Each of them has strengths that make up for the other one's weaknesses, and it sounds like they have a good enough working relationship to give each other constructive criticism when needed.

Re:Odd

[Soko (17987)]

I'm giving up mod points to try and dispell what seems to me a sensationalised headline.

Alan Cox is not showing disrespect to Linus here, read the whole quote:

"Linus is a good developer, but is a terrible engineer," said Cox. "I'm sure he would agree with that."

Alan and Linus have been working together for a very long time, so I'm sure Linus wouldn't give this statement a second thought. Each must know they compliment the other and make the whole of the Linux kernel better - even if they have the odd disagreement - or the kernel would have been truly forked (no pun intended) a long time ago. As it is, they work together on patches and ideas. We don't have much to worry about, I think, since Linus' sense of who's a good dev and who fits into his team well is uncanny.

Soko

Re:Odd

[LnxAddct (679316)]

Good points. But for those who didn't rtfa, or have no idea about kernel development, Alan Cox generally makes sure the kernel is stable. Linus likes to innovate and throw new ideas in without necessarily testing them thoroughly. Innovation and stability are usually separated by quite a time period in any development process, so all Alan was more or less saying was that (as the parent stated) they compliment each other. OSDL pays Linus to hack up new stuff thats needed, Red Hat pays Alan to make sure that new stuff is stable and can be effectively used to its full potential. You really do need them both, as one without the other won't achieve much, and giving those tasks to one man alone would be quite a burden and errors would be abundant. Both Alan and Linus are absolute geniuses at what they do and no one is arguing that. Since when did OSS need to sensationalize headlines?
Regards,
Steve

Re:Odd

[Abcd1234 (188840)]

Wow, someone hasn't grown up. Those who don't understand their own limitations, or get pissy when they're discussed/joked about in public, are nothing more than egotists and/or immature morons. Then again, I'm not surprised you wouldn't understand this, as the concept of humility is lost on a very large number of people in this world...

Re:Odd

[servognome (738846)]

Anyone want to guess which branch would be more popular?
For the geeks... Linus, for the companies with money... IBM
The money trail will probably end up deciding the winner.

Think selection, not direction.

[AJWM (19027)]

Linus isn't running the show. He's not paying anybody, he can't fire anybody, he can't make anybody drop one project or idea to work on another.

He can direct some developers to do something and they can tell him to take a hike, or they can do it because they think it's a good idea.

More often, though, there are just many ideas (patches, development threads, what have you) to choose from and Linus "rules" by choosing which goes into his kernel.

The cathedral is about direction. That isn't what Linus does -- he just selects what is best from what the bazaar has produced.

(Sure, he may also make suggestions and remarks that indicate what his selection criteria are, and that may in turn influence kernel developers, but that doesn't prevent someone from coming up with an even better idea that Linus hadn't considered before and changing Linus's mind. That doesn't happen in a cathedral -- do you think some workmen with a brilliant but different idea for St. Paul's would have been paid attention to by Christopher Wren?)

Christopher Wren and the Cathedral/Bazaar

[kahei (466208)]

Actually, the design of St Paul's and the surrounding area was the result of a complex interplay between Wren, the king, the church and the people:

Wren -- wanted to create baroque-style city center
King -- wanted to save money
Church -- gothic all the way, baybee
People -- wanted convenience and dense business development

In the end, the group that came closest to getting what they wanted was the people; that's why London has no great big boulevards like those of Paris (the people valued lots of living space above ease of riot control). Wren, the man with the 'brilliant but different idea' used the King's negotiating weight to slip change after change past the Church, but he couldn't change the design completely which is why it's a hybrid gothic/baroque.

Thus the design evolved by consensus, negotiation, and balancing bright ideas against established needs. Incidentally, I never used to like St. Paul's, but now that they have cleaned it all up and redeveloped the area north of it with some excellent postmodern work, it's looking really good. Weather still bad though.

Going back to the subject of software, I really don't think this whole Cathedral/Bazaar analogy was well chosen in the first place (especially as gothic cathedrals were striking examples of community efforts, as has been pointed out elsewhere). Major projects and projects that depend on a central authority can be developed in just as fluid a way as small, distributed ones -- if you have the right people. If you don't have people like Wren and the King, if you don't have people who change their minds when they find a brilliant idea, there can be problems.

Re:With all due respect to Alan Cox,

[bloo9298 (258454)]

That was quite funny, and I see that you managed to fool somebody into giving you an "Insightful" mod.

Mods take note, the parent post is deliberately nonsensical. For example, "It's been shown that creating working kernel based on a register machine like most modern microprocessors is NP hard".

Re:Linus is authoritarian?

[updog (608318)]

but Linus dictates main-line completely.

Uh, not really - Andrew Morton is the official maintainer of the 2.6 kernel, and Marcelo Tosatti is the maintainer for 2.4.

Re:Linus is a hacker

[happyslayer (750738)]

I was going to mod, but decided to jump in, instead... I don't know Linus, I'm not a kernel-demigod, and you may know a lot more about him than I do. And while I'm a linux-enthusiast (and therefore an admirer of all the work that goes into it), I'm not a groupie who automatically jumps up to defend the Order of the Penguin. With that said, I don't see how "contemporary ideas" have anything to do with his ability to manage and guide the development of an OS. I've read correspondence about kernel issues (as I've come across them), and it always seems to me that he tries to keep it simple and direct. "Does it work?" and "Will it screw things up later?" appear to be the underlying themes...very admirable ones, in my opinion. Even more to the point: Why should anyone care if he has little or know knowledge outside his project? (And it appears to me that he has a lot of experience...but I can't/won't try to rattle off his resume. See above.) If I have to have brain surgery, I don't give a damn whether or not my surgeon knows how to do an appendectomy; he's got one job to do, and that's all I care about. Well-rounded educations, backgrounds, etc. are great when your project has to cover a wide range of issues. (Ever get involved in a government software project? It's a nightmare!) But if your needs are specific, then the more of an expert you are in that one area, the better off you'll be. To me, he's a smart guy doing a pretty good job of herding cats. 'Nuff said.

Re:Question - balls in the air?

[Abcd1234 (188840)]

The question I have is, is it still possible for these kernel gurus/hackers to effectively have the kernel and all its nuances inside their head, fully functional at a theoretical/experimental level? Or does development at this point consist of sub groups that are specialized and don't require a level of understanding to 'run the kernel in your head'?

in short, it's the latter. However, keep in mind that, in a well designed system that's properly modularized, with neatly spec'd interfaces between components, it isn't always necessary for someone to have the entire picture, with all the nitty-gritty details, in their head. Instead, one need only grasp how the system operates at a high level, from a component-oriented standpoint, where each of the components themselves conforms to a particular contract.

Put another way, while Linus may not understand how the driver for a particular digital camera works, he probably does understand the interface that driver exposes, and how that interface ties in with the rest of the system.

Re:Maintainable? Stable?

[Alan Cox (27532)]

They get to be in conflict over short time periods - that was the point of that part of the talk. If you need a fix that is provably correct, simple and immediate it tends to be the ugly bandaid type fix. Those go into -ac and expire (in theory) for the next base kernel.

Linus instead is quite happy to say "ok that is a problem but the real fix is to rewrite the logic behind [randomcomponent] to properly ensure that this cannot occur'. That might take a month and is undoubtedly the right answer but it isn't the immediate answer for people hitting the problem currently.