Fork the Linux Kernel?

Joe Barr writes "Fork the kernel? Are you crazy? A blog entry on InfoWorld.com urged the Linux community to fork the kernel into desktop and server versions because, according to the author, all Linus Torvalds cares about is big iron. Sorry, but that's both wrong and stupid."

Well that's the beauty of Linux...

If you want to fork the Linux Kernel, there's absolutely nothing from stopping you from doing it yourself. Wanna tune a version just for Desktop or Server? By all means, just adhere to GPL. Your attempt at forking might even get some support from the community, however I'd think Linus's blessing on such a fork means something however...

Re:Well that's the beauty of Linux...

Or, alternatively, you could just custom compile the fucking thing to take out the "big iron" if that's what you want. I frequently custom compile kernels, particularly when I'm putting Linux on older hardware.

There's nothing quite like the grand proclamations of the idiots.

Re:No you can not

Putting a bunch of #if 0's into complex, bloated code doesn't make it slim and efficient. Statements elsewhere still make assumptions about one of 1000 things happening rather than one in 10. Slow, scalable algorithms are used rather than lean but limited ones. make config is not going to turn your Linux into FreeDOS.

Another approach is to use an object-oriented model, so you just include the implementation you need for the specific interface or class. I believe Darwin (the kernel used by MacOS X) already uses such an approach for some things?

Re:No you can not

The linux kernel already does this, with modules that can be loaded and unloaded at runtime. Whole subsystems (things like SCSI and DRI) can be loaded on demand. You can also enable or disable kernel preemption at compile time, and you can swap out I/O schedulers at run-time.

However, the modular approach can have some overhead of its own (though not as much on linux as on darwin). If you really need a small kernel, you can actually disable loadable module support at compile-time, if you know exactly which drivers you need.

Re:No you can not

Sure, linux kernel modules don't have any significant execution speed overhead, wheras darwin does because of excess context switches. The overhead I was referring to on linux was just that a modular kernel (with all the modules loaded) can take up more memory. If you know you will always need all your drivers (usually for server and embedded workloads), compiling in everything and leaving out loadable module support can cut down on your RAM usage (though this typically only matters for embedded systems).

Re:No you can not

Have you got any examples where there is significant overhead that can't be removed with a make config but could be removed with a specific, less scalable algorithm that isn't available in the kernel source?

I'm pretty sure your comment is mostly BS. The vanilla kernel source includes a lot of configuration options for embedded systems. Low on RAM? Turn off CONFIG_BASE_FULL to use several smaller, slower data structures. Don't have swap space? Turn off things like CONFIG_SHMEM. Using uClibc? For now, you might as well throw out CONFIG_FUTEX as well.

Re:No you can not

Your examples totally miss the point. The CPU scheduler is a *lot* more crucial to desktop performance than swap space, memory config etc. etc.

Have you even been keeping up with the whole CPU scheduler in the kernel issue that the article mentions?

The whole point is that the CPU scheduler is NOT modular and you cannot change its behavior by much by changing kernel options. Con(along with soemone else) made patches to make it modular, calling it plugsched, but it was nixed from getting into the kernel by Linus who said something on the lines of "The scheduler is not something you see frequent changes in."

Con wanted it because desktop users can easily plug his desktop-centric scheduler into the kernel. For a lot more details read here [apcmag.com].

Re:No you can not

You're missing the point. A pluggable scheduler is not necessary for desktop usage, because nobody (not even Con) has been able to come up with a scheduler that is significantly better than CFS for desktop usage, except by doing things that amount to hard-coded nice levels. All of the meaningful performance improvements have made it into the default scheduler.

Re:No you can not

except by doing things that amount to hard-coded nice levels. All of the meaningful performance

meaningful according to whom? and desktop users couldn't care less about 'hard coded nice levels' if it means their 3d games and/or X apps work better: yes, I know this is anathema to the linux developers where only super perfect code supposedly should go in, however if this supposedly super perfect code doesn't meet desktop users' needs as well as hacks, well, I'd all be for giving desktop users as many hacks as they want/need (as long as this could be changed via either a pluggable architecture or a difference in make config).

As much as Linus has done a great job into making linux a great server side OS, if he's not willing to make compromises to make the desktop faster (because either it's too 'hacky' or it will cause issues for big iron, which is what pays the devs' bills) maybe it IS time to fork under the stewardship of somebody with the desktop users' needs more at heart. If companies like, say, NVIDIA or Adobe paid a kernel developer to make linux better on the desktop this is what would probably happen IMHO.

I don't think a fork would be the end of the world, fork it and let the best survive: if a year from now we have a 'server linux' and a 'desktop linux' kernels, so be it, if instead the 'desktop linux' project flounders due to minimal speed improvements and so on, well, so be it as well. The vast majority of the patches/changes would apply to both the same way, so I don't see this causing issues and slowing development, if at all maybe people could spend less time flaming on LKML and more time writing code.

Re:No you can not

That is a gross over-simplication of what happened and almost qualifies as revisionist history and brushing things under the carpet. Let me summarize my understanding of what happened and someone please correct me if I am wrong.

Con Kolivas had been shouting from rooftops about slow desktop performance and was submitting feedback and bug reports. One of the kernel devs apparently said "I do not notice the issue on my quadcore machine with 4GB RAM". Rightly or wrongly, this lead Con to believe that the kernel devs do not care about desktop performance and only give priority to issues that big corporates complain about.

In the true open source style, he took upon himself to learn kernel programming and released a whole set of -CK patches and various versions of benchmarking tools and schedulers. On the other side, Ingo Molnar was the maintainer of the scheduler portion of the kernel and maintained that the O(1) scheduler(and the one before it?) is good enough and has no problems. Con conclusively started proving this wrong with his benchmarks. At this point, everyone assumed the -CK branch would be merged into the kernel at some point and Linus says he had been considering it.

At some point, Ingo starts making his own scheduler, which later evolved into the Completely Fair Scheduler. A number of posts claim that it was kind of rip off of the ideas behind Con's scheduler with which it was in a race to get included in the kernel. Then Linus decides to include CFS into the kernel instead of Con's scheduler. The reason he gave was that Con thought SD was perfect and that he ignored and flamed the users on the CK mailing list and that he(Linus) was far more comfortable working with Ingo since he knew him well. He also admitted that he might have formed this opinion on a single incident on the mailing list and he didn't have the time to follow the CK mailing list.

Some people on Con's side in the LKML tried to explain this by saying that the single incident was in response to a troll who submitted faulty bug reports and ignored the reasons for why they were rejected and that Linus was playing favorites. Con couldn't take the non-inclusion of -CK and plugsched(which would have given users a clean way of using a custom scheduler) and quit kernel development totally.

The latest twist in the story was reported on Slashdot here [slashdot.org]. The gist of it was that another hacker(Roman Zippel) was trying work on CFS. He had asked questions about what some parts of the code did, and also made some patches that considerably simplified the code and mathematically proved his patches made things better. In response, Ingo came out with a big patch that ripped out the code that was questioned and included Roman's Zippel's ideas(another rip off?) with hardly any discussion and a tangential acknowledgement of including his changes. Roman complained that talking in patches without explanation is detrimental to collaborative OSS development.

Re:How much improvement?

whether or not the gain balances out the pain

what's the pain really? business will continue as usual on LKML etc., there will just be a separate tree handled by somebody interested in this which will accept 'desktop patches' and will also integrate most, if not all, of the mainline patches.

So what if you get an extra FPS in Quake

and why shouldn't desktop users get that extra fps? desktop users couldn't care less that getting an extra fps in quake will lower some Oracle benchmark by 50%. Also what if, by really messing things up for databases or network loads and by hardcoding specific scheduler behaviour for the X binary, you could make xorg 50% more responsive? No way this would go in a mainstream kernel, but I bet a lot of users would run this quite happily if they could.

Personally I'd rather have a system with more internal checks and layers to ensure stability and to protect the kernel from hacks and attacks.

I am sure there are people that feel the same way you do, maybe you could consider a fork yourself? ;)

Re:No you can not

Yes, you are correct, but you're missing the point. The Linux kernel is made to provide maximum throughput at the expense of responsiveness. Throughput is great for a server, responsiveness is great for a desktop. There is a trade-off.

Re:No you can not

CONFIG_PREEMPT and CONFIG_HZ are pretty much all you need to worry about.

Re:No you can not

Really? I better tell the guys down in R&D that the 1.2 meg linux install we use on the embedded box devices does not exist and cant work.

Thanks for letting us know before shipping this thing it would have been embarassing that the linux would explode and become huge upon use.

Re:Well that's the beauty of Linux...

Can you, or this blogger, or anyone, site some actual evidence that shows what the fuck "optimized" even means? You know, you guys go around spouting stuff about how Linux is too serveresque, but no one so far as I've seen has even defined that. A decade ago there might have been something to the complaint, although I can tell you now that I can take a Pentium 233mhz and turn it into a router running the newer kernels, and have it run like a hot damn, so I think you, like some other folks, are just spouting ill-conceived crapola.

Re:Well that's the beauty of Linux...

Actually a lot of forks do exist and are supported. There are all kind of real-time and low-latency and security patches floating around that get a lot of attention. Most big vendors do not ship a exact copy of the version that linus creates, but add some patches/modules that they think their actual users need.

One time they may be get merged into the main linux kernel, or maybe their features are obsoleed by features that are accepted by linus.

Meh

I'd rather spoon it

Re:Meh

Me, I'm gonna spoon the fork. I'm gonna SPORK it. :-D

Re:Meh

spooning can lead to forking, if you are lucky ;)

Why is it stupid?

I can not see why is it a stupid idea. Forking the Kernel in desktop and server forks will mean that each specific kernel is optimized for such tasks and that the distribution makers have just a subset of the huge kernel to care about when creating their distributions.

A server is a relly different beast than a desktop and having this "all-in-one" kernel means that the operating system gets bloated with a) desktop specific features when using a server and b) Server specific features when installing a desktop.

I think that a controlled fork in the linux version control tree might be beneficial.

Re:Why is it stupid?

A server is a relly different beast than a desktop and having this "all-in-one" kernel means that the operating system gets bloated with a) desktop specific features when using a server and b) Server specific features when installing a desktop.

Perhaps the source code does, but there's nothing stopping you from leaving out all the server-specific stuff from your desktop kernel when you compile it. If you're producing a server-grade OS, leave off the desktoppy stuff. Simple.

That's exactly what Ubuntu does

Perhaps the source code does, but there's nothing stopping you from leaving out all the server-specific stuff from your desktop kernel when you compile it.

If I understand correctly, that's exactly what Ubuntu does with their "desktop" and "server" version. The desktop version have certain modules and patches that the server versions do not, and vice versa.

Re:Why is it stupid?

Which algorithms are bottlenecking your desktop? Is it the I/O scheduler? You can swap between one of four choices, at runtime.
Is it the CPU scheduler? If so, you're a liar. Nobody had produced repeatable benchmarks that show a significant shortcoming in CFS for desktop and gaming use.
Is the memory allocator really bad for your workload? Try using the new SLUB allocator, instead of the older SLAB allocator.
Is the system not as responsive as you want? Turn on forced preemption and set the tick speed to 1000Hz.

inconsistency

I can not see why is it a stupid idea.

Me neither, especially considering that's all the frothy-mouthed zealots tell you to do when you criticize the kernel developers.

Linux user: I like Linux but I think the kernel should incorporate feature X.
Linux zealot: If you don't like it, fork the kernel!
Linux user: I think the kernel developers aren't open enough to contributions.
Linux zealot: If you don't like it, fork the kernel!
Linux user: I think the kernel is too focused on big iron.
Linux zealot: If you don't like it, fork the kernel!
Linux user: Ok, I guess I'll fork the kernel then.
Linux zealot: OMG YOU CAN'T FORK THE KERNEL!!!

Distro should be doing the fork

Really, the distro should do the fork, and they actually do. While most have general compiled kernels, others have kernels compiled based on what is desired; server or desktop. Solves the issue.

Re:Why is it stupid?

Because the distinction between server and desktop is rather fuzzy these days. What could you leave out of the desktop OS?

RAID? Doubtful with it being so affordable these days.
ECC RAM? That can be had on many boards as well.
Support for SCSI tape drives? Does my box suddenly turn into a server if I get a cheap drive on ebay?
Ok, how about say, optimizing the desktop version for latency and the server version for throughput? Problem with that is that there exist server tasks that want low latency.
Years ago you'd say "remove SMP support, nobody uses that". Not so these days.

What could you leave out of the server?
Support for sound cards? What if it's a server that records audio?
Support for video cards? What if the server uses it for computation (rare but possible)

The Linux Desktop already crawls..

Call me stupid, but the Linux desktop already crawls.

There used to be a time I could download 5 shared files, burn a CD and watch a DivX movie at the same time. That was with Slackware 9.0 and Linux 2.4.20.

Nowadays it takes my browser 2 seconds to open a *tab*, and another 2 seconds per website. This happened because there was continuous I/O activity in the background. After the I/O completed everything was back to normal. Bottom line: every serious I/O activity causes the desktop to crawl.

It's still the same machine (AMD 1800 and DMA-enabled) but interactivity my Linux system had is unmatched by the recent kernels. The problem is too many commercial developers care about server performance alone, or test desktop performance with their quad-core raid array configuration. Patches get rejected too when they affect server performance.

I'm honestly not surprised people want a change here, or even start suggesting a fork.

Fork?

It's a blog post, so it's not like it's going to happen, but I don't see how forking the kernel would do anything than just lead to distribution craziness. Arguably that's Linux's biggest hurdle for new people -- deciding which distribution to get. And if people are checking out linux for workload purposes, forcing them to decide whether to get a server distro or a home distro and making that distinction at the kernel level? Buh?

Generally, if it's good enough for enterprise, it's good enough for home use. And things that are useful for desktop Linux are often utilized at the enterprise level anyway. So yeah, it's just a blog post; I'm not sure anyone will take it seriously.

It already is "forked"

... but only in the sense that it is customized for different purposes - mobile phones, desktops, servers, supercomputing clusters.

Besides, most people's desktops are much more powerful than any server you'd be able to buy years ago. With the cost of cheap disks going down, there's no excuse for even home users to ignore the benfits of such "server" features as raid.

What about SMP?

You could have made the same argument against including SMP a few years ago. And look, now ~90% of PCs (thats personal computers, for you me grandma and the king of Tahiti) have multiple processors. We don't know the direction computers are going to take in the future, but a lot of previous high end server stuff has trickled down into the consumer level hardware.

Why is this even controversial?

Despite all the warm, fuzzy talk of open source and community development, the fact remains that, at the kernel level at least, Linux is still controlled by a small group of elitist "prigs." Stick too close to the "approved" Linux path and you end up with a crappy desktop experience. Stray too far, and you risk having your customizations broken if/when the kernel team decides to take things in a new direction.

Why is this even controversial? If you don't like the way things work, the beauty of open source is that you can fork the code at any point. So...quit whining ("prings"?) and good luck with your fork.

Fork for other reasons

A different branch of distros for the desktop makes sense, but I'm not sure the kernel is what needs addressing.

It makes sense for Linux to fork into two branches: one, a conservative one, aimed at upkeeping what already works, and the second, a wild-ass anarchist, aimed at forging new and innovative technologies.

I think what the original author was saying was that he/she would like the Linux community to fork into two branches, one thinking like desktop software (Windows XP is the best example) and another thinking like big iron, where Linux already has a presence but could learn a thing or two from *BSD.

Fork? No. Seperate projects? Yes.

There is no need to fork Linux into a "desktop" version. Projects like Syllable [syllable.org] already exist, and we re-use a fair amount of code from Linux, GNU and other OSS projects.

So?

Shrug. Let 'em fork it. I doubt they'll be able to swing enough maintainers to seriously effect development on the main fork.

One of the great strengths of open source is that it allows for competing code. If the new fork is better (I view this as unlikely) then I'll switch. I'm about what works.

When the level of discourse falls to articles of faith and prejudice, it's not about what's best for the code anymore. It's about your personal ideology, y

Keep the code together; make it configurable

# Forking isn't necessary.

options BIGIRON
#options DESKTOP

Re:Keep the code together; make it configurable

At the moment I'm making this post, the parent post has been moderated "Interesting". I think "Insightful" or "Informative" would be more appropriate.

What the parent poster is saying is that C pre-processor [wikipedia.org] flags already allow the same kernel source code to contain features for both server and desktop without resulting in any bloat or compromise in the resulting binary.

Only those who don't understand C would fret about a "bloated" kernel in this context.

Now given a binary kernel that contains both feature sets you would have a legitimate concern, because then there would certainly be a bevy of both bloat and compromises. But this is linux, after all. We have the source code -- so none of that matters.

The Kernel Isn't The Issue

Linux has far too many varieties already, it makes mainstream hardware and software support almost impossible.

And they want to fork the only consistent bit ?

If they want to do a desktop version, it's time for the kernel developers to branch out into standardising Desktop libraries, desktops (KDE vs Gnome), devices, packages etc etc... so that we can have our 1000 versions of Linux and a single underlying version of Desktop Linux.

Maybe then, Linux may make a dent in the world of Desktop Windows.

Re:memories...

Uh, you know that it still does? You just have to pick and choose what you want. If Linux doesn't run in 16 MB of Ram, how do people get it running on things like the Nintendo DS with a whopping 4 MB?

Aside from the flamebait-ish nature of the post..

The less segregation in the Linux world the better, at least until desktop Linux is better at coping with new versions of the current kernel line (eg nVidia graphics drivers needing recompiling when a new version comes out and that sort of thing). Having different forks of the kernel would eventually also lead to software that can only be run on one fork without modification, and that's not much use either. The less work involved in porting to different distros/platforms, the better IMO.

One size rarely fits all

People who advocate this aren't necessarily stupid, just ignorant. The Linux kernel's flexibility is being taken to the limit, and people are forgetting the easiest way to improve performance for their particular rig: Customize your kernel! You can add all the code in the universe, and then you pick and choose the particular things you need or don't need! Say I run a 486/25 with 16 MB RAM as an IP Masq router. The hard drive is an old IDE with 600 megs of space. I have two network cards, and that's about it. Do I need SCSI support? Do I need to support joysticks, X, Pentiums, AX.25, or anything else? No! I compile a kernel specifically to run the IP Masq, and run it well. My P100 laptop, on the other hand needs a bit more. I use it for packet, so I need AX.25. It uses PCMCIA, so PCMCIA support needs to go in. I use Seamonkey and the GIMP, so I need graphics. But, my HD is not SCSI. I yank out SCSI. My CPU is subject to the 0xf00f bug, so that gets included. I brew a custom kernel, and boot time is a lot shorter. My big-rig is a AMD X2. I need just about everything, as I have a Nvidia card for Quake4; a SCSI scanner; and a connection to my Packet base station. I optimize compilation for the higher-end computers. I plan on getting a Mac Pro from Apple and putting SuSE on it. Again, by optimizing the options I optimize my system. Get the point? If you want a once-size-fits-all kernel, use Windows. If you want a kernel which can be adjusted for your particular and peculiar environment, use Linux and customize your kernel!

This doesn't sound like a good idea

The advantages gained in forking a kernel are minimal compared to the disadvantages. Not to mention a lot of those advantages of a fork can be obtained by simply compiling a kernel based on your server's hardware and computing needs. If someone forked it they would then have to maintain two separate code bases, two separate patch bases a new naming scheme. Not to mention the main advantage stated which is getting rid of bloat occurs because of compiled driver support which means that only a small subset of hardware would be supported in theory and most of the bloat he is speaking about comes from the GNU side of things and can easily NOT BE INSTALLED or un-installed if so necessary...

I don't see the need

The only difference between a "server" build and a "desktop" build, kernel-wise, is in which components/modules you compile. Functionally, there is no difference. Same goes for Windows, the "desktop" and "server" kernels are fundimentally the same, it is only what you put on top of them that differentiates the two.

Someone here does not understand the difference between a kernel and an OS.

Why not.

I think that right now the majority of development at the kernel level is server based. It is only logical after all since the majority of paying Linux systems are servers. When I mean paying I mean paying their way. The technical question is can one scheduler work well for both server loads and desktop loads. Is there an ideal scheduler that works every where? We know that isn't true when you are dealing with real-time systems so is it true for the desktop?
I don't think this is a dumb question I just happen to think that currently there isn't a need to fork the kernel.
I happen to think that currently there isn't really a need to fork the Kernel into a server and desktop version. I feel that most of the performance problems with Linux on the desktop are in X and not in the kernel. I think more work needs to be done in X to solve the problem than the Kernel.

Re: "wrong and stupid"

I've generally found "Wrong and stupid" goes hand in hand with blogs. The easier it is to be heard, the lower the signal to noise ratio is going to get. It'd be nice if we could just taser them but perhaps unconstitutional. :D

Relevant quote: "Don't taser me! Ow! Ow! Ow!" - opportunistic journalist at Democratic National Convention

WTF?

First, Linux is Linus' hobby that is kinda also a job. I've read somewhere where he said that he is more proud of Linux being on a digital picture frame that he bought his wife than having it on the top500 list.

Second, AFAIK, the kernel is fine for both desktop and server stuff. There are compile options to optimize for each, and patches, etc. Linux on the desktop is difficult because of a lacking standard and good software installation system and GUI environment and various other things. The kernel is fine for the desktop. There simply is not software on top of said kernel to make it desktop friendly.

I saw it on an Internet blog!

It's probably a serious concern!

Why isn't this front page news everywhere?

What's this? Someone with a blog pulled a half-baked idea out of his butt, and then posted it where the entire Internet could see it? And some other people don't agree with him?

That's amazing! An event of this magnitude only happens once in a billion femtoseconds! Why aren't we all paying more attention to this incredible discovery?

So why the attention?

Okay, so somebody made a stupid blog post. Why submit it to Slashdot?

Old troll is oooooold

See subject.
The linux development model is built on forking anyway.
Trying to fork linux is like trying to burn fire.