Kernel Changes Draw Concern

Saeed al-Sahaf writes "Is the Linux kernel becoming fat and unstable? Computer Associates seems to think so. Sam Greenblatt, a senior vice president at Computer Associates, said the kernel is 'getting fatter. We are not interested in the game drivers and music drivers that are being added to the kernel. We are interested in a more stable kernel.' There continues to be a huge debate over what technology to fold into the Linux kernel, and Andrew Morton, the current maintainer of the Linux 2.6 kernel, expands on these subjects in this article at eWeek."

Compiled Kernel not necessarily getting fatter.

[linuxhansl]

Surely nobody worries about the size of the source tarball.

Drivers that are not compiled are not taking any additional space. Drivers that are not used all the time can be compiled as modules...
So I guess I do not understand the criticism here.

Re:Isn't that why,

[Stevyn]

Even if you don't compile it and rely on your distro for it, don't they usually make everything that's not required for booting as a module? So if you don't have the hardware and you don't need the driver, the module is never loaded and you don't waste the memory.

They've Been Complaining about That Since 1.3

[Greyfox]

There've been concerns about kernel bloat since the 1.3 kernel. I recall there was quite a ruckus when the compressed kernel tarball went over 10mb. But yanno it's gotten more robust and added support for a lot of modern features (Especially in networking) that I really do appreciate having the choice of compiling in. And I'd be surprised if the source was anywhere near the size of the commercial UNIX kernels much less Windows or one of the mainframe OSes. The build system seems to be pretty well capable of containing the bloat as well.

Re:What about older hardware!

[McDutchie]

As a proud owner of a Celeron 500mhz machine, I must express my concern.

This proud owner of an AMD K6 300 MHz has compiled and runs Linux 2.6.11.7 without a hitch, and continues to not see the problem.

Re:this is nothing new

[Anonymous Coward]

it's already done. vendor kernels are compiled with nearly all non-essential things as modules and you load them as needed via udev/devfs. only the loaded modules uses memory/resources, the rest just uses a couple extra spare megs of space(of your new 150GB drive)

Re:this is nothing new

[McDutchie]

What would be cool is if the linux distros had default kernel options, much the way some of the majors have Workstation, Server, etc... that would adjust the kernel based on how the machine was being used.

Slackware has this (or something rather like this) -- it comes with a whole set of kernels compiled for different kinds of hardware.

Re:Heading Down the Windows Path

[marshall_j]

yes but NT does not have an option to remove those drivers from the system. the linux kernel allows you the opportunity to omit whatever you want at compile time.

you have options and one of those options is to not use stuff.

Microkernels...

[argent]

A lot of kernel functionality could be moved outside the kernel if the intrakernel communication was regularised into some mechanism (messages, for example) that could be efficiently marshalled and moved through a user-kernel boundary.

Obviously you can't do this for everything, but you could simplify the kernel significantly if instead of separate interfaces for every kind of object a single regular interface could be used at least as a starting point. The Amiga did this, and the result was a system of rare simplicity that made quite complex components easy to implement. Matt Dillon, a long-time Amiga hacker, is working on turning the BSD kernel inside out in this way. There's no reason Linux couldn't be cleaned up and pared down the same way.

Re:WTF?

[Lemming Mark]

[ disclaimer: I'm a Xen developer ]

I'd say the parent is a fair question, not a troll.

Morton's point appears to be this:
* x86 is notoriously unco-operative to full virtualisation
* trying to fully virtualise it (as VMWare and Virtual PC do) is a work around for the fact you can't modify the guest OS because it's closed source
* fully virtualising x86 in software results in rather painful performance hits for many workloads and a very complex hypervisor
* for open source OSs, it therefore makes sense to use paravirtualisation. This involves porting the OS to a special virtual machine-oriented "architecture", closely resembling the real hardware but without the costly-to-virtualise parts.
* paravirtualisation can be argued to be better than full virtualisation because (esp. on x86) the performance hit is much lower.

Porting of open source OSs is happening: Linux 2.4 and 2.6, NetBSD, FreeBSD 5.3 and Plan 9 can run on Xen (although currently only the Linuxes are supported as "host" or "Dom0" operating systems).

Thanks, CA

[SamMichaels]

At the risk of getting flamebait or troll, I'll speak my mind anyway.

How about trying out this GREAT utility called "menuconfig"...then you can unbloat your kernel. In the time it saves you from manually editing your .config, you can unbloat YOUR products.

Re:Compiled Kernel not necessarily getting fatter.

[dbIII]

to find it loading support for pcmcia, bluetooth, and many other completely unnecessary modules

Does it actually load them or does it just print a message which indicates it's going to do so, check for hardware, and exit when it can't find it? If it does actually load the modules, won't they be unloaded after a while if they aren't used at all?

For example, if I try to load a pcmcia module on a destop machine from the command line it indicates it cannot find the hardware, and exits. I suspect the only difference with the Mandrake script is that it is quiet about it.

The "support every device possible" approach has very little in the way of a downside with a modular kernel if you have the disk space (ie. not trying to fit it on a floppy).

Re:WTF?

[dvdeug]

Yes. You need the source code of an OS to change it to run under Xen.

Re:Just my $0.02

[shellbeach]

n the other hand if you cannot download 40 megs buy a distribution on cd/dvd or use windoze

Or just download the patch instead. That's what those patches are there for, you know ...

Re:Compiled Kernel not necessarily getting fatter.

[Wdomburg]

Ummm.. you don't need to recompile a module to turn off pcmcia or bluetooth. /sbin/chkconfig pcmcia off /sbin/chkconfig bluetooth off

There's also a GUI tool for this. For that matter, you could not select those services to start in the first place. There's a dialog for it in the installer.

Re:WTF?

[derinax]

Don't underestimate NetBSD's top-flight support for Xen, which has supported domain0 for Xen 1.2 since April of 2004, and for Xen 2.0 since January 2005.

http://www.netbsd.org/Ports/xen/howto.html [netbsd.org]

Please see "Installing NetBSD as domain0".

Re:What about older hardware!

[J. T. MacLeod]

Anyone buying a new PC at that point, really.

Re:I've noticed my Compaq laptop having problems

[nutznboltz]

# cd /usr/src/linux
# make menuconfig
Then locate
Configure standard kernel features (for small systems)

This option allows certain base kernel options and settings to be disabled or tweaked. This is for specialized environments which can tolerate a "non-standard" kernel.

Re:WTF?

[Anthony Liguori]

* fully virtualising x86 in software results in rather painful performance hits for many workloads and a very complex hypervisor

Something I think Sam missed is that Xen also supports VT which provides full-virtualization on the x86 (which makes Xen undeniably a true-hypervisor).

Compiler-driven para-virtualization is an interesting emerging area of research too that should make porting OSes to Xen much simplier.

All we need now is a really cool hypervisor-aware file system.. like a XenFS ;-)

Re:Just my 5 bytes

[zborgerd]

Agreed. Anyone who knows enough about the kernel to warrant a complaint about the kernel being "bloated" should simply build it on their own (that goes for Greenblatt and the goons at Computer Associates).

There is no reason that these "experts" can't tune a 2.6 series kernel to around 1 MB (maybe less). Kernels with modest support for lots of hardware are still around only 1.5 MB at best. Anyone complaining about it is simply talking out of their asses.

You don't want "game drivers and music drivers", then exclude them. There is no science to it. But I *want them* in my kernel, and many other people do as well.

Additionally, if Greenblatt and co. want more "enterprise features", they're certainly welcome to add time and money into developing these components.

This e-week article is misleading. It's not drawing "concern" for anybody, especially not the "open-source community". Computer Associates is not the "open-source community".

Re:What about boot-time loadable drivers?

[atomic-penguin]

Umm, could you clarify that? There is something called an initial ramdisk which loads critical drivers required to boot. So you can have a smaller kernel image by making these critical drivers loadable modules. No matter what, you still have to compile them.

I must be missing your point Mr AC.

Re:I guess this faq is wrong then.

[rk]

No, you're both right. The OpenVMS kernel was written in VAX Assembler, but components of OpenVMS were written in a variety of languages. Like someone [stallman.org] we all know and [love|hate] likes to say about what is popularly called Linux, OpenVMS is much more than just a kernel.

Re:What about older hardware!

[ChadN]

Googled for "Longhorn specs" and this is what I found. There seems to be a reasonable explanation.

http://technovia.typepad.com/technovia/2004/05/lon ghorn_specs_.html [typepad.com]

In a nutshell, it comes from a slide at a developer's conference, indicating the kind of machines that may be around for Longhorn's lifetime, and that the OS should be able to take advantage of such high specs, not that it will require such high specs.

menuconfig is too hard

[mwa]

Dear CA,

Try this:

# cd /usr/src/linux
# cp /boot/config-$(uname -r) .config
# make oldconfig

If it asks you any questions, those are new features that you weren't using before so just answer "N". when it's done, proceed to build your kernel, and it will be no more bloated than it was before.

Re:What about older hardware!

[rgmoore]

My concern, though, is that Linux, while acceptable now, will take the bloat route that KDE and GNOME have.

That seems unlikely. Embedded Linux is a big deal, and embedded systems are far more squeezed for memory than even quite old desktop systems. That keeps pressure on Linus and Co. to keep the compiled kernel (though not the sources) from getting too bloated. If you read summaries of the LKML (like Kernel Traffic) you'll find that there is a set of developers who are constantly looking for ways of trimming fat out of the kernel. There's also some pressure from the high performance end, since a slow, bloated kernel will steal memory and processor cycles from running applications. Between those two groups, there's enough pressure that we can reasonably hope for the kernel itself to resist excessive bloat.

Re:A newbe question

[DavidTC]

Disabling modules won't protect you from cracking attempts. If, God forbid, the kernel has an exploit in it, it's really unlike that it would be in an unloaded module that the cracker can somehow cause to be loaded. Either the module's loaded, or it's compiled in, and you're screwed.

What it will stop is rootkits. Things that alter files on disk to install backdoors, and then install a kernel module to hide said changes. Sometimes this kernel module will also hide the listening ports, or even keep them closed until it detects a special knock. It will had backdoor processes. It can do anything, and you'll never find it.

So completely disabling them? Yeah, that's a good idea for a server that isn't going to see much hardware changes. I've done it for a router before. It won't keep crackers out, it will just make them slightly more visible. Or at least keep them from being completely invisible. (Barring, of course, their patching and compiling a new kernel, but that requires, at least, a reboot.)

Merely limiting the number of modules, however, while having them enabled, doesn't help anything at all. A rootkit modules can load and then remove itself from the list and you'd be no wiser.

CA's kernel demands

[sloanster]

We are not interested in the game drivers and music drivers that are being added to the kernel. We are interested in a more stable kernel.'

No offense, but he sounds pretty clueless here - not to mention the fact that there is no "game driver" or "music driver", perhaps he is referring to device drivers and/or low-latency features, which allow for a better gaming/multimedia experience...

In any case, he completely misses the point that the kernel, as shipped by the distros, is modular. That means, if a device isn't present, or isn't used, the driver for that device never gets loaded into memory. So it doesn't really matter how many devices are supported, the only device drivers affecting the size of the kernel are the ones loaded into memory on the machine in question.

I find Greenblat's attitude ridiculous, since he seems to be saying that the kernel developers need to focus on what Sanm Greenblat is interested in, and to hell with people who want to do cool and interesting things with linux, which aren't part of CA's business plan.

I could go on, but that's enough for a first impression.

Re:Just my 5 bytes

[Anarke_Incarnate]

This is nothing like patching an XP/2000/2003 machine to protect against the exploit dejour, right?

All you have to do is recompile one time and then just transfer the kernel you want to all the machines, change the boot loader and voila. In the Windows world you have 3 choices: You can download and install the updates by sneakernet, you can set up a patch management system for them or you can get a company image up and going and then reimage all the machines

(I am sure there are other ways, but these come to mind in my cough syrup infused brain quite readily)

Re:WTF?

[Lemming Mark]

Anthony's point was that it will be able to run unmodified guests using Intel's Vanderpool / VT-x extensions and using AMD's Pacifica extensions. VT-x is shipping some time this year, IIRC. I'm not sure when Pacifica is shipping.

Re:Compiled Kernel not necessarily getting fatter.

[John_Booty]

Have you ever installed a late version of Windows? Watch the installer load device drivers for every known weird form of RAID before it even begins to ask you how you want to install the OS?

That's only during the install, though. I agree that Windows' habit of loading all those bizzarre disk drivers into RAM during installation is kinda... crazy and inefficient, but that's only during installation!. It does not load "every known weird form of RAID" during normal operation.

Re:Just my 5 bytes

[MyDixieWrecked]

i dunno... I compiled my kernel (make menuconfig, w00t), and tried to only have it support the hardware I was gonna use... no joysticks, limited video drivers (just the base RADEON ones, no Xfree DRM or anything), USB (for keyboard), drivers for my ethernetcard, CUDA, CDROM, my filesystems (ext2/3, hfs), iptables, and whatever else it was I was gonna use...

I'm surprised the kernel's as big as it is:

spike@fingerbib spike $ du -h /kernel-2.6.10-r6
4.3M /kernel-2.6.10-r6

CA may have a point.

Re:The Big Bloat

[Cid Highwind]

I really do wish things would "just work" sometimes. I get tired of hearing about how only SCSI CD Writers matter, when every CD Writer I've seen in the last 5 years has been IDE, for instance.

You really need to update your "why linux sucks" talking points for kernel 2.6. :)

SCSI emulation is dead (finally!), and ATAPI is taking it's place. Update to the latest CDRtools package, and you can use "cdrecord -dev=/dev/hdc" instead of mucking around with fake scsi devices.

Re:The Big Bloat

[Anonymous Coward]

Part of the problem with that is that the Linux kernel has never had a particularly stable ABI. Therefore, kernel changes need to be accounted for in the drivers. (This, incidentally, is much of the reason why binary-only drivers suck so hard)

Mod me troll if you will, but stable binary interfaces would make driver maintenance a whole bunch easier, and perhaps result in fewer companies put off from supplying Linux drivers.

Of course, it would also be much harder to significantly improve a lot of things, being as it's fundamentally a macro-kernel. Ah well.

Modules that work with different kernel versions

[gilesjuk]

A lot more work needs to be done to make modules work with various kernel versions.

This way binary driver modules can be placed on company websites and then installed.

Sure, this is more microkernel like, but Microkernels are a lot more modern and easier to manage.

Good thing $0.02 is pretty worthless.

[grishnav]

quote osdl.org (emphasis mine):

OSDL - home to Linus Torvalds, the creator of Linux - is dedicated to accelerating the growth and adoption of Linux in the enterprise. Founded in 2000 and supported by a global consortium of IT industry leaders, OSDL is a non-profit organization that provides state-of the-art computing and test facilities in the United States and Japan available to developers around the world. OSDL's founding members are IBM, HP, CA, Intel, and NEC. A complete list of OSDL member organizations is provided on the member page at OSDL Members.

Re:"fatter"

[cowbutt]

That sounds like a lazy excuse for not implimenting a decent architecture for loading outside drivers. I'll use DLL's as an example. I can replace a DLL to my program, without recompiling it and it will still work properly. You can make your own DLL and replace the one I have, and the program will still function properly.

...provided that the API supplied by the new DLL matches that provided by the old.

The Linux kernel needs this for drivers, and it's probably similar to the method Windows uses.

It already has it, has had it for about nine years, and it's called modules. I've many times compiled driver modules external to the kernel and used them as replacements for kernel-supplied modules. It works. Now, if you're saying that the Linux kernel should better support closed, binary-only driver modules (e.g. nVidia), that's a whole other argument.

Re:Just my $0.02

[Chreo]

My problem is when the freeloaders start telling Linus that he can't take what is still his hobby (and now lots of other people's hobby too) in the direction he wishes to take it.

Last I checked, Linus was getting paid to do work om Linux. At one point it sure was ONLY a hobby but now (and a while back) it is his work also.

Re:Just my $0.02

[lanc]

IIRC he admitted it.
http://www.dina.kvl.dk/~abraham/Linus_vs_Tanenbaum .html [dina.kvl.dk]

True, linux is monolithic, and I agree that microkernels are nicer. With a less argumentative subject, I'd probably have agreed with most of what you said. From a theoretical (and aesthetical) standpoint linux looses. If the GNU kernel had been ready last spring, I'd not have bothered to even start my project: the fact is that it wasn't and still isn't. Linux wins heavily on points of being available now.

unfortunately, popularity of a kernel doesnt depend only on the technical advantages.

Re:Just my $0.02

[FooBarWidget]

You are allowed to do that. For example, the Galeon developers explicitly added an exception to their GPL license to allow linking to Mozilla, which was MPL at the time. The MySQL developers added an exception to the GPL license to allow the non-GPL'ed PHP to link to the GPL'ed MySQL libraries.