Missing Kernel Patches 159
BlueEar writes: "There is an interesting, short story posted on the Gentoo Linux site. It talks about kernel patches created by Linux distributors that while publically available never get submitted. It even gives an example of one 'no brainer' patch that has been sitting over a year, without being incorporated into the 2.4.x distribution. The article ends with an appeal to Linux community to keep those patches flowing to Marcelo."
They Don't Get There By Themselves (Score:5, Insightful)
What most people don't realize that someone who puts together these releases like Linus or Marcelo is by no means omniscient. The kernel is a huge piece of work, and no one person can know what's happening in every corner of the thing. Most of Marcelo's time goes to merging patches, so he surely cannot go around the net looking for what ever fixes some distributor might have used or even checking out how come some fix that was circulating around before was never submitted to him.
What's nice is that nowadays there seem to be a couple of "patch harvesters" on the lkml who create their own releases (Alan Cox is now one of these people!) and persistently keep submitting forgotten patches.
I am wondering... (Score:2, Insightful)
I am wondering if the distributors themselves don't have too much interest in offering patches upstream, not only with the kernel. Commercial distros have a chance to become "pseudo-proprietary" this way.
I think this is a rather childish behavior and use Debian [debian.org] instead.
Re:I am wondering... (Score:1)
Read lkml thouroughly for a while and see why.
Using debian or *bsd doesn't endow anyone with any moral superiority. Deal with it.
Re:I am wondering... (Score:2)
It would make things *easier* for them (Score:3, Insightful)
--
Re:It would make things *easier* for them (Score:2)
Re:I am wondering... (Score:5, Informative)
This plain isn't true, and whoever wrote the article on gentoo.org just shows he doesn't have the slightest hint of a clue.
There are some good reasons not to blindly apply distributor patches into the main kernel (for example, we have quite a few workarounds for bugs, but the right way to fix them in the official kernel is to fix them, not to add workarounds), and there are some other things preventing other patches from getting in (e.g. Linus not having the time to handle them immediately).
Other stuff is controversial (such as Red Hat Rawhide kernels putting in the Rik VM rather than the AA VM currently in upstream).
The patches are sent upstream, but at least Red Hat doesn't believe in forcing upstream maintainers to accept all patches we send.
Agreed on the article (Score:4, Interesting)
From Kerneltrap's wonderful interview with Andrew Morton [kerneltrap.org]:
The above article should be required reading for those following/concerned about kernel development.
Re:I am wondering... (Score:1)
And he certainly does have a clue.
Re:I am wondering... (Score:1)
Re:I am wondering... (Score:1)
Re:I am wondering... (Score:1)
Thats absurd (Score:4, Insightful)
It is extremely difficult to be proprietary when you are bound by the GPL. If your referring to Red Hat's using Rick's VM, there would be no stopping you from nabbing a
I also use Debian and must tell you that they make changes to the kernel. That is good, however. It just isn't practicle for a distro to try and update to the latest kernel. Plus if you like me, the first thing you do on any distro is nab a tarball from ftp.kernel.org.
Re:I am wondering... (Score:5, Funny)
Then why are you posting on /.?
Re:Should have vote for kernel features (Score:1)
That would be a neat idea in your Linux distribution - why don't you go and do it?
For now, the "real" Linux is a benevolent dictatorship of Linus for Linus (and sometimes not all that benevolent either :). I'm not saying this is a bad thing, it just means that what you get is exactly what Linus wants, either because he likes the idea or because people he trusts talked him around to liking it.
I'm not sure he could maintain that oversight if people were voting in changes all the time. Although I could see a system where various patches get votes, and then the winner(s) are given a chunk of Linus precious time for him to consider them. The big stumbling block in the Linux kernel process at the moment is not lack of patches, it's lack of time for Linus to consider them and integrate them.
The quality? (Score:4, Insightful)
Re:The quality? (Score:4, Informative)
I don't understand why a distro would bother shipping a kernel (or app for that matter) with a patch that was "ad hoc". It wouldn't exactly endear their customers to provide repeat business.
I think you will find that most distros test their patched kernels thoroughly before releasing them to the world. This would include not only checking that the patch fixes the problem, but that it compiles on all supported architectures and does not jeopradise future modifications to the same bit of code or adjacent or related pieces of code.
Why they don't submit all the patches to the kernel maintainer I don't know? Maybe the patch was submitted and was passed over or missed and then not resubmitted.
marty
Re:The quality? (Score:2, Informative)
Re:The quality? (Score:3, Informative)
Easy: Because a kludgy workaround is preferrable over a bug, and we don't always have the time to fix things the right way.
I think you will find that most distros test their patched kernels thoroughly before releasing them to the world. This would include not only checking that the patch fixes the problem, but that it compiles on all supported architectures and does not jeopradise future modifications to the same bit of code or adjacent or related pieces of code.
This is true - but it doesn't include checking for stuff that's just a workaround for a bug with a relatively bad code quality.
Why they don't submit all the patches to the kernel maintainer I don't know?
Because the guy who wrote the article either didn't check the facts or lied.
At least as far as Red Hat is concerned, patches do get submitted.
Re:The quality? (Score:2)
Re:The quality? (Score:1)
True, I arn't the one whos gonna work on it in future... but when its BROKEN, it's BROKEN! Whatever happened to the days of 0 day patches to fix even the smallest typo errors?
These patches can hardly be critical (Score:4, Interesting)
Obviously with the number of people (especially "power" users) who run the "generic" kernel any critical flaws are going to get uncovered and patched. I think these kind of issues, that directly affect the stability of the kernel are more important than "clean up" type patches this article describes.
Obviously they're nice to have, but it's hardly a priority when there are bigger fish to fry.
eR: no - These patches can be critical (Score:2, Insightful)
Admittedly this can't be giving that bad an effect, as it would have been fixed in the main kernel but it looks like it could make the system go BOOM !
Re:These patches can hardly be critical (Score:1)
I'm no kernel expert, and this is no place for a kernel argument, but doesn't that code actually fail to reserve a page that it's supposed to? It looks like a potentially serious bug, though I'd have to see the reserve loop in context to decide one way or the other. What if eidx is stored somewhere as the highest reserved page, and some other code relies on that page being available for use? We'd have the kernel overwriting a page that may contain vital data.
Re:These patches can hardly be critical (Score:5, Informative)
Re:These patches can hardly be critical (Score:1)
I don't doubt that the kernel janitors does a great job and that it's a useful resource for kernel maintainers, but the article seems to have given a false impression of the nature, and possibly the scale, of the work to be done.
Re:These patches can hardly be critical (Score:3, Interesting)
consensus was thats its tricky to prove correct, its 1 page of memory
So nobody can actually figure out whether this eidx thing points to the last page or just beyond the last page. I find that quite worrying...
Re:These patches can hardly be critical (Score:1)
My point was that the purpose of any variable should either be obvious from the context or documented (heard of comments anyone?). e.g. if somewhere in this piece of code there was a line that said:
eidx = NumPages - 1
or
/* eidx points to the last page */
eidx = obscure_variable_name
you'd know whether the patch was correct or not.
Alan Cox was effectively admitting that parts of the 2.4 kernel are very poorly written and OK this part might not be that critical, but I worry because it might be a reflection of the quality of the whole kernel.
Everyone should start helping? (Score:4, Funny)
Re:Everyone should start helping? (Score:3, Funny)
Patches might narrow focus (Score:5, Insightful)
An example of why a particular patch might not be accepted, even though it seems like a "no-brainer", is because it would be for too specific a purpose. It might optimize the kernel for one particular application, at the expense of others. One of the best things about Linux is that it is general-purpose: suitable for everything from palmtops and embedded systems to servers and enterprise applications.
A patch to aggressively cache the disk in memory, for instance, might be good for servers but not for embedded systems. So, I could understand how a patch would be rejected in this case.
As an example, a company I once worked for made many minor changes to the Linux kernel. Since Linux is GPL, I made a webpage publishing these changes, and unlike the company, my webpage is still in existence!
Splash [krellan.com] Open Source Page
These changes would be too narrow in focus to apply to the Linux kernel for everybody, so we never submitted them.
Re:Patches might narrow focus (Score:2, Insightful)
Re: who maintains it? You? (Score:1)
These are no special purpose patches (Score:2)
--
Not just linux (Score:3, Insightful)
It seems to me that much of this could be automated... for each patch which gets added into the xBSD source tree, compare the contexts to the yBSD and zBSD source trees and alert a human if it looks like there's a match.
But for this to be effective, I think that patches would have to have labels attached, since it's really only bug fixes for which this is necessary.
Sounds like something from a big business to me... (Score:5, Interesting)
Re:Sounds like something from a big business to me (Score:4, Interesting)
Yes, fortunately the kerneljanitors project does this. And I think they do it out for altruistic reasons rather than because someone assigned them to.
This is not backwards logic. I am not suggesting you do it. I am suggesting you stop whining about there being noone else to do it when you can't be bothered either.
Re: plz don't make fun of my head brace plz, k? (Score:4, Interesting)
I think the assumption he operates on is that you can't be bothered to go through and submit patches, as you were complaining that someone should to do it.
It's not an issue that people aren't working on the kernel enough, it's that there are too many mad scientists, and not enough henchmen.
Re: plz don't make fun of my head brace plz, k? (Score:1)
:)
Re: plz don't make fun of my head brace plz, k? (Score:1)
Actually, this is almost exactly what I was trying to say.
O, Henry? (Score:2, Offtopic)
No, there's a short article posted on the Gentoo Linux site. A ``short story'' is a form of fiction [randomhouse.com]. (Not that anyone at Slashdot cares, but some of us can't help tilting at windmills.)
--Jim
Re:O, Henry? (Score:2)
Re:O, Henry? (Score:1)
1. An account or recital of an event or a series of events, either true or fictitious
Issue (Score:5, Informative)
Careful.
one possible explanation... (Score:4, Informative)
another possible explanation... (Score:1)
Another possible explanation is that there are a small group of whiny people who are tired of Linus controlling the linux development process just because he happens to have invented it in the first place, and are waging a propaganda campaign to replace him with a committee that will rubber-stamp every ill-concieved patch submitted from anywhere on the internet, with little or no review. Goodness knows, any time any random undergrad or script kiddie changes a few lines of code in the linux kernel, it must be an incredible improvement that we cannot live without. What does Linus know about kernel development, anyway?
Re:another possible explanation... (Score:1)
And then, people in black helicopters, payed by the Melissa and William Gates Foundation continuosly harass Andrea, Alan and Linus, so the Kernel work gets disturbed.
And there's a special comission of the CIA that manipulates the coffee beans used by those people, so they are decafeinated cofee beans, agravatting the situation.
Nevertheless, RMS is behind the situation. He is orchestrating an alternative-media based advertisement campaing that will reveal the isze of Bill's and Steve's undergearments. This will surely make MS developers stop fixin bugs and go back to develop new features and buffer overflows in Microsoft applications.
Also, a team of mutant ninja turtles ha been seen with spray cans on Redmond, WA, trying to change (defacement) all the advertisements from ".Net" ot ".Slash".
Goodness only knows who will win this battle. But rest assured, it will show up on the Season Finale of X-Files.
Re:Not only kernel phenomenon! (Score:2)
Re:Not only kernel phenomenon! (Score:1)
I figure since theres been so much speculation, its probably better to ask somebody whos likely to have done both.
Automate it with Visual Sourcesafe (Score:3, Interesting)
Visual Sourcesafe has the ability to merge back changes automatically in branch B from branch A when they have the same parent.
Say, you have the kernel v2.4.10. You branch off another project from it, call it v2.5.0. When you fix a bug in 2.4.11, you can merge it back into 2.5.0 without a hassle, it can be automated or you can do a visible merge when there are conflicts. The other way around also does work. So you can do this even further: branch of a prerelease 2.4.11-pre branch and a 2.4.11 branche from the 2.4.10 branch. Create fixes in 2.4.11-pre, merge them back into 2.4.11 after testing and when you're done, release 2.4.11 and get rid of 2.4.11-pre.
This is inside a versioncontrol system, you don't have to hassle around with a lot of files you have to merge by hand which will increase the risk for errors.
Of course, Visual Sourcesafe is just 'a' tool, you could use another which has the same functionality and is perhaps Open Source (I don't know of any but I'm sure others will). Doing this job by hand TODAY is erm... not understanding why we have computers in the first place. That's right: to serve mankind.
Re:Automate it with Visual Sourcesafe (Score:2, Insightful)
We pushed and pushed for a change (2/3's of us wanted vanilla cvs over VS!) but management would never listen. And in fact we could not do any remote development with VS as it was not TCP aware... it only worked across MS networks (netbios). We later found another product that integrates TCP support into VS for you. But that added another point of failure for our remote developers (across the country). And those of us that preferred a unix workstation where SOL.
Basically we never used any features that make VS compelling over cvs. And its lack of support for anything but Netbios is unexcusable (especially for java developers who need that cross platform support). The parent poster has probably never used another version control system, and is just pushing MS products.
Re:Automate it with Visual Sourcesafe (Score:1)
No, actually, he's just using it as an example:
I use Visual Sourcesafe here as an example, but any tool with the same functionality described below will do
Exactly :) (Score:2)
Re:Automate it with Visual Sourcesafe (Score:1)
you should check out SOS [sourcegear.com] if you're still using VSS.
Re:Automate it with Visual Sourcesafe (Score:1)
If you have a choice to use VSS - just say no.
Re:Automate it with Visual Sourcesafe (Score:3, Informative)
I'll agree with the versioning/branching comment, although I'd say that ClearCASE, cvs, pretty much anything would be more stable than VSS. Also, VSS doesn't make it nearly as easy to branch as ClearCASE - in VSS you seem to have to branch the whole project, in ClearCASE you can branch individual files and directories, so you never have to merge more than you need to.
Unfortunately, the Linux kernel configuration management paradigm seems to be more of developers maintaining separate trees, and then handing off patches between trees instead of patches that move between branches. I think this is because for a branching scheme like ClearCASE, you need a centralized authoritative repository to say who has branched from where, and when. Linux has no central branch directory like that, and the patch format commonly used doesn't encode this sort of information. So you can't do automatic conflict resolution (or at least you can't do as much as you'd like) without a branch directory under central control.
Branches make sense to me - I use them every day. But Linux, at the moment, isn't set up to use them very well. And in moving to bitkeeper they're going even farther down the path of handling trees rather than branches.
Vendor patches (Score:4, Informative)
Re:Vendor patches (Score:1, Insightful)
Mod this AC up: Re:Vendor patches (Score:1)
Exactly.
Mod this A.C. up!
Re:Mod this AC up: Re:Vendor patches (Score:1)
Re:GPL fault? (Score:1)
This hardly applies to just the kernel! (Score:3, Insightful)
I'm not sure how this could ever be considered a good thing, as the project authors must spend time searching through distribution source releases looking for patches, which takes time. The distributions must continually apply their patch to a changing source tree (and I'm sure it'll eventually break and need reworking), so they loose time as well. This is one case where communication really could be a very positive thing.
sigh... It's about time I went to search for patches again...
Re:This hardly applies to just the kernel! (Score:2)
Re:This hardly applies to just the kernel! (Score:2)
Re:This hardly applies to just the kernel! (Score:2)
Re:This hardly applies to just the kernel! (Score:2)
On a side note, the redhat bug database really needs a way for me to be able to say "send me mail for any problem from package X". Sure, you can subscribe to a particular bug, but I need to subscribe to an entire package. Last I checked, this isn't possible. It would certainly be an easier way to help keep package authors in sync with the distribution packagers.
If Redhat and Debian had a fight ... (Score:2)
And if one wants to ensure that one is running the most stable, but well-patched system
Then who has it - Redhat, Debian, Mandrake, etc.?
Or is this even a fair comparison? And should one make this comparison when planning a Linux install?
Shameless Gentoo Plug (Score:1)
Re:Shameless Gentoo Plug (Score:1)
In simple terms : Gentoo rocks.
The package system is amazing. I had apt-get like features, FreeBSD ports like features, its compiled for YOUR system. (there is a binary distro in the works) and most of all, the people in #gentoo arent the elitist bastards you find in #freebsd and #debian. You ask for help, you get help.
I also try to support the project in any way I can. I cant code. I dont have bandwidth to give. I donate the only way I can, which is to send money or if I have spare computer parts which can be used by the Gentoo team, I donate those as well. And I try to tell as many people as I can about Gentoo.
dvNull
What about vendor branches in BitKeeper? (Score:2, Informative)
It seems like it would be trivial for vendors to maintain their patches in their own BitKeeper repository. If done consistently across vendors, it would allow the kernel maintainers to merge patches into the standard distribution with minimal effort.
Moreover, this would probably make it easier for anybody to track different sets of patches. Imagine being able to use an SCM tool to help minimize the pain of tracking patches through several kernel revs. Many of us do this on a daily basis anyways and would love to see such tools used properly in the open source community.
This would never happen in *BSD... (Score:3, Informative)
People talk about the exchange of ideas between the BSDs and Linux, and I think that a core group like FreeBSD's would be a great idea for the Linux world.
It seems like we are running into more and more scaling issues with the people behind Linux than with Linux itself. This is no fault of theirs. Linux is too big a project for a "the buck stops here" kind of person like Linus.
Obviously, Linux is Linus's brainchild, and he can do whatever he likes with it (yes, I know the GPL allows forking, but think of how a kernel fork would be recieved on
I don't believe that Linux can attain the kind of consistency (and that is not the goal anyway) of FreeBSD or NetBSD, I think they might be able to fix some of the kernel patching and architecting problems if an elected core team could work on this.
-Peter
Re:This would never happen in *BSD... (Score:2)
There has been already reports of bugfix in one BSD distribution which hasn't been reported in another distribution.
So the "this would never happen in *BSD" is just wrong, and presumptuous.
Re:you get what you pay for ! (Score:2, Informative)
Then, what is http://kerneljanitors.org [kerneljanitors.org], mentionned at the end of the article?
Re:you get what you pay for ! (Score:1)
Interestingly, however, much of this stuff can be achieved with LCLint [virginia.edu], which if I recall correctly has a mechanism which can be used to extend it to statically check almost anything. Now that sounds like a good idea (tip: if running lclint for the first time on a program, use the -weak command-line option!)
Patches? I don't need no stinkin' patches! (Score:1, Troll)
The only thing I've seen that was more patched than Solaris were some of Novell's products.
I'll stick with Linux, thanks.
Re:What do you expect without revision control? (Score:2, Flamebait)
What kind of quality assurance is that
It isn't. Read the licence. It's called the GNU GPL. I'm surprised you haven't come across it before. There is no "quality assurance" If you need someone to hold your hand, or you are using Linux for production purposes, go and talk to Redhat or another distributor who will provide the quality assurance you seem to want. They test all their kernels with test suites and simulated production workloads.
QA (Score:2)
It isn't. Read the licence. It's called the GNU GPL. I'm surprised you haven't come across it before. There is no "quality assurance" If you need someone to hold your hand, or you are using Linux for production purposes, go and talk to Redhat or another distributor who will provide the quality assurance you seem to want. They test all their kernels with test suites and simulated production workloads.
Open source software is no excuse for a lack of QA. There are people who want to use Linux for serious work, there should be a -stable that is truly stable. That's the point of concurrently having a 2.4 and a 2.5 release. Red Hat's job is to create a distro from various components and test them together. They can't be expected to transform an unstable kernel into a stable one. Their job is hard enough as it is (keeping track of hunderds of packages is not easy).
Of course there is always a compromise between stability and (fast) progress, the *BSD's are far more focussed on stability. IMHO Linux is focussing far too much on progress, creating great instability. The newest developements belong in the unstable releases for the adventurous to use and test (new VM's for instance). Kernels in the 2.4.x tree should be well-tested before they are released into the open.
Re:For anyone running at 1024x768 or lower.. (Score:3, Funny)