Bug Hunting Open-Source vs. Proprietary Software
Posted by
Zonk
on Sat Oct 07, 2006 12:41 PM
from the i-am-a-big-fan-of-quality dept.
from the i-am-a-big-fan-of-quality dept.
PreacherTom writes "An analysis comparing the top 50 open-source software projects to proprietary software from over 100 different companies was conducted by Coverity, working in conjunction with the Department of Homeland Security and Stanford University. The study found that no open source project had fewer software defects than proprietary code. In fact, the analysis demonstrated that proprietary code is, on average, more than five times less buggy. On the other hand, the open-source software was found to be of greater average overall quality. Not surprisingly, dissenting opinions already exist, claiming Coverity's scope was inappropriate to their conclusions."
This discussion has been archived.
No new comments can be posted.
Bug Hunting Open-Source vs. Proprietary Software
|
Log In/Create an Account
| Top
| 244 comments
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
not to mention... (Score:2)
Number of Bugs vs Bug types (Score:5, Insightful)
(Last Journal: Saturday October 27, @10:19AM)
The summary just says all bugs, which is not fair if the proprietary has 5 times the number of critical or super-critical bugs.
Even worse. (Score:5, Insightful)
He refuses to accept that different projects have different requirements. When the project results in people dying if it fails, you spend a LOT more money and time finding all the "bugs".
When the worst that happens is that you don't see a web page, your money/time requirements are not so high.
Even so, from his finding, Open Source is, on average, better than the closed source projects (not counting the closed source projects that result in loss-of-life in the event of a failure).
He's an idiot for confusing the different requirements.
Re:Even worse. (Score:5, Insightful)
(http://cs.byuh.edu/~andrew | Last Journal: Friday October 12, @12:12AM)
What this guy is trying to say (besides 'buy my software') is that open source can do better (the title of his article is "...what open-source developers can learn....."). He wants people to use stricter development practices; things like automatic testing, nightly builds, etc.
Furthermore, he is probably right, automatically testing code ala j-unit or cpp-unit is a great idea when you are getting contributions from many different people. If that became common practice in the open-source world, the code quality would improve. He's not saying open-source is bad, he's saying it could get better.
This guy is not an idiot, you just didn't understand his point.
OpenOffice is a bad example. (Score:4, Informative)
(http://youtube.com/watch?v=FCDJ0jhWKno | Last Journal: Tuesday November 14 2006, @01:31PM)
It's too bad because it actually works kinda okay, but it's a real effort to get your hands dirty with.
Blender is also like that... it seems when a codebase has 'gotten around' it tends to pick up the bad habits of all the hands its been through.
MySQL is a bad state because it's really only developed by MySQL AB -- no one else is contributing to it so they have no reason to make it any more maintainable than it is. PostgreSQL, on the other hand, had the luxury of being the fruit of some academic research projects and was rewritten once or twice, so it's a little more maintainable.
Re:Number of Bugs vs Bug types (Score:4, Insightful)
However as others have pointed out they are comparing mission critical software to non mission critical software. What should have been done (as has also been pointed out) is to cluster by usage case or software field. So databases to databases, browsers to browsers, generic office usage to generic office usage, etc.
LetterRip
Re:not to mention... (Score:4, Insightful)
(http://jfctravelclub.com/travelblog/)
Now these days you often get studies claiming that proprietary software is less buggy than free software, but it misses some very significant points, the ones we used to respond to MozillaQuest articles still apply very much to today:
- Free software projects very often have an open bug database so it's easy to see how many open bugs are in a project, most proprietary software doesn't have an open bug database so you have to trust the manufacturer and your own testing
- Not all bugs in open databases are really bugs. Some are requests for enhancement, some are duplicates and some are rants
- In some cases one persons bug may be another persons feature (e.g. if an application does something differently to the platform guidelines, some people may like this alternative behaviour, others will consider it a bug).
- The profit motive - companies have a lot to lose by letting people know about bugs, volunteer led projects tend to want people to know about bugs in the hope someone will help fix them (this is getting a bit blurred now that more and more organisations are making money off free software but the fact still is with proprietary software you can't fix the bugs so they gain nothing by telling you about them)
Sorry if this is redundant, I'm working on call at the moment and was halfway through typing this when I had some work to do!So how did they test the proprietary software? (Score:4, Insightful)
(http://www.pembo13.com/)
Re:So how did they test the proprietary software? (Score:5, Informative)
I don't know if the closed source statistics are online somewhere, but these are the open source statistics.
http://scan.coverity.com/ [coverity.com]
and if you ask me the "Defect Reports / KLOC" is pretty low, and such software would normally be considered "good" software.
What's a bug? (Score:5, Insightful)
Well, what is a bug?
I doubt he'd send me a check if I told him that TeX doesn't have an easily accessible iconic user interface. No, his concept of a bug is a deviation from the specified functionality.
But what if that functionality is wrong or sucks?
Apple does really well at creating functionality that doesn't suck. They suffer from the same problems of deviations from the spec as much as anyone, but they manage to mold their spec around what users want. Microsoft, to some extent, does the same and they release products that conform to what users want (generally) because they change the spec as necessary when customers demand change.
If you are implementing towards a standard (like most OSS projects with any traction are wont to do), then you are necessarily restricted by what that spec says. If the spec says to do something inane, the standard-follower must implement it that way.
I don't really have a point here except to say that unless they say "this is what we mean by bug", there can be no way to really examine their results.
Re:What's a bug? (Score:4, Informative)
(http://www.ajwm.net/amayer/)
I think you're conflating two things. The check was (is?) for $50 or some such. The version number of the software is pi (or e) to whatever number of decimals, where each subsequent release adds a decimal place (becomes a closer approximation to the real thing.)
No, his concept of a bug is a deviation from the specified functionality.
That's the only reasonable definition of a bug in the software.
But what if that functionality is wrong or sucks?
Then that's a bug in the specification or in the requirements. I spent the better part of six months debugging the requirements on a major project once. Part of that was getting mutual agreement from three major customers, part of that was resolving internal inconsistencies in the requirements document, and part of that was a high level design process in parallel, to be sure we had a chance of actually satisfying the requirements.
Of course the end user (especially of off-the-shelf software) generally doesn't differentiate between a bug in the software vs a bug in the specification or requirements. The end user generally never sees the spec, and only has a vague idea of the requirements. (Sometimes worse than vague -- how many people do you know who use a spreadsheet for a database?)
(And to BadAnalogyGuy -- I'm not disagreeing, just amplifying.)
A bug can be many things (Score:4, Informative)
(Last Journal: Friday August 24, @03:21AM)
- This is the typical meaning behind the word "bug".
- That's where your TeX complaint would fall under. It's "by design" that it doesn't have an iconic user interface, but that doesn't mean it's something that shouldn't be addressed ever
- This is actually a result of the bug tracking system that we use. Rather than sending e-mail, which often gets lost, we often track work items as bugs. For example, "Need to turn off switch X on the test server when we get to milestone Y"
To further complicate things, there is a severity and priority attached to every bug. Severity is a measure of the impact the bug has on the customer/end-product. It can range from 1 (Bug crashes system) to 4 (Just a typo). Priority is a measure of the importance of the bug. It ranges from 0 (Bug blocks team from doing any further work, must fix now), to 3 (Trivial bug, fix if there is time). (I don't know why the ranges don't match, BTW, seems silly to me)
As anyone who works on large-scale project probably knows, there are always a wide range of bugs, across all the pri/sev levels. To me, a simple count of all the bugs isn't terribly useful. A project could have a ton of bugs, but most of them being DCRs (which are knowingly going to be postponed till the next release) and/or low pri/sev bugs. Or maybe it's the beginning of the project and they're all known work items. Or a project could have only a few bugs, but with all of them being critical pri/sev ones.
So, whenever I see a report that simply talks about bug count, I take it with a huge grain of salt. If I had to guess (I skimmed the article), it seems like OSS projects have far more bugs, but perhaps lower pri/sev since the product itself has been evaluated as being higher quality. In the end, it's the quality that the customer really cares about.
Not quite (Score:5, Interesting)
(http://www-cdf.fnal.gov/ | Last Journal: Wednesday June 13, @11:39AM)
How much is it really true? (Score:1)
(http://sandeeppro.blogspot.com/)
Re:How much is it really true? (Score:4, Informative)
(http://www.oomentor.de/ | Last Journal: Tuesday November 22 2005, @10:48AM)
No they are not to the extend of a experienced developt.
going through the code dow not find bugs. Either you do a formal correct approach, that is a walk through or a code inspection then you may find bugs, or you only have the chance to find occasional off by one errors in a loop or array index. Just by looking over code as you say in your n00b appoach you only find suspicious pieces of code.
What now? You change it to be less suspicious? And then? You commit it? So you don't know if somethign elsewhere is breaking now because of your change? Ah
Testing means to DEFINE how individual pieces of code should behave and writing a test case exactly for that. Changing software and fixing bugs means to have tests, lots of tests, not eyeballs.
angel'o'sphere
P.S. that does not mean that formal walk throughs / inspections don't work, they do!! But informal ones are only for educational purpose intersting.
old rant? (Score:2)
Isn't this an old rant? Sorry if I come out as a troll!
Horrible Comparisions (Score:5, Funny)
"Deanna Asks A Ninja: What is the circumference of a moose?!"
"It's michael pailum with his face in a pie times douglas adams squared."
This answer makes as much sense as the article.
Except "Ask A Ninja" made more sense. And was more accurate. And more entertaining.
Can I just get a Ninja hit out on this guy something so these articles will not make it slashdot anymore?
Exactly what constitutes a software bug? (Score:2, Insightful)
(http://www.myspace.com/dapsychous)
Somebody please explain to me exactly what kind of software bug can be found by automatic scanning that isn't found by standard debugging and compile-time checks. If a computer can ascertain exactly what the programmer intended to do, why do we need programmers?
The simple answer to this is that they can't. That's the point behind hiring human codeslingers to write applications. Considering that most software bugs are logic bugs (off by one, etc) that can't be directly seen in the code without actually, you know, RUNNING the program, I find it difficult to believe that AI has come to the point where it can guess the coder's intentions and infer the purpose of an application.
Additionally, where exactly did this proprietary code come from? I don't know which companies these programs were released by, but other than the SharedSource initiative which mostly caters to programming students (if they still even have this program), Microsoft would call their next o/s Windows Sh*tburger (I think I'll trademark that) before they would release their code, especially to a third party consulting firm.
I find this article highly suspect
just an example of how "buggy" OSS software. (Score:3)
linux 2.6: 3,315,274 lines of code, 0.138 / 1000 lines of code.
kde: 4,518,450 lines of code, 0.012 bugs / 1000 lines of code.
based on this I would say we are doing pretty good with open source.
but we shouldn't forget that this tool only scans coding errors, not coding logic.
wine for example only has 0.112 / 1000 lines of code as well.
and we all know it by far doesn't always do what we want it to do.
Re:just an example of how "buggy" OSS software. (Score:4, Funny)
> and we all know it by far doesn't always do what we want it to do.
Well duh! It is an implementation of the Windows API. And when considering how often the WinAPI does what you want, I think they have made a perfect copy.
Re:just an example of how "buggy" OSS software. (Score:4, Informative)
(http://www.offworldpress.com/ | Last Journal: Thursday October 31 2002, @12:57PM)
linux 2.6: 3,315,274 lines of code, 0.138 / 1000 lines of code.
kde: 4,518,450 lines of code, 0.012 bugs / 1000 lines of code.
So far so good! But for contrast, I'll add this stat from TFChart:
Gnome: 31,596 lines of code, 1.931 bugs / 1000 lines of code.
Eeeep!!
(No wonder I prefer KDE
my open-source project was scanned by Coverty... (Score:4, Interesting)
Rather than brag (I won't say who I am or the name of my project), I'm just going to sit back and read all the defensive flames from self-appointed "security experts" whose open-source project didn't do so well. After all the flames from these "security experts" that I've endured, I'm going to enjoy watching them squirm.
It's karma.
Why is this surprising? (Score:3, Insightful)
(http://hallert.net/)
In the end, it comes down to black box vs. white box testing. Commercial software has a strong QA engineering component. Open Source software relies primarily on a black box testing approach.
Open source has MANY benefits and MANY advantages over commercial software. This just doesn't happen to be one of them, but unlike the commercial software, the bug fix cycle on open sourced stuff can be a LOT quicker, so it evens out in the end.
Re:Why is this surprising? (Score:5, Informative)
(http://lucernesys.com/)
Says who? QA and testing covers the entire gamut, from formalized unit-testing at every level, to 'throw it at the beta testers and hope nothing breaks'. it's got nothing to do with 'proprietary' (not 'propriety') vs open source.
Where on Earth did you get that? Are you completely oblivious to all the testing methodologies and systems developed by the open source community? Here's a few for you to research: JUnit, Test::Unit, and Selenium.
Again with the generalizations! Commercial software development is, by definition, proprietary, so you don't know how they do it! They might tell you they have a 'strong QA engineering component' (whatever that means) but they could be full of shit!
Misquoting TFA (Score:5, Informative)
(http://members.virtualtourist.com/m/51ebe/ | Last Journal: Monday August 20, @09:15PM)
TFA says that no open source project is as good as the BEST of proprietary, but it also says that the AVERAGE open source is better than the AVERAGE proprietary.
Not quite... (Score:5, Insightful)
(Last Journal: Friday June 15, @08:57PM)
No, *popular* open-source software is 5x as buggy as *safety-critical* closed software. The linked dissenting opinion [fortytwo.ch] is at least partly right; they're comparing apples to oranges.
Maybe they should try comparing open- and closed-source software that's actually trying to solve the same problem? That'd be a bit more valid of a comparison...
And even more dissenting opinions (Score:2)
(http://www.bash-shell.net/)
Actually (Score:2, Informative)
It was not an apples to apples comparison, more like apples to diamonds. Dom't worry, just fix any real problems identified. Many of the bugs found are theoretical, not real. Many others are style questions. the experts will probably never quit arguing about what is 'good' and what is 'bad'.
There are also some very real race conditions, memory leaks, and things like that. A real list by line number would be nice.
That said, what would really matter is a comparison program by program. I don't think my quick and dirty one off is really in the same class as the Kernal, or Firefox,I havn't seen how this diferentiates.
Open or Closed ? (Score:3, Insightful)
Open-source software is expensive if you want a commercial support contract (because you are asking a professional to spend a lot of time learning).
Closed-source software doesn't have the function that you want, and you cannot fix it to add the funcion that you want.
You pays your money and you takes your choice. You can always stick to pencil-and-paper, and not use this 'software' stuff at all, if you prefer.
It was about mision-critical software (Score:5, Interesting)
Basically, my own conclusion from reading the article was that it IS possible to write excellent software with very few bugs, if that is a top priority. And, that the author seems to say that while mission-critical software (which happens to be proprietary) is fortunately much better than the rest, among all that other non-mission-critical software, open source tends to be better than proprietary.
Not surprising, and quite encouraging...
Strange comparison (Score:1)
(http://www.kirun.co.uk/ | Last Journal: Saturday November 29 2003, @11:55AM)
It also notes that average open-source beats average closed-source.
So, it tries to ask the question: "How do we make open-source stuff as good as aerospace quality?", but comes across as being more controversial than it needs to be.
meaningless, no data, and probably biased (Score:5, Interesting)
Furthermore, Coverity simply cannot accomplish what they claim to accomplish: there is no way of detecting "bugs" automatically--if there were, compilers would already be doing it. Coverity effectively does little more than compare code against a set of internal coding conventions; that can be useful if it's done right, but it's not a measure of code quality. Some completely correct code will score thousands of violations against their tool, while other code may contain thousands of bugs, none of which register. Furthermore, it is likely that a lot of their customers are Windows based and that Coverity is biased towards Windows-based coding conventions, giving more false positives on non-Windows code. Before publishing such comparisons, Coverity first would need to demonstrate that their tool does not contain such biases.
Finally, and perhaps most importantly, the company isn't publishing its data, so nobody can verify or even evaluate their claims. Not only do they fail to publish their raw data (obviously, they can't do that for proprietary software), they also fail to list their summary statistics by vendor and project (which they could, but obviously won't do). They don't even give a summary statistic by class of application, class of organization, and code size. Their results are meaningless because they're not reproducible.
These numbers tell you nothing about FOSS code quality relative to commercial code quality. What they tell you is that Coverity apparently doesn't know how to do statistics, misrepresents what their product can do, and doesn't know how to report experimental results properly. Now, do you want to put your trust in such a company?
Nice way to generate publicity (Score:3, Insightful)
Now to compare every open source software application to aerospace software is really comparing apples to oranges. There is a big difference in the expected quality between an editor and an aerospace application. It's alright even if my editor crashes once in every 20 times I invoke it. Is that acceptable with an aeroplane?
I'm sure the folks at Coverity understand all this. But if they really speak what is right, they will not get all the eyeballs and publicity. In classic slashdot lingo:
1. Do something (anything) that involves open source and proprietary software
2. Make claims that sound outrageous / controversial
3. Profit! (with all the free publicity)
Proprietary vs Open Source (Score:1)
(http://erroraccessdenied.com/)
misleading summary (Score:2)
(Last Journal: Saturday November 03, @09:51AM)
From the summary:
From the article:
In other words, they're comparing the best proprietary code---in this case some aerospace controller program---with the best open-source code. Of course, we would expect the controller program to be more closely scrutinized. There hasn't been life-and-death deployment of open source software nor a need for it.
On the other hand, the article doesn't clarify whether the top quality proprietary software can be purchased by a consumer. I'm guessing these programs are generally not available to personal computer users because of their highly specialized nature (e.g., running a nuclear power plant or jet turbine engine).
Lies, damned lies, and statistics (Score:5, Insightful)
Maybe I'm Misreading this (Score:2)
Now, if Open source is better at finding more subtle errors, even if it fixes them, doesn't his methodology penalize OS code against proprietary code where they didn't find and correct the error in the first place?
Pug
This is basically nuts (Score:3, Insightful)
What they have not done is compare comperable projects -- IE to Firefox, Open Office to MS Office, Windows to OS-X to Linux-KDE. There is, as far as I know, no Open Source software product that is really intended for mission critical applications -- I guess maybe SSH might qualify, but I don't see it in their list.
So, I think what we have here is a comparison of Apples to Turnips using a dubiously calibrated error-o-graph machine that uses an unknown technology to perform undefined tests on software.
Don't get me wrong. I sure as hell wouldn't run a nuclear power plant with Linux-X-Windows-whatever. Nor with Windows -- neither Windows 9 nor NT based Windows. They don't meet my admittedly subjective standards of quality either. But if we waited for near perfect software quality, we'd still be trying to get text mode right. Personally, I 'd vote for that because I think building major structures on weak foundations will likely lead to big trouble a decade or three out, but I think I'd lose that vote about 93 to 1 with maybe 6 abstensions.
Talk about non-sequitur... (Score:1)
If the author of this article knew that, then the article would probably not exist at all. So, the tag [FUD] fits perfectly.
Quality of Programmers is critical... (Score:3, Interesting)
Which would you rather have 100 monkeys programming on a project or 10 skilled programmers?
More programmers and more 'eyes' on a project does not mean it is going to be inherently more bug free. In fact with a group of bad programmers in the mix, it can cause severe harm to a project.
I'm not knocking Open Source, but for people to just expect it to be better because more people have access to work on it, have obviously not met as many programmers as I have.
There are a lot programmers that put time into project (and yes Open Source) that have no business developing a VB application for a 10yr old kiddie game, yet they are taking part in large scale coding projects that truly would be better off with them not working on it.
When working with XWindows years ago, I ran into a few people that scared the hell out of me and other people. They had no vision or scope past the specific things they were trying to do, and would often come up with modifications or 'features' that would break more than it added to the project.
In the Windows world of 3rd Party developers I have also found 100s of people I wouldn't want them to develop Hello World. As they had no concept or regard to security, Unicode or many other features that would fail when the applications would run on a non-English system with a user having administration privledges.
You can even find many commericial products in the 3rd party Windows world that also have these problems, but are yet produced by big companies are popular products.
I wish that all ideas would be welcomed into a project, but the people having the final say could trump crap programmers and crap ideas if they are detrimental to the project.
When you look at the Linux Kernel or BSD, you can quickly understand why Linus and others don't want to let the 'deciding' control into the masses, or both of these core OS would become crap in a matter of months of unregulated programmer additions.
Why all of the complaints? (Score:1)
What is the reason for all of the complaints? So what if comparing open source to mission critical proprietary is an apples to oranges comparisons. When attempting to better yourself to become the best possible you don't compare yourself against your peers - you compare yourself against your betters. It may be the case the OS(open source) can't use the same methods as mission critical PS(proprietary software), but it at least known that there a better defect rate is actually achievable in reality and is something to strive for.
Typical PS advocates could also make arguments for an apples to oranges comparison - we have constraints placed on us by customers and managers, we have to be concerned about making a profit, we have to beat others to market and so on (and yes OS advocates will make arguments against this - the point is everybody can make arguments about why a comparison is unfair - but a difference in quality exists regardless of fairness of comparison). In the end, it doesn't matter - typical OS is better and the typical PS needs to improve, just at OS needs to improve compared to mission critical PS.
What this article really enforces is the idea that no model is perfect (which should have been obvious to everyone but apparently isn't) and all of the models have room for improvement and the people using these different models should learn from each other.
wheres the scanners source? (Score:1)
How can anyone be sure of how reliable the study is without reading the program that the entire article is based on, if the security scanner is flawed then the article i pointless.
Would you believe someone who told you he had built a device which could measure something that noone new existed, but refused to allow you to see his device, I sure wouldn't.
As many people have said, the scanner can not detect ALL faults.
The question is How was it taught to find faults (maybe taught is a bad word, maybe it should be 'designed').
If it looks for common programming errors how did they determine what was a common programming error?
Asking programmers is a risk, do they always know when they've made a mistake?
The only real way is to look at code where a bug has been found. Proprietery software won't let you do this, Open Source will.
If they built thhe system based on common mistakes in Open Source systems, then tey are more likly to detect mistakes in Open Source code, as open source and proprietry code development techniques differ but may produce differant kinds of bugs.
Did they run it on its own source code? (Score:2)
(http://powerlord.livejournal.com/)
IEC 61508 (Score:1)
Having trouble identifying what your actually supposed to be doing?
Then I suggest you give the Norm Internationale - IEC 61058, a read! It's a favourite of mine...
In general most of the engineers throw their hands up in the air and scream blue murder when I quote this stuff at them... (We're a Medical Device company). But I f you want to get a good idea of whats required..
Cheers,
P.S I'd suggest a quick peruse of part 3 Appendix B
- No Dynamic Objects, Limited use of pointers, limited use of recursion....
It'l' take you a wee while to wade through it... (If you can actually lay your hands on a copy)
Bugs in Open Source Code Are "Better" (Score:2)
Hook it up ... (Score:1)
Ask MS this question, they know the answer. (Score:2)
(http://www.google.com/intl/xx-hacker/ | Last Journal: Thursday August 05 2004, @04:50AM)
However, Debian (which I am running here) and Open Office are not closed source.
Simple logic will tell us that someone or someones in MS are tasked with keeping a close eye on open source and how it compares with MS own operations, hell, this is MS core business strategy, just ask Jobs...
MS aren't likely to tell anyone else these results, but, we can maybe infer them from MS actions.
eg
"Vista sucks" ---> MS ain't worried about Debian / KDE
or
"vista allows you to pause and resume multiple file transfers, and don't abort the whole list on one error" ---> MS is very worried about *nix in general coming to the desktop
one factor missing I'd bet (Score:2)
(Last Journal: Saturday October 07 2006, @07:46PM)
Proprietary software tends to have requirements of what bugs they MUST fix. In this case once the product goes 'live/GA' users can request bugs to be fixed AFTER 'live/GA'. Not just security issues either. We usually do a few maintainence fixes, after GA because of what our users require us to do.
I also think that there is a QA issue in open source. Open source does not usually have a QA department, it has a beta period for people to find bugs. Proprietary software almost ALWAYS has a QA department or QA period where stuff is tested over and over for bugs and then these bugs are fixed.
Maybe open source needs OPEN QA!
And how did they get the data? (Score:1)
Yeah right. Like a vendor making money selling a product is going to be truthful when telling just how crappy his product really is. ROFLMAO
You have to define quality (Score:2)
(http://knoppixquake.webhop.net/)
OK, what is the definition of quality? Most people like to say "free of bugs", but that isn't a true indicator of quality. My working definition is that a high quality product meets or exceeds the client's expectations. Now with OSS, my expectations aren't really that high. I mean, if it crashes, or the UI is kind of clunky, or it is hard to install... I just kind of expect there to be some of that. But if I pay for something, I expect it to be of higher quality in these regards. And if a client hires someone to create software for them, it had better meet their requirements. Of course, that is all pretty much a game nowadays, where promises are made, requirements are poor, and it all just turns into a used car sale. Roll out the dog and pony show, let the smooth talkers "work" the clients to explain why they didn't get what they asked for...
Coverity is involved, so ignore the study (Score:3, Insightful)
(http://foobazco.org/)
Coverity, of course, knows that reports like this will be written up in exactly the way this summary was, clearly associating their company with the idea of enumerating the bugs in a piece of source code. While not illegal, this type of marketing is of course deceptive; while published papers describe the type of defects (or non-defects) actually detected, the overwhelming volume of commentary will reflect the broader, and incorrect, view that Coverity == bug-finder. It would be just as meaningful (which is to say, not very) to publish the number of lint warnings or missed opportunities to qualify pointer arguments with the const keyword, and neither would require an expensive piece of overhyped software.
Just Say No to Coverity's marketing gimmicks.
Why dont i trust the institution that wrote this ? (Score:2)
(http://www.webgeekworld.com/ | Last Journal: Thursday April 27 2006, @07:47AM)
talk me money (Score:1)
open software makes proprietary software buggy (Score:1)
There are code reviews... (Score:2)
(http://www.dangercollie.com/music/)
...and there are code reviews. One of the customers I work at hired EDS to do a code review (not one of my apps). EDS sent five people...none of them actual programmers. They spent the first two days in a conference room figuring out how to approach the job. 80 man-hours later the EDS team reaches the startling conclusion that they're going to need programmer support for the actual code review.
And people say Big 5 consultants aren't worth the money. Pshaah.
uuuuuhhhh (Score:1)
In the end, it is all a matter of work. Much money means much work (usually). So there you go. I just hope that the independence of open-source software leads to a rather prefect end-product (somewhen).
BULL CRAP (Score:2)
(http://www.aesgi.com/)
I will have to check in the mirror again as I might be living in a different universe where my Linux boxes always have to be remotely rebooted on a weekly basis and my Windows machines have 400 days worth of uptime.
-Hack
Why hasn't anyone said this yet? (Score:1)
(http://athingis.boldlygoingnowhere.org/)
Fair comparison? (Score:2)
This type of quality (CMM level 5, I might hope) doesn't happen overnight. It can only be reached by subjecting the whole development process to rigorous methods. Also, someone is hired and appointed as the responsible person for the software in question. Not just any software company is capable of reaching CMM level 5, even if it was Mr Gates himself who founded it (I'm not even sure MS writes software at CMM 2). We're talking about best-of-the-breed software companies here.
In contrast, no warranty is given on software that is released under the GPL. It is in the nature of the GPL that anyone can contribute, but that if you wish to use it, you do so at your own risk.
Which makes me draw the conclusion: Duh- *Of course* open source software has more bugs than mission critical nuclear power plant / jet engine / hospital equipment software.
Whoever said "The 2002 NIST study estimated that $22 billion of the cost of buggy software would vanish with better testing" doesn't realize that there is more to software quality than testing. Software quality only happens it is guaranteed in every step of the development process, from specification to rollout.
Mediocre article, bad summary (Score:2)
Well since there is a very limited number of airplane manufacturer (for a large number of very bad reasons), and since it is quite difficult to have a handy Aibus 380 or Boing Skyliner around for testing, it is clearly not a very "OS Friendly" field of development.
Actually what is really compared is:
Open Source applications
Closed Source applications
and a class of very specific Bespocken applications.
Now the methodology is not very scientific, and therefore the results quite disputable.
But what it might show if anything, is that if there would be more openess in the airplane industry and more cooperation and information sharing, planes would probably be cheaper and safer.
So to make a slightly too fast shortcut: patents are bad not only in the software industry but in practice, everywhere.
Formal methods/safe languages, anyone? (Score:2)
Second, he is not comparing apples to oranges. He is using a metric, bug-rate in a wide sample. Then, he finds, interestingly enough, that proprietary software is very much scattered, but the ones on the top are 5 X less buggy than open source. This begs the question: why? This is what people should be discussing. Not saying he works for Microsoft and all that childish bullshit...
The *real* question is: what are those methods people developing mission-critical software use that open source hackers do not? My hunch is: formal methods, safe languages.
For instance, Ocaml http://www.astree.ens.fr/ [astree.ens.fr] was used in a sofware verification system for the Airbus A340 fly-by-wire system. Haskell is used by Galois Connection http://www.galois.com/ [galois.com] to develop secure protocols for the DoD. And there are many other examples, just look at the clients of vendors of Erlang (well known), Common Lisp, and Eiffel.
As long as the open source community sticks to C (and C++), we're all going to remain in this ridiculous situation that we are in today. In this day and age you can use a fast compiler for safer languages like ML, Lisp or Eiffel, but people insist programming like we're in the 70s.
Re:Smell Microsoft? (Score:1, Insightful)
Me too, as how can this statement even be verified:
This might give a hint on what is up:
The US government politicians control government, including Homeland Security, this we know and we all know of Microsoft soft and hard contributions to politicians.
But the key is in what Coverity is trying to sell, a code analyzer.
I guess we cannot blame Micro$oft (directly) for this, but their results on IIS and Outlook were suspiciously not posted. Or perhaps they didn't even look at Micro$oft? In fact, we don't know what commercial source it was compared to do we?
Another flawed and cooked report for sure. Certainly the "dissenting opinions already exist" link in the original post has more merit.
Re:How do they do this? (Score:1)
Porbably not. More likely green hills (Score:2)
(Last Journal: Friday December 01 2006, @10:51AM)
Re:How many times? (Score:1)
(Last Journal: Wednesday August 14 2002, @12:33PM)