Free Software, a Matter of Life and Death 197
ChiefMonkeyGrinder writes "Software on medical implants is not open to scrutiny by regulatory bodies. Glyn Moody writes: 'Software with the ability to harm as well as help us in the physical world needs to be open to scrutiny to minimise safety issues. Medical devices may be the most extreme manifestation of this, but with the move of embedded software into planes, cars and other large and not-so-large devices with potentially lethal side-effects, the need to inspect software there too becomes increasingly urgent.' A new report 'Killed by Code: Software Transparency in Implantable Medical Devices' from the Software Freedom Law Center points out that, as patients grow more reliant on computerized devices, the dependability of software is a life-or-death issue. 'The need to address software vulnerability is especially pressing for Implantable Medical Devices, which are commonly used by millions of patients to treat chronic heart conditions, epilepsy, diabetes, obesity, and even depression.' Will making the source code free to scrutiny address the issue of faulty devices?"
I've got to say... (Score:5, Funny)
Re: (Score:2, Funny)
... or potential lack thereof when you need it.
Re: (Score:2)
Haha, just think... somewhere out there is someone who is thinking it would be a great idea to run Windows Embedded in a pacemaker.
Re:I've got to say... (Score:5, Funny)
Blue Screen of Death, now with real death?
Re:I've got to say... (Score:5, Funny)
Roy: [answers phone] Hello, IT. Have you tried turning it off and on again?
Re: (Score:3, Informative)
Make sure you leave it off for at least 15 seconds before turning it back on...
Re:I've got to say... (Score:5, Funny)
Thanks!
At least I didn't say it'd be the first killer app for the platform. Man, these jokes write themselves!
Re: (Score:2, Funny)
Re: (Score:2)
Re:I've got to say... (Score:4, Informative)
I wish we could up-vote comments ourselves, I'd give this a ++.
We do. You just have to earn them, that's all. And once you earn them, you can waste them on as many +funny's as you want.
Re:I've got to say... (Score:5, Funny)
SNMP (Score:3, Funny)
"Well, do you want your pacemaker to have intuitive manageability through Group Policies, or not?"
No, just a good SNMP MIB and traps.
I was joking with an ex-sys-admin friend of mine who just underwent open heart valve replacement, and commenting on the idea that the wireless ECG gear needed SNMP to alert when it detected a loose lead. She laughed - not good when you were doing a good impression of an Aztec sacrifice just a couple of days previously.
Re:I've got to say... (Score:5, Funny)
Just think... Somewhere out there is someone who writes pacemaker software who is thinking "There are alternatives to Windows Embedded?"
Same article different day (Score:5, Informative)
Dupe! This was covered a couple of days ago.
Re: (Score:3, Informative)
Re:Same article different day (Score:5, Funny)
Re: (Score:2, Insightful)
That's not specific to software-controlled devices though. If you're dependent on taking a pill every week to keep you alive and/or healthy, you're in trouble if the supply chain gets disrupted in any way.
Re: (Score:3, Interesting)
Running out of pills is one thing.
Having your pace-maker shut down due to non-compliance with an EULA is quite another.
Sure, corporations can make a killing...but it will come with a murder conviction.
I seriously doubt it would EVER be legal to remotely disable a pace-maker.
Re: (Score:2)
"We didn't sell you that pacemaker, we leased it to you. You read the EULA before it was inserted, right?"
Re: (Score:2)
I seriously doubt it would EVER be legal to remotely disable a pace-maker.
Why do you doubt that? "Our licensing is reasonable and non-discriminatory."
Do you also doubt that a pacemaker manufacturer would refuse to provide a critical software update unless each pacemaker user pays them for it?
Re: (Score:3, Insightful)
Do you also doubt that a pacemaker manufacturer would refuse to provide a critical software update unless each pacemaker user pays them for it?
Wait, do you mean before or after I talked to Action 9 news?
Re: (Score:2)
Sure, corporations can make a killing...but it will come with a murder conviction.
I seriously doubt it would EVER be legal to remotely disable a pace-maker.
Kind of like it would be illegal for an insurance company not to cover all prescription medicines, or for a pharm company to ever stop manufacturing a pill?
Re:Same article different day (Score:4, Informative)
Re: (Score:2)
Hmm... so by posting a ilnk to a single instance of a medical device on which testing was not sufficient, the point to which you're drawing attention is that no bug has ever escaped the scrutiny that goes with an open source philosophy?
No significant system can be guaranteed bug free, regardless of the number of eyes on the source code. There are defined standards for safety of medical equipment. Would the overall safety be greater if you added "many eyeballs" to the testing already required? Maybe, mayb
No kidding (Score:2)
And if OSS is so immune to bugs, how come Firefox has so many problems? How come they patch tons of security updates with each release? This isn't just an OSS project, it is a popular, well funded one. And it has bugs left and right.
So, if OSS is so good, what's going on?
Or maybe, just maybe, is the real trick the quality of the review and the methods for writing code, which can be done closed or open?
Re: (Score:2)
And as people pointed out the first time around, medical devices are tested extensively before being deployed.
Not just tested. Safety-of-life systems are usually subject to extensive checks on the development process and in the most significant cases to extensive static analysis. Even if you opened the code up to inspection it would be pretty much useless without all of the design documentation and all the other documentation that explains why the code is sufficiently safe. I don't see how the free software model gives any significant safety advantage; all that would happen is the developers would get bogged down b
Re: (Score:2)
Hell, what if some third-party finds your pacemaker infringes on their patent and demand monthly royalties from you? replacing, say, a pacemaker is a *bit* harder than uninstalling your copy of ffmpeg.
Re: (Score:2)
And as people pointed out the first time around, medical devices are tested extensively before being deployed. I am an ardent free software supporter, but the safety/reliability issue is simply the wrong argument. I would say the more important argument when it comes to medical software is control -- do you really want to have a corporation that you have absolutely no control over to be in control of a device that sustains your very life? What happens if that company goes bankrupt, and the source code dies with the company? What if they decide they want to start charging people a yearly fee for using their pacemakers (a situation that does not seem too far fetched, given what I have seen proprietary software companies do in the past)?
I'd say those are all safety and reliability issues.
Re: (Score:2)
If any of those scenarios are possible they are possible with hardware just as easily as with software.
Re: (Score:3, Interesting)
do you really want to have a corporation that you have absolutely no control over to be in control of a device that sustains your very life?
I'm glad my eye implant [slashdot.org] is mechanical and doesn't have any software.
medical devices are tested extensively before being deployed
I'm damned glad of that, too, but as it was only approved three years before I had the surgery, I do worry that the struts might break twenty or thirty years down the road. OTOH, I wouldn't have wanted to wait until I was 70 to have it; it's an
Re: (Score:3, Insightful)
Do you really want to have a corporation that you have absolutely no control over to be in control of a device that sustains your very life?
It can't be any other way.
The development, testing, and licensing of the device could cost ten million dollars, a hundred million. There is no upper limit - and any company taking over the production and distribution of the device is going to see costs on the same scale.
There simply aren't very many companies with the strength and experience to do that.
Re: (Score:3, Insightful)
And how is that worse having a group of random self appointed individuals, over whom I have absolutely no control, in control of a device that sustains my very life?
From the number of FOSS p
Re:Same article different day (Score:5, Informative)
http://hardware.slashdot.org/story/10/07/22/2346253/SFLC-Wants-To-Avoid-Death-by-Code?art_pos=46 [slashdot.org]
Makes sense (Score:5, Insightful)
To me, this is just common sense. This code doesn't necessarily have to be FL/OSS in my mind - let them keep the copyright, but it most definitely should have code available for public review. Would you be willing to take a new wonderdrug where the drug company won't tell anyone what's actually in it, but assures you that it'll work? If they must disclose the formula to their drugs, then they ought to be required to disclose the code to their software. Let existing laws like copyright ensure that no one else uses it.
Re: (Score:3, Insightful)
Re: (Score:2)
While that presents some difficulty, it's still not that bad. IMHO, that's akin to releasing the code with the comments stripped out. Sure, it makes testing more difficult, but it does ensure that you can reproduce their pill/executable to run tests on yourself.
Re: (Score:2)
Re: (Score:2)
In that case the engineering drawings for a 777 (or anything else) should also be open to public scrutiny. Is that reasonable?
Re: (Score:3, Insightful)
Some level of documentation for such things will be available. How do you think A&P's all over the country work on them? Just pop the hood and figure it out as they go along?
And yes, I think that anything on which the safety of a life depends should be open to scrutiny. Alarm clocks and keyboards? Not so much.
Re: (Score:3, Interesting)
The cheap Chinese copies made from the "open to scrutiny" plans are probably not going to be as safe as the Boeing originals.
Re: (Score:3, Insightful)
It's the same argument that an automobile manufacturer doesn't release the detailed specs of a vehicle, because the owners manual doesn't show a breakdown of the engine. They are available (for a price, of course) to the people that need the information.
Here's the list of manuals for a Boeing 777 [boeing.com].
But for both aircraft and auto manufacturers, I don't believe they release detailed specs of say the software that makes their vehicles work. I doubt A&P mechanic
Re: (Score:2)
Mechanics also don't fix the cruise control hardware. They throw out the unit and buy a new one. Ditto with pacemakers and 777s. If the part doesn't work, you get a new one. The provided information usually only describes the system down to the level of replaceable components, not below. The software in each of those replaceable components is part of the black box.
Did your computer come with the circuit diagram for the motherboard (the Apple II did)? Suppose something goes wrong and you need to fix it
Re: (Score:3, Interesting)
I almost totally agree. Most parts are made to be replaced as a module. When's the last time you replaced the bearings in a hard drive, or soldered a pin on a memory stick? But, sometimes we do, and that's without the assistance of technical schematics.
A few months ago, I was given a hard drive to fix. The machine had been hit by a power surge, and a chip on the control board (the one on the bottom of the hard drive) melted. I replaced the board, and the drive started work
Re: (Score:2)
I very much doubt the full engineering documentation is available, and it is almost never available publicly. Many critical systems manufacturers will already make some parts of the software specifications, testing methodology, troubleshooting guides, etc. available to people authorized to make repairs or do troubleshooting.
Re: (Score:2)
Re: (Score:2)
Maybe but this problem has already been solved.
The aerospace industry has been flying code that can kill for decades. They have a procedures to develop and test mission critical code for everything from navigation to flight control systems.
Is it prefect? No but the system does seem to work well. Just base the certification process for medical software on the certification process for aerospace software and you have a good working solution.
Re: (Score:2)
What you're describing is patents on software.
Re: (Score:2)
No, I said copyrights. Patents are a completely different animal. I personally see no need to protect the manufacturer's algorithm on detection of a particular trait, even if it could be reimplemented in different code. And yes, I feel that that algorithm should be publicly reviewable if someone's life depends on it.
Re: (Score:3, Informative)
This code doesn't necessarily have to be FL/OSS in my mind - let them keep the copyright
Authors of open source software retain their copyright.
Re: (Score:2)
True. Poor choice of wording. Authors of open source code keep their copyright, but it really only applies to the opportunity to close the code again. Any version that has been legitimately released under the GPL and makes it into the wild can't be retracted. Others could still re-release the code again with modifications. That ability doesn't need to be there at in in this case. The point here isn't to promote or enable derivative works, but rather to ensure safety and security of the code.
Re: (Score:2)
This is what the FDA (in the USA) is for. There are similar organizations in other countries.
I do quality assurance for a medical device manufacturer. Anybody who has dealt with medical devices and the FDA knows that you don't mess with the FDA. Period. They take public safety very, very seriously.
Opening code to the public is not necessarily the best way to do this. I can't say it's definitively a BAD thing, but I can see it causing more issues than it's worth. If everybody knows how the device works
Not a question of badly written software (Score:2)
Re: (Score:2)
Re: (Score:2)
This is exactly why the code should not be 'open'. You just wind up with hysterics like yours, and very little in the way of actually useful information. Did it occur to you that security may be way down on the list of concerns when making such a device? Actual real-world things are somewhat more important. Like battery life. Trying to maintain communications under all circumstances. Heat generated. Reliability.
Yes, theoretically someone could kill your friends by hacking their insulin pumps. Or, th
Re: (Score:2)
Re: (Score:2)
Most of the other ways that someone could kill another person leave significant evidence as to who committed the crime. Right now, there is ver
Re: (Score:2)
The lack of security is not a flaw, it is a design choice. The company (and regulators) know very well that the device is insecure. What could possibly be gained by exposing that situation to the world?
Secondly, you can't magically add security and not affect one of the other things. Encryption is compute-intensive, which means more heat, less battery life, and more complexity. Now, maybe if new longer life batteries come along you could add security and not sacrifice battery life. Or you could say 'wh
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
What would be the point of putting an intentional back door or kill switch in a medical device?
Once you have reached your level of paranoia you should know that just because you are looking at SOME source code does not mean it exactly matches the binary in your device. There could always be something in the tool chain that inserts backdoors and kill switches independent of the source code. So unless you are planning on building your own tool chain from scratch, inspecting, compiling, and actually installi
Open - not necessarily free (Score:2)
and maybe free as in reusable to improve it,
but not necessarily free as in beer!
People will die (Score:2)
Re: (Score:2)
Medical software should be libre, there is just no arguing that. When it comes to software that sustains a person's life, no single corporation or entity of any sort should
Re: (Score:2)
Their software makes you 'uncomfortable'? Well, don't use it - that should ease your discomfort. OK, you may be dead, but hey, at least those greedy bastards aren't getting any of YOUR money.
Re: (Score:2)
Personally, I would not be very comfortable if a company I have absolutely no control over has control over the software that runs the pacemaker in my chest.
If you needed one, you'd be a damn site less comfortable without the pacemaker. Briefly.
What if they decide to start charging a yearly fee for their pacemaker software?
How would they enforce it? The things are not connected to the internet, you know.
What if they refuse to provide critical software updates unless I fork over more money to them?
Er -- do you actually know what a pacemaker is? What would comprise a "critical software update"? The whole thing gets replaced every few years, anyway, because of battery life.
Re: (Score:2)
People will die from shortcomings of technology, whether the tech is FOSS or not.
And since we're talking about medical implants here, even more people will die without the technology. That's no reason not to try to improve the safety of the implants, but we need to keep it in perspective.
Re: (Score:2)
Crowd sourcing your Quality Assurance department? (Score:3, Interesting)
Re: (Score:2)
From what I've seen, most FOSS projects seem to skip the QA step unless it's a dual licensed product with a company behind it.
Re: (Score:2)
Operator Error (Score:3, Insightful)
Double-edged sword (Score:3, Interesting)
Making the code available for scrutiny is a double-edged sword. Sure, it might help to catch critical bugs in the software faster. But with some devices, it really raises a host of new issues. Some pacemakers out there are configurable wirelessly once they are in the patient's body. This is actually a very critical feature. But do you want to risk everyone being able to reverse-engineer the protocol used for adjusting the settings for such a device? A wrongly configured pacemaker can be deadly. Right now these things are fairly secure because they are rather rare and not interesting enough as targets for hacking.
Besides, proving that some piece of code works as intended (or at least fails gracefully in all circumstances) in an essentially uncontrolled environment is quite a feat. Embedded equipment is hard to service, has to have a longer hardware lifespan than normal hardware, is often custom designed for a single application and thus may have subtle hardware defects not reproducible on similar test systems... oh, and who says that the compiler or even the CPU is bug free? This is all common knowledge around here, but I just want to give the full list. What this means is that just opening the source code might not cut it. The validation would have to be performed on the hardware it was designed to run. So, where's the call to open up the hardware design and documentation to public scrutiny as well?
Re:Double-edged sword (Score:5, Insightful)
But do you want to risk everyone being able to reverse-engineer the protocol used for adjusting the settings for such a device?
Yes. Security through obscurity is essentially no security at all. The only thing that should be secret is the private encryption key that is uniquely associated with the remote control, which should be under strict physical security at all times.
What you say? There's no encryption implemented in these devices? That's a big problem whether the code is open or not.
Re: (Score:2)
Security through obscurity by itself is to be avoided. Obscurity does add security to an otherwise secure system. Ask a secure facility sometime if you can have a copy of their guard patrol schedule.
Giving up the extra security requires something in return. It's far from clear that the extra bug-finding ability (if there actually is any) that is associated with open code is enough to be worth it.
Re: (Score:2)
You know, that's just security by obscurity, and you know what they say about that...
The code is going to do you a whole lot of good .. (Score:4, Interesting)
Really. Finding security holes in software that runs on a plain vanilla PC is one thing, finding the cause of glitches in the nanosecond range on an embedded system is another thing entirely.
Re: (Score:2)
No, but it makes it easier for an auditing body to do so, or for competitors to point out (and prove) that their device is safer.
Re: (Score:2)
Official auditing bodies could have the source code any time they ask for. They don't.
, or for competitors to point out (and prove) that their device is safer.
Re: (Score:2, Insightful)
^THIS
Implantable pacing devices, cardioverters, and pumps (life-sustaining devices) depend on complex custom hardware designs as their platform, and that hardware is *highly* interactive with the software. Many of these devices can only achieve their miraculous longevities on a primary cell by deferring functions to hardware. If you don't have access to the information re: the hardware, the code itself might as well be inscriptions in Atlantean glyphs. You'd have to bust trade-secret protection to get a
If i get a pacemaker... (Score:2)
After Therac-25, there is no excuse (Score:3, Interesting)
Then some numbskull decided, "Hey, let's let the software handle the safety interlocking, and we can cut down on hardware manufacturing costs!" The result was the Therac-25 [wikipedia.org], which maimed and killed people.
After the machine was recalled, someone finally sat down and did a real analysis of the code, and found a whole raft of problems and bad assumptions. Nancy Leveson wrote the definitive report [mit.edu] (PDF) on the failures in the R&D processes that made the Therac-25 so deadly.
Yet, armed with this warning (among many others), both manufacturers and purchasers keep human lives as transactions on a double-entry ledger. It simply comes down to, how many deaths per thousand uses are "acceptable"? Manufacturers and medical facilities already have so many costs. Is it worth it to add on the cost of formal code analysis?
But nobody will ask the Therac-25 victims and their families.
I decided early on in my I.T. career, that I didn't want the stress of people's lives depending on my correct code. I hadn't had any training in formal verification. In hindsight, I see my worries would have come from incompetent management, more than from myself.
Re: (Score:2)
So what? Engineering hardware is just the same. The Pinto is the classic example.
Licensed Professional Software Engineers? (Score:2, Insightful)
Some mechanical devices and most bridges and buildings require licensed engineers or architects to put their stamp of approval on the designs. They do not require publication of the engineering or architectural drawings though.
I for one would welcome professional licensing for certain "it can kill you if it goes wrong" software, particularly in isolated devices whose software can't be tampered with undetectably.
If a licensed Professional Software Engineer puts his seal on a pacemaker or airplane, and the s
An instance of a larger problem... (Score:2)
This is simply one more instance of a larger problem. I once heard a brilliant observation; "If uncertain, you can always discover the game you're actually playing (vs. the one you think you're play) by whichever game you happen to be winning." Our society is predicated on a game called "PROFIT". Period, end of discussion. There are a lot of other games... satisfaction, fulfillment, sustainability, quality of life, or even personal freedom. All of these other games we say we're playing. All these other game
Re: (Score:2)
Or was that MySpace?
Re:Lots of tedious details (Score:4, Insightful)
Re: (Score:2)
What good is that going to do? Who is qualified to look at it? It seems to me you would not only have to understand programming, but also the actual hardware it is controlling, the conditions it is supposed to be treating, biology, etc. If you can do all that chances are you work for a medical devices company. If someone does inspect the code, and says it is OK, does that absolve the manufacturer from all liability?
Re: (Score:2)
This is pretty much what I'm thinking too, you can't just throw up an open source repository and say that's good enough testing. You have to run through a proper process with testing and documentation of all that testing, and if that is proof of sufficient testing then what does open source add to that? Sure, there's the possibility that some random open source review will find flaws that the official, documented process missed but it'd be no necessity. If it was a necessity it would mean the formal process
Re: (Score:2)
So what if you can prove that the software does "X" as designed? Very many bugs are because the spec/design should be to do "Y" instead.
That's why math proofs of "software correctness" are not that useful in most real world scenarios.
Re: (Score:2)
GIGO
Re: (Score:2)
GIDI (Garbage In, Death Imminent)
Re: (Score:2)
'It may work as intended/designed. But the intention/design could be wrong/mistaken.
So what if you can test that the software does "X" as designed? Very many bugs are because the spec/design should be to do "Y" instead.
That's why tests of "software correctness" are not that useful in most real world scenarios.'
Modifying correct (verified) code to be valid is many times easier than modifying incorrect code to be valid.
Re: (Score:3, Insightful)
Formal methods on their own are not enough -- at least, not with the current state of formal methods. Formal methods and testing tend to expose different bugs. But the principle is right: maybe an independent safety assessor evaluates the process and products, and the manufacturer submits their argument as to why the system is acceptably safe to a regulator.
We need to be careful about what is "sufficiently safe" though. If somebody would die for sure without the implant then "the implant probably won't kil
Re: (Score:3, Funny)
Look, if you don't like the idea of implanted medical devices controlled by software, just say so.
Re: (Score:2)
In close code, if a good guy... well, forgets it, licences dont enable you to do almost anything, in fact, reverse engineering or even trying to find bugs could gets you prison, and if you av
Re: (Score:2)
Actually, I live in a world where I don't get access to the source code to Windows, but the Russians (et al) that you seem to fear so much do. Now, I don't actually use Windows for anything critical, so this doesn't affect me as much as it does some, but if you are so concerned about Russian (et al) hackers, shouldn't you be _for_ open source, so that you and the rest of the good guys can look at the source and fix and/or defend against any flaws that are found?
Re: (Score:2)
FOSS advocates are known for saying things like, "Release the code, and the community will port/fix/whatever!" So Chrome is released and initially isn't ported to Linux. So what does the FOSS crowd do in the real world? Complain that Google doesn't support Linux because *they* won't port their open source product to Linux for them. And then of course is the ATI driver fiasco.
Only in the minds of OSS heads (Score:2)
It is pure fantasy that there are tons of people with tons of time on their hands who will work diligently to solve the boring and difficult problems (which bugs are) in software. In reality, many OSS projects get almost no help at all from anyone but the core developers.
My favourite example, since I use it all the time and it annoys me, is Firefox. This is a major OSS project. It has financial backing, and large status. None the less it is riddled with bugs. Many are annoyances (like all the plugin trouble
Re: (Score:2)
The "right" way to do this is to make sure that the software escrow release is conditioned upon filing for bankruptcy or other protection from creditors or non-payment of the escrow fee.
Effectively what this does is destroy any possibility of using the code as an asset in selling the company and ensures that any type of financial problem becomes terminal. This is pretty much equivalent to using cynaide as a cold remedy - see, no more cold or cold symptoms.
Re: (Score:2)
Second of all, the GPLv3 prevents you from signing a binary to run on a specific piece of hardware. So no GPLv3 on medical devices.
The GPLv3 does not prevent you from designing a binary for a specific piece of hardware, it just prevents making it impossible for any other software to run on the hardware. If you're going to run untested code on a pacemaker, you deserve what you get. I see no problems here.