Should Linux Have a Binary Kernel Driver Layer? 944
zerojoker writes "The discussion is not new but was heated up by a blog entry from Greg Kroah-Hartman: Three OSDL Japan members, namely Fujitsu, NEC and Hitachi are pushing for a stable Kernel driver layer/API, so that driver developers wouldn't need to put their drivers into the main kernel tree. GKH has several points against such an idea." What do you think?
Why post this link without the followup? (Score:5, Informative)
Some misunderstandings were made. But of course, if they posted this link, there'd be no point to posting TFA or the arguments that will almost certainly follow.
Re:Only one word (Score:5, Informative)
Re:Amen! (Score:5, Informative)
It is not welcome. Linux is about Open Source, and allowing people to link-in binary closed drivers goes against this.
Too much politics surrounding Linux? Where have you been? It has been the policy of the Linux kernel for a long time that it would never stablize a binary driver interface, in order to prevent people from not making their drivers open source.
The idea behind Linux is that an Operating System should be Open, and Free (as in speech), and that nothing should hinder this. Binary drivers are exactly this sort of hinderance.
You may be upset that you don't have drivers for product XY because that company doesn't want to play along, but if you're trying to change the way the world does software, you can't go "ok, just because we *really* want your drivers, we're going to bend the rules for you."
Hell, no! (Score:5, Informative)
Re:Amen! (Score:3, Informative)
What gives you the right to impose that restriction on me ?
Um... I write the code? If you don't like it, then you can go use something with better driver support, like Windows.
If binary drivers were supported and it were up to companies to choose to support them or not, then they wouldn't
Again, this is all an ideological argument. If you don't like the ideological position that Linux is forwarding, then you're free to use only operating systems that do support binary drivers. What right do you have to tell me what I must do with my code to make you feel better?
Re:Oh, I'm all for it. (Score:5, Informative)
Re:Excellent suggestion! (Score:5, Informative)
Your statement has been the subject of several discussions here lately. If you are asking if "I" want the average Joe Blow to start using Linux, the answer is "I don't care". I belive the Linux is an excellent operating system that is stable, robust and powerful. I think average Joe Blow should use it for those reasons. Do I care if he does use it? As long as he doesn't call me to clean up his spyware and viruses, or at least doesn't get pissed when I charge him to clean it up, I'm fine with Mr. Blow using any OS he wants.
Thing is, if the Linux development community starts making compromises that jeopordize the security and stablility of the OS just to intice ol' Joe to use Linux we haven't gained anything. All of the reasons to use Linux go away and we haven't progressed. Linux has gained market share due to it's quality, stability and performance. Linux will continue to improve and continue gain market share because of these reasons. Eventually a free market should change to embrace a better product. There is no reason to compromise just for short term acceptance gains.
Re:Only one word (Score:3, Informative)
Also a lot of people mix up the issue of "binary vs C" with "closed vs open". You can have open source binary drivers, and that's often a convenient thing for end users to have (because the user experience of installing binaries is generally better than that of installing things from source code).
Re:Userspace, anyone? (Score:4, Informative)
Re:Absolutely (Score:3, Informative)
You might aswell read the greg's blog. Linux can't have a stable binary interface unless you: 1) lose performance (WDM-like interfaces come with a cost) or 2) lose freedom to configure your kernel (you don't allow to change some of the current kernel options like ej: regparm)
Re:Excellent suggestion! (Score:3, Informative)
Sure, it takes NVIDIA some time to catch up to the latest kernel changes, but they do a damn decent job considering the kernel is a moving target. If that were not the case, their drivers, which are already the best available, would be even better.
Many hardware manufacturers are not in a position to open up their drivers. It's unfortunate, but that's the reality of the market.
And also, there's no reason we have to accept shoddy drivers if we allow binary-only drivers. Microsoft put an end to that big mess on Windows, and there's no reason we couldn't use similar policies to ensure stability, and we could be even more strict about it. If we had a concerted effort, we could easily pressure hardware companies to fix bugs in their drivers. And for really important drivers, I'm sure there's plenty of folks who'd be willing to sign NDAs in order to get access to the drivers for improving them.
Of course, it's not my call. If the key devs don't want it, it won't happen. Depending on their goals, though, they may be shooting themselves in the foot.
Re:Userspace, anyone? (Score:4, Informative)
Re:Oh, I'm all for it. (Score:2, Informative)
If Microsoft themselves are certifying the drivers and putting them through 'very rigorous driver tests', why the blue screens? Or is it that un-certified drivers are to blame?
The latter. Not all peripherals are made by large corporations that have the means to get each driver signed.
Third-party IP (was Re:Excellent suggestion!) (Score:5, Informative)
Already done (Score:3, Informative)
For graphics, GGI and KGI would allow direct binary-only drivers to be written that applications can use, again without modifying the kernel.
Not sure whether you could make any use of the ABI/IBCS work for drivers, but they certainly allow "foreign" binaries to run under Linux, without anything foreign being put in the kernel itself.
In other words, a binary-only driver layer would seem unnecessary, given alternatives and mechanisms that already exist. It may be useful in some cases, but I can't see how it would be essential.
You could also use Xen as a reverse microkernel. Have foreign drivers in a "driver-only" mini OS, running as a parallel kernel. Then all Linux would need would be a way to call the other OS across Xen - and that need not be binary-only/closed-source. Companies interested in binary-only Linux work might even jointly fund development of such a capability.
The problem is not with the kernel, or even with the kernel developers. The problem is that corporations have unofficial choices rather than something they can put the blame on if their coding is crap. Officially sanctioned solutions are always preferred when being able to blame someone else is important.
Already Got One: NDIS (Score:4, Informative)
This is an inflammatory issue, but a stable interface doesn't necessarily open the kernel up to proprietary drivers. It's a matter of licensing. Any third party could introduce a GPLed abstraction layer. There are big practical advantages to being able to take a GPLed driver's object file and plug it in to any old kernel. In that case there would be no really fundamental reliability or debuggability problems. The remaining problem would be the increasing mismatch between the abstraction presented to the driver and the abstraction supported by the kernel as it develops.
It would be good to separate the discussion into two, one for inflammatory license- and/or ideology-related culs-de-sac, and another technical, to address legitimate needs for stability in drivers that are not (yet) in the kernel tree.
Re:Absolutely (Score:3, Informative)
and people defrauds it (Score:5, Informative)
"In a comment to one of my earlier entries, someone mentioned a driver that bluescreened under normal conditions, but once you enabled the Driver Verifier (to try to catch the driver doing whatever bad thing it was doing), the problem went away. Another commenter bemoaned that WHQL certification didn't seem to improve the quality of the drivers.
Video drivers will do anything to outdo their competition. Everybody knows that they cheat benchmarks, for example. I remember one driver that ran the DirectX "3D Tunnel" demonstration program extremely fast, demonstrating how totally awesome their video card is. Except that if you renamed TUNNEL.EXE to FUNNEL.EXE, it ran slow again.
There was another one that checked if you were printing a specific string used by a popular benchmark program. If so, then it only drew the string a quarter of the time and merely returned without doing anything the other three quarters of the time. Bingo! Their benchmark numbers just quadrupled.
Anyway, similar shenanigans are not unheard of when submitting a driver to WHQL for certification. Some unscrupulous drivers will detect that they are being run by WHQL and disable various features so they pass certification. Of course, they also run dog slow in the WHQL lab, but that's okay, because WHQL is interested in whether the driver contains any bugs, not whether the driver has the fastest triangle fill rate in the industry.
The most common cheat I've seen is drivers which check for a secret "Enable Dubious Optimizations" switch in the registry or some other place external to the driver itself. They take the driver and put it in an installer which does not turn the switch on and submit it to WHQL. When WHQL runs the driver through all its tests, the driver is running in "safe but slow" mode and passes certification with flying colors.
The vendor then takes that driver (now with the WHQL stamp of approval) and puts it inside an installer that enables the secret "Enable Dubious Optimizations" switch. Now the driver sees the switch enabled and performs all sorts of dubious optimizations, none of which were tested by WHQL.
(IOW: it doesn't guarantee stability or quality at all. It's just a false sense of "stability")
Re:out of touch linux kernel 'hackers' (Score:5, Informative)
Linus has said publically many times that the reason that there is not ABI is because he doesn't want one. Binary drivers for *anything* end up screwing stuff up. When they do, there is NOTHING anyone but the original author can do. The code stagnates, users get shut out without any help and nobody is any the wiser as to how that hardware actually worked in the first place.
That's WHY there is stuff like the kernel module license tainting, so that the kernel developers look at a problem, see that you have the massive unknown of a in-kernel binary loaded and can instantly filter your report out. They don't care that your binary driver doesn't work. They can't help you.
Additionally, setting anything into a static position means that development of it ends and stagnates. You'll never get a static interface that you can use to extend the drivers when new features come along. You end up with all sorts of kludges and interface versioning to try to take account of new things.
Linux is developed as an independent operating system, not Windows 2006. No-one wants to make you use it if you don't want to. I doubt Linux was ever intended as anything other than a "pure", almost theoretical, system; that is, one that can be constantly redesigned from the ground up to the way it should have been, not kludged to make it fit your eight-year-old driver (which the author is no longer available to update) for a mouse that happens to still use the old interface.
"Frankly, linux desperatly needs both a kernel debugger, and an ABI to be a REAL alternative for many customers."
Whoa, magic word customers. Linux doesn't have customers. Your company may have customers. There's no obligation on Linux to help you get/keep your customers. People use Linux because they want to. How often does Linus appear on your telly begging you to buy into Linux? Never. Because he doesn't care if you do or not. However, I do imagine it feels pretty nice to him that you do want to use it.
"It also needs the ABI for driver developers so that we can write a single driver and expect it to work on the dozens of flavors of linux we are expected to support."
*You* are expected to support whatever you decide to make. Unfortunately, the linux kernel developers are expected to support YOU, your hardware and everyone else in the world. They don't because they cannot and have no reason to. Even if they had your complete source code, they cannot be expected to maintain your driver for you (which is what will happen when your company goes bust / gets bored with OS).
Sometimes the best-written drivers in the world are not taken into the kernel because they don't quite fit and the maintainance involved in keeping them in the kernel is too difficult. Your driver, if it is to have any support in the kernel, needs to be able to be updated on any kernel-coder's whim in order to make the whole a better system. You can't do that with binaries, you can't even do it with stable interfaces. You have to have the source.
The kernel coders have never promised that your stuff will always work (unless it is designed to run purely from userspace... several times Linus has says that userspace interfaces will not MUCH change over time.). They haven't because they cannot.
The nature of the system is changes to bring improvements, from the interrupt system to the IDE interfaces, from the schedulers to the userspace interfaces such as sysfs or procfs, Linux changes and evolves over time and they cannot guarantee that anything other than userspace syscalls and the like will not be broken, changed or improved between one kernel release and the next.
When the linux kernel people discover a new way to write drivers that sees enhancements across the board, chances are that they are going to break any of your "single driver" models. That's why they won't give you one. Them im
Re:Learn to read. (Score:3, Informative)
That's at best debatable. To make any sense, you would have to give not only the specs but essentially the capabilities of the GPU away. I must admit I don't know that much about GPU design, but I imagine there's a lot of timing data to keep the texturers and pipelines and shaders and whatnot occupied. The exact details about how long, which operations can be executed in parallel, the break-down of time required for different steps to render the final image and so on is fairly critical. It would also make it a lot easier to reverse-engineer the closed source code, or at least to benchmark for comparison (it's usually a lot easier when you *know* there's a way to do this faster). Where and how it is hardware-assisted, and what is in pure software can certainly be mapped out with exactness. It's not impossible to do this otherwise either. But it would be lot easier if you had a well-documented hardware interface.
Kjella
Re:Learn to read. (Score:3, Informative)
What is the difference between a $600 quadro and an outdated $85 Geforce4? THe firmwire is flashed differently and the drivers will optimize for accuracy vs performance between the 2.
That is why they dont want to open.
Also designs are copyrighted and if you open the source a competitor could argue that companyA neglected its copyright by opening it to the public, so therefore its ok to steal the design. May not be entirely true since copyright is designed to share work, but an ignorant judge could look at it as carelessness for being open.
WIFI is required to be closed and proprietary by the FCC under Powell. The government can revoke its license to produce wifi cards otherwise.
Face the facts. In this day and age of litigation most companies make profit from suing each other and protecting themselves for a *potential* lawsuit. Their shareholders demand closed source and will fire any CEO who willingly let go of its crown jewels to make a few geeks happy.
Last, many vendors want profesional certified drivers. Not the crappy community ones. Yes commerical ones can suck too but I have had problems with my new geforce6600GT for example. I downgraded to an earlier driver that is "Windows quality certified" and performance is alot better.
I am sick of the windows trolls mentioning windows is bad because of the drivers. Not because of its design. The fud was started by Microsoft and I have yet to see solaris crash because of bad drivers as an example.
I find windows just works. No funky problems with acpi turning off the sound, no strange behavior with usb pen drivers, etc. Its because the drivers are good quality on WIndows and not made by hackers who do not know all the details about the hardware.
I support the kernel api since hackers can use it as well. Also we need a layer of api's of gcc and older kernels. This is why ISV's prefer solaris over linux. You can run a 10 year old app on solaris without a problem. Try that on linux where there are no layers or dynamic linking?
Re:GO FOR IT (Score:3, Informative)
You make some good points - but having a stable API/ABI doesn't resolve them.
Right now, the Linux market doesn't seem to have enough pressure to force driver manufacturers to spend resources developing drivers for it. That's not going to change if Linux provides a stable driver API/ABI. What it will do is lower the barrier to providing a closed-source driver. Some may think that's a good thing, but there's enough evidence in this conversation & its links to suggest otherwise:
1. If you let manufacturers produce a closed-source, binary driver, that's what they'll do.
2. If they produce a closed-source, binary driver, it'll only work on a specific version of Linux, for a particular architecture, since binary compatibility depends upon the compiler & compiler options being used.
3. Manufacturers still won't put more resources towards driver development, so you'll get a bunch of half-assed drivers with bugs that now live in kernel space, with the potential to crash your system (ala Windows BSOD)
4. You also get a bunch of unknown binary code running on your system in kernel space, meaning security flaws in those products can now compromise your entire system. And then we can't fix kernel security bugs, because that might break binary compatibility.
So, allowing binary drivers gives no real advantages over the long-term. Companies that care about Linux can and do produce Linux drivers. Companies that don't, won't. I see no advantage to making it easier to produce Linux drivers for companies that don't care about Linux - all that will do is give Linux a bad name as companies with faulty drivers pollute users' machines, and make it more difficult for kernel hackers to debug problems due to bad binary drivers.
Re:out of touch linux kernel 'hackers' (Score:5, Informative)
Additionally, setting anything into a static position means that development of it ends and stagnates.
Really? Could have fooled me... Windows can still run DOS and Windows applications from 20 years ago, linux can still run application from 10 years ago. When an API is created, usually there are hooks for extending it in the future. I wouldn't say either one has stagnated, over the course of the last 10 years. If that were the case windows wouldn't run on my quad 64-bit athlon.
*You* are expected to support whatever you decide to make. Unfortunately, the linux kernel developers are expected to support YOU, your hardware and everyone else in the world.
But its our brand which gets damaged, when plugged into a linux box, and it works at 1/4 speed, or looses data because the developer who implemented the driver forgot some important edge case.
They really don't care about you and your binary drivers
No, but the weekly calls by linux users asking for them says that Linus and friends are only a small part of the community.
Stick a Knoppix CD in a computer and see how much of the hardware is supported by default. 90% if not more? Include binary-only winmodem/USB ADSL/philips webcam etc. drivers and you get close to 99%.
Sure, linux supports a lot of hardware, but there is a lot it doesn't support, or supports in a seriously half ass way. I would venture a guess that about 40% of the drivers in the kernel don't actually work based on my experience running linux since the early '90s. When I say don't actually work, i'm saying the driver cannot recover from rare hardware error conditions (like FC cable pulls for example), doesn't fully work. Doesn't work in all situations, for example SCSI adapters that work fine with harddrives but don't work with tape drives because they cannot deal with the rare condition of a tape drive returning check conditions and data at the same time. Or, works fine at some fraction of its rated capability. This isn't counting the fact that the kernel developers often break functionality in the kernel drivers when they change stuff and it doesn't get discovered for 6 months, because the developer making the change didn't test the driver that was affected. Now, your average slashdot weeny doesn't see this because they have standard whitebox or dell machines, that are the same as the other 90% of slashdot weenies. In real life there is a vast amount of strange hardware out there, professional audio cards, hardware encryption engines, 4 port fiber channel cards, ficon, 10G ethernet cards, large disk arrays, big memory systems, 10's of different types of system management interfaces for monitoring things like, system ECC soft errors, redundant power supplies status, etc. The list goes on. In many cases the linux driver was written and tested a year ago, a bunch of people are using it in a production environment and it has been broken for 6 months in the mainstream kernels. I was responsible for fixing a number of race conditions in the VMM a couple of years ago, that only occurred with a SMP box that had more than a couple G of RAM (a rarity back then). This was suppose to work, but it didn't because the people with 6G of ram didn't test it with SMP applications and the people with >2 CPU's didn't test with more than a few hundred megs of RAM. Recently I tried bench marking a 3TB disk array in linux, only to discover that it didn't work in over half of the file systems I tried (XFS, Reiser, JFS, ext2, ext3) and in at least one case worked but was so slow it was unusabl
What OSS is all about!!! (Score:3, Informative)
Open Source drivers are the cornerstone of OSS... because if you can't see how something works, or even HOW to make it work then you have nothing but a plastic box on your desk. The point is that you paid for the device, printer, video card, cpu, so you should have to have anybody's permission telling you how or when you can use it.
While I'd like proprietary drivers too, some stuff is patented or secret and they don't want to share it, offically it will never happen... Linus's view is that he can't fix things if he can't see them.. and he's not going to sign a bunch of NDAs just for drivers, he's too busy. Nor, can he gaurantee what the kernel is doing if you're running a bunch of "secret" stuff that could be doing "god-knows-what" behind your back!
Re:Learn to read. (Score:4, Informative)
That's not protecting IP, its protecting the fact that they are scamming customers.
"Also designs are copyrighted and if you open the source a competitor could argue that companyA neglected its copyright by opening it to the public, so therefore its ok to steal the design. May not be entirely true since copyright is designed to share work, but an ignorant judge could look at it as carelessness for being open."
You have absolutely no clue what you are talking about. Copyright does not need to be protected to be valid, that's trademark.
"WIFI is required to be closed and proprietary by the FCC under Powell. The government can revoke its license to produce wifi cards otherwise."
Yeah, that would explain why several companies have fully open docs for their wifi chips, thus enabling completely open source drivers for them.
The rest of your post is irrelivant windows nonsense that has nothing to do with the topic.
Userspace v. kernelspace. (Score:3, Informative)
That's not exactly a legit comparison. gstreamer is an application; X may run in userspace, but it's part of the system in the same way udev is; it's not in the kernel because it doesn't have to be.
It seems we have two competing driver models in Linux: some are in the kernel, and provide a consistent interface (sound cards, SCSI/IDE/... cards, network cards), and some aren't in the kernel at all, but expose them at a low level and rely on userspace programs to provide actual drivers (X11 for video cards, CUPS for printers).
Not exactly. For instance, CUPS may have printer drivers, but it relies on the USB, parport or network communications exported by the printer. There's no compelling reason for it to be in the kernel, since those drivers don't talk directly to the hardware in the same sense that the port drivers do.
Or consider gphoto. The port drivers are part of the kernel, but the camera drivers are in userspace--because there's no compelling reason to put them in the kernel, since there's nothing in there they need.
So that's my understanding of why X or CUPS or gphoto have their drivers in userspace, while sound drivers and port drivers are in the kernel.
Re:Crossbar? (Score:3, Informative)
Oh Lord. Save me from the ignorant. Hardware systems are NOT that easy to design. The GPU doesn't do JACK other than crunch some numbers, and it's already one of the most complicated devices in existence! (Primarily thanks to the complex pipeline designs, the number of parallel pipelines, and the microcode space and execution hardware carried on chip and intertwined with each pipeline.) ALL the real work is in the software. The hardware just executes its instructions. If you tried to cram all the work the software is doing into the hardware, you'd have something more complex than the latest Pentium and NVidia GPU combined. Not to mention that you'd never be able to increase performance through driver updates.
And you need to be careful with your concepts of Ports. Bus design is tricky business and can easily bottle up the entire computing platform if you're not careful. Many commands on one bus is something you see more of in general purpose computing where the raw performance isn't an issue. In the case of a GPU, raw performance IS an issue, and the software directly controls the entire bus + the GPU.
What the hell are you talking about? I wasn't complaining about GPUs or CPUs.
That's because you have no idea what you're talking about. The hardware doesn't do video processing. I don't know how many times I'm going to have to repeat that. It's all software, and it's all trying to compile the instructions, reorder them, JIT microcode, parallelize the data, and do other complex optimizations that will actually feed the computations into the GPU. The GPU spits out an answer that ends up in VRAM. How nice.
And, before you ask, I do indeed program, although admittedly it's been a decade since any assembly language.
No offense, but ASM is way off in left field. We're talking hardware design here. And the only way to truly understand hardware (including the software that drives it and WHY it drives it that way) is to go design some yourself. Unlike yourself, I've actually taken the time to do so, and now understand why things are the way they are. I'm not sure how I can get this through your skull, but complex hardware is HARD and EXPENSIVE. It never makes sense to use hardware when software will do the job just as effectively (or in this case, more so).
When it does make sense is when you're trying to accelerate very specific and abstract mathematical operations. e.g. Matrix math can be easily accelerated in hardware. Drawing a line on the screen? Not so much. If you want to accelerate something like drawing a line, you'll want to use Microcode. Microcode is tiny software that loads up and executes on the CPU itself. Microcode is pretty micro though, so it can only go so far. To do something more complex, (say, draw a shape with those lines) you want to bump it up to software. The on-chip cache will keep the code running smoothly inside the chip so that the software isn't constantly crossing the bus during the operations. Thus you end up with software that drives the hardware.
Now imagine that you had software that waited until you had drawn all the objects with lines, then sorted them to figure out which ones get drawn in what order, removes any lines that are hidden, checks which lines can be drawn in parallel because they don't cross, then does a final reordering of the line drawing commands so that the hardware can draw four lines simultanousously (complicated by the fact that some lines are shorter than others, and will take fewer cycles to complete). That is what your big blob of "useless" software is doing.
If you think there's a better way this stuff can work, then by all means. Go design some hardware and show the world. (It's not that hard to get started. Really.) In the meantime, though, just be quiet. You're making yourself look bad.