Ryan Gordon Wants To Bring Universal Binaries To Linux 487
Posted
by
timothy
from the grossly-obese-binaries dept.
from the grossly-obese-binaries dept.
wisesifu writes "One of the interesting features of Mac OS X is its 'universal binaries' feature that allows a single binary file to run natively on both PowerPC and Intel x86 platforms. While this comes at a cost of a larger binary file, it's convenient on the end-user and on software vendors for distributing their applications. While Linux has lacked such support for fat binaries, Ryan Gordon has decided this should be changed."
Only useful for non-free applications (Score:5, Insightful)
If you have access to the source, you can always compile a version for your platform. The 'fat binary' principle is only useful for non-free applications, where the end-user can't compile the application himself and has to use the binary provided by the vendor.
Since most apps for Linux are free and the source is available, this feature isn't as useful as it is on the Mac. Not that it shouldn't be created, but it makes sense to me why it took a while before someone started developing this for Linux.
Re:Linking problems (Score:1, Insightful)
Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?
Not really, it'll just make bigger binaries that'll run on a system you'll never use.
Does Linux even need them? (Score:5, Insightful)
We need 1-file installs (Score:4, Insightful)
We don't need the universal binary, so much as we need the "1-file install" idea that MacOS has. This would greatly simplify installing a standalone application.
For those of you who don't know, if you download an app for MacOSX (say, Firefox) you are presented with one icon to drag into your "Applications" folder. This is really a payload, a "Firefox.app" directory that contains the program and its [static?] libraries. But to the user, you have dragged a single "file" or "app" into your "Applications" folder - thus, installing it.
It's dead simple. We need something like this in Linux.
Re:Linking problems (Score:5, Insightful)
Could this technology also help binaries to link against multiple versions of standard libraries (glibc, libstdc++)?
Probably not. Or not without getting headaches like you get with assemblies on Vista. Keying off the system architecture (32-bit x86 vs. 64-bit ia64) is much simpler than keying off library versions.
The fix with standard libraries is for the makers of them to stop screwing around and stick with ABI compatibility for a good number of years. OK, this does tend to codify some poor decisions but is enormously more supportive of application programmers. Note that I differentiate from API compat.; rebuilding against a later version of the API can result in a different - later - part of the ABI being used, and it's definitely possible to extend the ABI if structure and offset versioning is done right. But overall, it takes a lot of discipline (i.e., commitment to being a foundational library) from the part of the authors of the standard libs, and some languages make that hard (it's easier in C than in C++, for example).
convenient for _closed source_ software vendors (Score:4, Insightful)
Sofwtare vendors? This only makes life easier for _closed source_ software makers. For everyone else this is a solution looking for a problem as package management and repositories don't really have a problem with different arches and versions.
I'm not saying this is useless (people do want to run closed source software), but the kernel, glibc and other patches better be good and non-invasive if this guy wants them to land...
Re:Only useful for non-free applications (Score:5, Insightful)
Re:convenient for _closed source_ software vendors (Score:3, Insightful)
Actually, having to maintain packages across several architectures can be tricky at times. Some packages need to be patched to run correctly on different architectures, and the upstream maintainers can accidentally break those patches (e.g. if they are not personally testing on a given architecture). It could even be the case that different architectures have different versions of the same packages, because the distro maintainers are busy trying to get everything to work.
I am not saying that this "universal binary" solution is the answer, but it might help streamline the build process at the distro level. It might help.
Not scalable (Score:5, Insightful)
To a first approximation, the size of the binary will increase in proportion to the number of architectures supported.
This is something you might decide to ignore if you are only supporting two architectures. Debian Lenny supports twelve architectures, and I've lost count of how many the Linux kernel itself has been ported to. I really don't think this idea makes sense.
(Besides, what's wrong with simply shipping two or more binaries in the same package or tarball?)
Re:Only useful for non-free applications (Score:5, Insightful)
"Not everyone is skilled enough to compile the source on their own"
By "end user" we can understand here "distribution maintainer" which already has the skills to compile the source (and that's not but a part in the lot of things that have to be done in order to integrate some software in a distribution).
"I would think this might be useful for distro maintainers who do not want to maintain separate packages across multiple architectures"
But they have to: they still must build and integrate for their supported platforms, then rebuild when bugs are found or the software is upgraded, then test... It's just the last step (producing the very binary packages) that changes so instead of multiple packages you'd end up with a single multplatform package. The distributor still need (almost) as much disk space and infrastructures as before, but then each and every user will end up with spending much more space in their hard disks (imagine the fat binary for, say, Debian, supporting eleven platforms).
And then, please note that this will allow for single binaries for diferent hardware platforms but not for different version compilations (so it won't be useful to obtain binaries for, say, amd64 for Debian, Red Hat and SUSE).
It seems it will only benefit to those that want to publish their software in an only binary form outside the framework of stablished distributions and that means closed source software. Of course they can look for their bussiness the way they feel better, it's only they don't get my simpathy so I don't give a damn about them.
Re:Only useful for non-free applications (Score:3, Insightful)
Furthermore, I think you mean to say that it's "only useful for non-open source applications" as there are tons of free software applications out there that are not open source but are free (like Microsoft's Express editions of Visual Studio).
He means free as in Stallman-Free, not the free as in cost. That's what I don't like about this 'free' as in 'freedom' thing, it's needless confusing by trying to change the meaning that first comes to mind to people. They could've just gone with libre or something.
Re:convenient for _closed source_ software vendors (Score:5, Insightful)
"Actually, having to maintain packages across several architectures can be tricky at times."
Of course yes. But let's see if the single fat binary reduces complexity.
"Some packages need to be patched to run correctly on different architectures"
And they still will need that. Or do you thing that the ability to produce a single binary will magically make those incompatibilities to disapear?
"the upstream maintainers can accidentally break those patches (e.g. if they are not personally testing on a given architecture)"
That can happen too with a single binary exactly the same way.
"It could even be the case that different architectures have different versions of the same packages, because the distro maintainers are busy trying to get everything to work."
Probably with a reason (like new version needs to be patched to work on this or that platform). How do you think going with a single binary will avoid that problem? It's arguably that in this situation you would end up worse. At least with different binaries you can take the decision of staying with foo 1.1 on arm but promote foo 1.2 on amd64 in the meantime; with a single binary it would mean foo 1.1 for everybody.
"I am not saying that this "universal binary" solution is the answer, but it might help streamline the build process at the distro level."
Still you didn't produce any argument about *how* it could help.
Re:Only useful for non-free applications (Score:4, Insightful)
Re:Only useful for non-free applications (Score:5, Insightful)
You've got to be kidding? Super-easy installation and automatic security updates for all applications is 'awkward'?
If I understood you correctly, your suggestion is that desktop software should be hard to find, it should be installed from whatever website I happen to ultimately find and it shouldn't automatically get security updates. Sounds fabulous.
Don't get me wrong, I agree that package management systems have their flaws (even inherent ones) but you just aren't making a good case against them... You could start with explaining what's unnatural about "Open 'Add applications', check what you want, click Install", and then continue with explaining what's awkward about totally automatic security updates.
Re:Only useful for non-free applications (Score:1, Insightful)
>>> which transfers only part of fat binary that is actually used on client
Brilliant. First people try hard to pack executables for different platforms together in a single file, then (other?) people try harder to separate them back, and everyone looks busy pushing the cart of progress...
Re:Only useful for non-free applications (Score:5, Insightful)
Please elaborate.
I too agree that this is pointless for the end user in Linux, at least when it comes to free software. Only closed binary blobs will benefit, which IMHO is not something worth putting effort towards helping. They did their design choices and accepted the reality in doing so.
As for the end user, she should just use the package manager of her distro and find whatever she needs. Not worrying about neither compiling nor platforms. /etc/apt/sources.d. Not only will the user be able to use the package manager to install your app like any other, she will also get security updates you publish.
For example, in Debian/Ubuntu you could more easilly package your installer to simply drop a file in
Let the package system handle these things, they do it well and does not bloat your boat.
Re:We need 1-file installs (Score:4, Insightful)
That is great for software supplied by your distro's repository, and most distros have lots of software available in their "contrib" or equivalent repository. Firefox of course usually comes installed out of the box, so it isn't an issue.
Where this could be beneficial is for software that isn't popular enough for the distros to package. At the moment, you have to publish different packages for each distro and for each architecture, and you probably won't bother about much beyond i386 and amd64.
Re:Only useful for non-free applications (Score:4, Insightful)
No, my example wasn't a Linux one. Who cares. The main point is that it's not just that easy to build from source.
Well, since TFA is about a fat binary system for Linux, it is kinda relevant to narrow your scope to just Linux. How stuff in Windows or any other operating system work has nothing to do with this new Linux-specific feature.
That said, Windows is probably the worst platform for consumers wanting to compile their own applications. It doesn't provide any tools to do so by itself and if the source you want to compile doesn't include something like a Visual Studio project file, you're in for a very hard time. Linux doesn't suffer this fate at all. Compiling an application is in most cases nothing more complex than typing "./configure && make" and you're good to go.
Besides, what is so horrible about having fat binaries on Linux?
Nothing. I'm not saying it is. In fact, I'm saying it isn't. It just doesn't surprise me that it took a long time before someone started to develop something like this, while other platforms had this feature for quite a few years, because the need for this on Linux isn't on the same level as it was for Mac OS X back when Universal Binaries made their entry.
Re:We need 1-file installs (Score:1, Insightful)
Yeah, and who actually uses GNUstep? NOBODY.
Re:Apple dropped it (Score:3, Insightful)
Wouldn't surprise me if this is to encourage users to demand a native x86 version of software - once every significant application exists as x86 binaries, Apple can drop support for Rosetta altogether and that's another developer or two freed up to work on furthering their products rather than backward compatibility.
Re:Only useful for non-free applications (Score:3, Insightful)
Less Fat, more Unreal (Score:3, Insightful)
Use the source, Luke! (Score:3, Insightful)
Now why would I want to do anything that fucktarded, when I can just use the source?
And if I needed cross-platform that badly, I can always ship ONE java app with ONE instance of data.
The '90s called, they want their obsolete fat and universal binaries back.
Re:We need 1-file installs (Score:1, Insightful)
If you are talking about the delay e.g. Debian Unstable has, that is the price you pay for the packaging effort: Someone has to package the new version and make sure it co-exists nicely with other software. Sometimes it takes time (Firefox 3.5 is a good example: it brings in a ABI incompatible xulrunner which breaks other apps), but I'm glad someone does that job for me.
If you are talking about the delay until things show up in Ubuntu or Debian Stable repos... Well, maybe you shouldn't pick a release-based distribution if you want constant updates?
Re:Only useful for non-free applications (Score:5, Insightful)
But... "compiling for your platform" is just another way to install software. You could wrap this in a little application (call it "setup"), where you click "Next >" several times, and as a result you have a binary for your platform.
Wow, the lack of grasp on reality around here really amazes me sometimes. But it looks like it worked for you. The open source fanatic fan boys shot your karma through the roof. Congratulations!
Compiling non-trivial applications from source can take a long time. That fact alone can make precompiled binaries a big win for most users.
I did the "compile from source" thing for a long time on FreeBSD before finally realizing the pointlessness of it all. Not only was I completely unnecessarily beating up on my hardware, but spending far too much time waiting for compiles to complete.
These days, I grab precompiled packages whenever possible, and you know what? It's a hell of a lot better.
Re:Only useful for non-free applications (Score:3, Insightful)
Something else occurs to me... if you can't be bothered to create packages for
distinct platforms (in this case CPU architectures) then it seems that you
probably can't be bothered to go through the even greater trouble of actually
testing them.
If you support both x86 and PPC with any level of diligence then some sort of
fat binary format really doesn't buy you that much.
If you are any good, compiling source and building packages is a COMPLETELY AUTOMATED.
This is a very important fact lost on the clueless n00bs that don't understand
anything and see this as yet another opportunity to make lame pronouncements
regarding "why Linux hasn't taken off yet".
Packaging is the ver tail end of the process. It's the trivial sh*t that they
assign to college interns.
Re:Somebody should tell this guy about ./configure (Score:3, Insightful)
Calm down. Gentoo has almost finished building. In a few hours, you'll be able to use your machine again.
Didn't we do this before? (Score:0, Insightful)
And wasn't it called Java?
Are any of you guys old enough to remember CP/M running on the 8086? It was a solid, no-glitch way of running binaries from one OS to another. And Z80 code seemed as complex to most techs then, as protected-memory schemes do now.
But they *did* it, and did it well.
If we are *ever* going to actually have a 'universal binary' we need to make the hardware do the job: not software.
Case study: 1988, SCI Systems, Huntsville, Alabama.
Three software guys huddled over their 80186-based site-controller motherboard our company built. These guys were GODS, able to write compilers AND embedded control software, all interrupt-based, pre-emptive long before Linux. All three agreed it wasn't software, this *must* be something in the hardware. There'll be yelling and noise, but let's get a hardware tech here.
"Neil!" someone said. And this odd-looking, couldn't-be-nicer guy snagged an oscilloscope cart on his way over. He seemed to already know the question.
He clipped on the grounds, checked about 4-5 pins and dropped the probe. 'Software problem.' and walked out. I thought he had to be the most arrogant guy in the room, until I did the math:
He wrote the schematics. He laid out the parts, he did the prototype, he did the solder, and every other masks. He wouldn't have sent the device to production if the Chip Enable lines weren't working. Because of this, if they're trying to talk to the memory chips (using Intel's STUPID, 16-ways to describe-every-location scheme) then it was a software problem.
The point: hardware guys have more they can cross-check; they can't move on until one level of production checks. Based on that, future levels don't _tend_ to need a complete re-fit. You just can't get that in software: people don't work that way. Software is just too elusive. I've been saying this since 1978.
They were trying to work out a 'java' at that time, they're still trying today. But different 'runners' and different code, and we STILL AFTER ALL THESE YEARS HAVE NO REAL PROGRESS. If anything works at all in the customer's hands, it's a surprise.
Make all processors have unified code, or drop the project. There's SO much more we could be doing, please?
Re:Gee, just 14 years (Score:4, Insightful)
The problem is that disk space is NOT cheap at all, or plentiful. ARM-based Linux is used on a lot of embedded devices where there's only 16 or 32MB of flash space, total. This "fatELF" idea makes no sense, because adding in x86, x86_64, MIPS, Alpha, and SPARC binaries to your ARM binary will make everything take so much space that much more (expensive) flash memory would be needed.
This just isn't a very good idea. It makes sense for Apple, which only has to worry about 2 architectures on the desktop, and wants to make things easy for consumers, but that's it. It doesn't make sense for Linux. And I'll bet that Apple doesn't use this "universal binary" thing on their iPhone, either.
Java (Score:4, Insightful)
Kind of like Java then.
Re:Gee, just 14 years (Score:3, Insightful)
As a former Autopackage developer, no, it isn't what Autopackage is about.
Autopackage attempts to do more than just packaging: it also tries to fight the binary compatibility problem. Probably the most widely known example is this one: compile a binary on distro A, run it on distro B, and get "symbol foobar@GLIBC_2.8 not found" errors. (There are a lot more binary compatibility issues than that though.)
However, making cross-architecture binaries is not one of Autopackage's goals.
Re:Gee, just 14 years (Score:5, Insightful)
And I'll bet that Apple doesn't use this "universal binary" thing on their iPhone, either.
You'd lose that bet. Apple provides complete support for universal binary on iPhone, allowing developers to compile for ARMv6 (compatible with every iDevice) and ARMv7 (newer ISA; works on iPhone 3GS + iPod Touch 3G).
It makes sense for Apple, which only has to worry about 2 architectures on the desktop
Actually, 4: PowerPC, PowerPC 64, x86, and x86 64. Though for the purposes of your argument its probably an immaterial difference.
Re:Gee, just 14 years (Score:3, Insightful)
Re:Only useful for non-free applications (Score:3, Insightful)
16 2.0 Ghz cores and compiling will only be marginally more time-intensive than binaries.
On what kind of battery?
It appears you don't play video games (Score:3, Insightful)
I do not propose anything. Why should I?
You propose no solution for distribution of non-free software. You propose no solution for funding development of free video game software. Therefore, you appear to propose the elimination of the video game industry. A lot of Slashdot users who like to play video games disagree with your proposal.
Re:Only useful for non-free applications (Score:4, Insightful)
You can't tell me you've never run into the situation where installing a single open source package ends up taking you down a three hour maze of dependencies. Sure, sometimes you get lucky and everything just works, but many other types, you discover that application A needs libraries B and C installed, and B needs libraries D, F and G, and then the version of F you downloaded isn't compatible with package Y, so you try to upgrade Y only to discover that it doesn't work with package Z, until you say "fuck it" and just go try to find a binary.
Re:Only useful for non-free applications (Score:3, Insightful)
If you are any good, compiling source and building packages is a COMPLETELY AUTOMATED.
If that is like you say, then please explain why, e.g., Debian has a whole legion of people they call "developers" whose sole purpose in life is to take sources from other people and create deb packages?
It's called JAVA (Score:1, Insightful)
I thought this was the whole point of Java. Why are we going back and trying to solve a problem that was solved FOURTEEN YEARS AGO