The Future of Packaging Software in Linux

michuk writes "There are currently at least five popular ways of installing software in GNU/Linux. None of them are widely accepted throughout the popular distributions. This situation is not a problem for experienced users — they can make decisions for themselves. However, for a newcomer in the GNU/Linux world, installing new software is always pretty confusing. The article tries to sum up some of the recent efforts to fix this problem and examine the possible future of packaging software in GNU/Linux."

The five ways

[Harmonious Botch]

For those who don't TFA: There are currently at least 5 popular ways of doing it:
1) Installing directly from source code,
2) Ports-based installation (where the source packages are held in a repository and can be automatically downloaded, compiled and installed), like BSDs ports of Gentoo's portage,
3) Installing from distribution-specific packages like different versions of RPM, DEB, TGZ, and other packaging formats,
4) Installing from distribution-independent binaries (most proprietary software is delivered this way),
5) Using another distribution-independent system like autopackage, zero-install or klik -- none of them gained a significant market share so far.

Re:The solution!

[tigerflag]

PCLinuxOS uses a combination of Synaptic with RPMs from the PCLOS repository. Easiest package management I've ever used.

Hasn't explored other packaging methods

[shermozle]

(while discussing RPM)

Still, a lot of other systems like Ubuntu, Debian, Slackware, Gentoo or Linspire do not use the RPM format and do not plan to incorporate it.

What that means is he hasn't used any other packaging formats. Common mistake that people think RPM is somehow "best" because it's used by a few distros. Do some searches for "circular dependency RPM" to see why that's just not true.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:The solution!

[Max Littlemore]

Using multiple package formats is great idea, IMO. I use alien on Ubuntu for those situations where the software I want is only avaliable in RPM, but as it says in the summary, new users can be a bit confused by this and building from sources is often too much. I would like to see GUI tools get the smarts to automatically figure out dependencies across all formats, allowing all distros to become package agnostic. Perhaps Linspire's CNR interace would be a good candidate for this.

Also, the option to resolve dependencies and install as a statically linked blob would be awesome for legacy stuff. I've lost count of the number of times I've wanted to install an app, only to find that it relies on some obscure version of xyz.so and won't work, so I find the source for the old version of xyz, only to find it depends on some older version of abc.so. If I could get this xyz.so, etc without conflicting with that xyz.so, create a static binary and put it somewhere under /opt, I'd be happy. I know it's not elegant, and that it uses more storage, but as a work around for difficult to support stuff, it ain't so bad when storage is cheap. Some apps I always install as blobs anyway, such as blender.

BTW, from TFA: Network Access Message: The page cannot be displayed
Slashdotted :-(

Re:Hasn't explored other packaging methods

[DoubleRing]

Hear hear! I have mod points, but I'd rather post.

Circular dependencies, aka RPM hell, is what actually prompted me to make the switch from the Red Hat family to the Debian family. I used to be a pretty die hard Red Hat user. It used to be that Fedora was the cutting edge, back in the core 2 and 3 days. I would have those days when I would wrestle with the packages, but I just took my hits and moved on. Then Ubuntu came along, and I realized how much time I was wasting with that stuff. It "just works." APT is great (it's a pity POSIX decided to go for RPM). Gentoo's portage is really cool too, but IALAB (I'm a lazy bum--if you can't reconcile the acronym, then you probably shouldn't know what the missing word is).

Re:How about we take the easy way out?

[QuantumG]

Apt rules, shame about dpkg. Even bigger shame that apt is built on dpkg, eh?

What pisses me off is the 32 step process to making a deb (that's what dpkg calls a package btw.. just incase you're playing acronym bingo out there). So if you want to install something you built from source, and be able to remove it later, you need some freakin' magician to have made it into a source package.. cause there's no way in hell you're doing it yourself.

What really depresses me is that debs, dpkg and apt, that's about the best anyone has done. Unless, of course, you actually like building everything from source.

Re:How about we take the easy way out?-Fallback.

[seifried]

up2date (front end for RPM) includes the option ("5. enableRollbacks yes/no"). RPM supports rollbacks, config files would be saved as [filename].rpmsave or [filename].rpmnew depending on exactly what you are doing.

You want "checkinstall".

[khasim]

What pisses me off is the 32 step process to making a deb (that's what dpkg calls a package btw.. just incase you're playing acronym bingo out there). So if you want to install something you built from source, and be able to remove it later, you need some freakin' magician to have made it into a source package.. cause there's no way in hell you're doing it yourself.

Checkinstall http://www.debian-administration.org/articles/147 [debian-adm...ration.org]

It's not the answer to all issues regarding installing from source ... but it does handle some of them.

What really depresses me is that debs, dpkg and apt, that's about the best anyone has done.

Any suggestions on what would make them even better?

Re:How about we take the easy way out?

[seifried]

#1. It must make installing new software as easy as it currently is with apt.

up2date -i [package name]

"This package will require the installation of these additional packages, accept?" Yes/No

#2. The same for upgrading the software.

up2date -u [package name]

"This package will require the installation of these additional packages, accept?" Yes/No

#3. The same for removing the software.

rpm -e [package name]

#4. The same for handling dependencies. Including the order in which dependencies must be installed.

Already done, see above. up2date will find dependancies, dependancies of dependancies, etc. until it is done, then present you the list and confirm to install all the packages, you hit "Y" and you're done.

If you just want to check what would be a dependancy you can:

up2date --solvedeps=[package name]

which accepts a comma deliminated list of packages as well as a single package name

#5. The same for validating the installed software against the original software (checksums or whatever).

To verify packages installed (or package files not yet installed) you use the verify option:

rpm --verify

Which can check the GPG sig of the package file, the MD5 sigs of the files, etc. allowing you to detect any tampering or changes.

#6. The same for re-installing the software over the existing installation when you accidentally delete or over-write something.

up2date --force [package name]

rpm --force [package name]

#7. The ability to point the updater at your own repository or multiple repositories.

/etc/sysconfig/rhn/up2date

serverURL[comment]=Remote server URL serverURL=https://xmlrpc.rhn.redhat.com/XMLRPC serverURL[comment]=Remote server URL serverURL=https://www.centos.org/XMLRPC

#8. The ability to recompile (automatically) any software that you install for your specific hardware.

rpmbuild -ba

But in general major vendors provide optimized packages for various architectures that rely on heavy math/etc (kernel and OpenSSL being two of the important ones)

A total load of bullshit, and here's why

[Overly Critical Guy]

My favorite thing in package discussions is when someone with an agenda towards a particular implementation writes out a list of steps. The alternative is always padded with extra steps to make it more difficult-looking while their favored implementation is reduced to look squeaky clean and easy.

You padded the Mac list with the following:

• "Open disk image that contains the program." - DMGs are auto-mounted by Safari.
• "Open Applications folder." - There's already an Applications shortcut on the Finder, so you just drag to that when the disk image window automatically opens.
• "Create new icon in dock." - The fuck? You don't have to do this
• "Have to recheck the site periodically to check for a update for a specific program" - Bullshit. This doesn't even have to do with package management, and it's an OS X convention for apps to auto-check for updates when they're run. You don't have to recheck any websites.

Your Debian list conveniently leaves out having to click the KDE start menu, fire up a Terminal window, type in the root password, waiting while the package manager goes through dependencies, etc. What a phony comparison of steps. I could just have easily reduced OS X's step to one line of "Drag app icon to Applications shortcut" in the same the way you reduced Debian's steps.

Re:Ubuntu

[deek]

Why is Ubuntu in that list? I may be misinformed, but it was my understanding that Ubuntu was based off of Debian.

  You're not misinformed, although the author may still have a point of including it on the list of base distributions. There's a slew of Linux distributions based on Ubuntu. Still, you're right. The grandpappy of them all is Debian.

  Here's a fairly comprehensive list [debianadmin.com] of these distributions.

Re:Possibly.

[QuoteMstr]

Such functionality would probably be beyond the basic options of installing an unpackaged app from source code. But there's no reason that additional information couldn't be included with packaged apps detailing the more common options.

At least in the RPM-speaking world, we already have this functionality in the SRPM, though not in a formalized sense. Often, compile-time options require only very small changes to the spec, or can be given on the command line. For example, the bytecode compiler option for most freetype RPM packages falls into this category.

The fewer apps you're running, the fewer avenues of attack there are for you to be compromised. This could be a great feature for securing boxes.

I believe that the set of people who want to go gung-ho securing their boxes, who know what they're doing, AND who can't be bothered to recompile packages from SRPM form is very small.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:Hasn't explored other packaging methods

[Alioth]

Circular dependency hell?

So package foo depends on package bar, and package bar depends on foo.

All you do is:

rpm -Uvh foo.rpm bar.rpm

Circular dependency solved. The circular dependency 'problem' (it never actually really existed) was more of a problem of lack of good documentation than a problem with the actual 'rpm' program. However, this is a problem that was solved years ago - I haven't used a distro in the last 5 years that hasn't had a system like yum, up2date or apt which does all the dependency resolution for you.

Re:goddammit

[Salsaman]

Well, the system may be perfect, but the packagers certainly are not. It's next to impossible to get a new package into their repositories (I've been asking for over 2 years now !), which is I why I have to rely on the good people at getdeb.net (Ubuntu), and debian-multimedia.org (debian).

Re:You want "checkinstall".

[cortana]

dpkg --root=$HOME --install foo.deb

That option also makes it easy to set up chroots.

Re:Hasn't explored other packaging methods

[Anonymous Coward]

Perhaps you should have followed his advice and Googled it instead. He's talking about huge circular dependency trees like these [centos.org] that thwart your attempts to install a single small package. Or would your advice be to install all five of those packages at once? If you follow the thread it turns out that some of the dependencies are bogus: an all too common problem with RPM's where the spec writer either doesn't know what the dependencies really are, or doesn't care, or doesn't understand the spec file format.

Re:The solution!

[petermgreen]

MSIs are one of the best ideas for Windows in a while
note that MSIs are nothing new, office was using that system (with a stub exe to make sure the microsoft installer was installed first) from office 2000 onwards and i think win2k shipped with it as standard.

Some people seem to think that the fact that Debian does things differently from Mandriva that does it different than Fedora is what makes the distribution "special". Be that as it may, I think it's only hurting Linux users as a whole.
It is certainly a problem, trouble is the moment you get standards bodies involved progress slows to a glacial pace, distros understandablly want to be able to improve the way thier libraries are packaged to meet thier users needs.

In many ways windows represents the opposite extreme, they are only just now (with the 64 bit versions) dropping support for win16 apps, windows contains a huge ammount of cruft for backwards compatibility.

another problem is that library authors act in ways that make it very hard to build binaries on a newer system that will run correctly on an older system.

a final problem is that menu systems vary so even once you get a binary on the system in a way it will run (though i belive freedesktop.org is trying to work on getting this standardised)

tools like autopackage try to work arround theese problems but doing so is easier said than done.

Re:The solution!

[penix1]

the simplest solution may be to just include all dependencies inside it.

That's exactly how Microsoft tried to solve it and why their distributed software is huge. It also has led to problems of competing dlls which are often incompatible. If you think you have dependency problems now, just wait until you implement your idea! Imagine installing 16 applications each having a dependency on 16 different versions of the same lib.

B.

Re:The solution!

[Enderandrew]

His point was not that everyone should use Gentoo, but rather that Portage was designed as a repository that handled binaries and source easily side-by-side.

And I agree, the solution really is to have one major package repository. How you package it isn't horribly important.

The problem is that each major distro is built using different versions of libraries with completely different toolchains, and they use different patches, etc.

That is why most repository systems won't work.

However, here is why Portage would work for all distros. Keywords.

By using "SUSE" as a keyword, it could pull the binary compiled using the SUSE toolchain if that is what you needed.

The world won't start using one distro. And SUSE can have their binary packaged in their format, and Ubuntu can have it packaged in their format, but we really, really do need one repository.

Re:The solution!

[ElleyKitten]

Why do Linux fundamentalists believe that all users are idiots and they should go somewhere else? Until the majority of Linux users and developers get past this mentality we will never see Linux accepted into the main stream desktop market. Yes, most general users can find all the software they need in a single distro, but most users don't know Ubuntu from Fedora from SUSE. If they pick a distro that doesn't include a software package that they want it shouldn't require uninstalling the OS and installing a new one.

If a user picks Fedora or Ubuntu or SuSe, then they should be able to find just about anything they need in their distro's package manager. They all include alternative web browsers, chat clients, games, KDE/GNOME/XFCE, programming tools, image editing software (as best as it gets in Linux), wine, and even different file managers and shells and stuff that average users would never care about switching from the default. If a newbie picks Slackware or DSL or FreeBSD and figure out what to do or how to install programs, then yes, they should switch to a more mainstream and newbie-friendly distro. But there's not that much differences in what's in the repositories of the main distros, so they shouldn't need to switch from Fedora to Ubuntu because of what packages are available.

Distros shouldn't have to include every single piece of software that a user might want also. If they stopped doing this distros wouldn't require 5+ CDs or a DVD or two. Now, don't get me wrong, I appreciate having most of the programs I will need available on a set of five CDs, but this shouldn't be a requirement of distros.

It's not a requirement of distros. Ubuntu and other distros are available on single CDs, with all of the rest of their programs available in the repositories, so you only have to download multiple CD or DVDs if you're installing to a computer without internet or if you just like having all that stuff offline.

Re:The solution!

[ElleyKitten]

Agreed. Multiple package formats are a non-starter, and shouldn't even be a consideration. The LSB has standardized on a subset of RPM, and that should be the universal package format. The Debian-based distros are harming the package standardization processes, and should learn to play with the other children. Debian's strength is not in the .deb package format, but in the package management tools. Those same tools will still be available upon switching the the LSB RPM format, so I don't see Debian's hangup about cooperating with the LSB.

Why should the Debian-based distros scrap their package format? There's just as many if not more users of Debian-based distros than there are Red Hat based distros, so why is it that the Debian distros have to do all the work to change? Couldn't you just as easily accuse the Red Hat distros of not playing with the other children because they don't want to switch to .deb?

Software installation needs to be simple as: click on package, enter password to authorize installation, use newly installed software.

If you use a package manager, that is how installing programs on Linux is.

Re:The solution!

[M. Baranczak]

The difference between OS X and Linux is that the base OS X installation is huge, and already includes many shared libraries. For example, you can assume that libcurl will be always available on OS X - not so on Linux. You seem to be saying that OS X apps don't depend on shared libraries, which is just not true.

Re:The solution!

[ElleyKitten]

Wine is in the repositories of pretty much every distribution that has repositories (and actually, Oracle is in Ubuntu's repository [ubuntu.com]). As for the specific Windows applications, the discussion is package management, not what programs are available for Linux. If you are very attached to Windows apps that don't work with Wine, then unfortunetly Linux is not for you. We should continue to complain to software developers to make Linux ports and make our own equivalant programs, but that's a seperate issue from the best ways for users to install the software that is for Linux.

Re:The solution!

[ElleyKitten]

Last time I tried Ubuntu apt-get package_i_want failed to locate the program more than half the time.

When you use the command line you have to make sure you spell the package name exactly right, for example "sudo apt-get install flash" won't work, but "sudo apt-get install flash-nonfree" does. Synaptic has a really good search feature that I use when i don't know the exact name. If Ubuntu really doesn't have half the programs you want, then what programs do you use and how do you normally get them?

Relying on distros for your software has lead to the sad state we're in now. I don't rely on Microsoft to hand stamp and prepare every piece of software I used on Windows, and I certainly shouldn't have to do the same on my Linux machine. Until we get a method by which I download a file, click on it, and install a program (regardless of which distro I'm running or which version of GTK I'm running), Linux will lag behind. SEVERELY.

I personally like the package management system. I like having one place to look for software for my system, software that I know has been tested with the programs I likely have on my system, software that I know will update with the rest of my system, software I know isn't spyware. It sounds like it wouldn't work too well, but it really works rather well since there are so many programs in the repositories. Even for the programs that don't want/can't be in the repositories, there's ways for people to install those easily as well. There's java programs that install easily regardless of your Linux, there's autopackage, and some developers just put the program and all the files in a zip file that you can extract and then run where ever you want. There are solutions, they probably need better development, but they're not in terrible shape and that's not the most pressing issue for Linux. Much more important is getting the software people really want on Linux (or at least working really well and easy with wine) and making really good oss equivalents to proprietary software (we need something better than gimp to compare to photoshop) and we also need more device drivers, especially wireless. Those are much more important than package management.