AutoPackaging for Linux

Isak Savo writes "The next generation packaging format for Linux has reached 1.0. With Autopackage officially declared stable, there is now an easy way for developers to create up to date, easy installable packages. There are lots of screenshots available including a flash demo of a package installation."

The purpose of autopackage

[FooBarWidget]

No doubt lots of people will have all kinds of questions about autopackage, such as:
- "What idiots!! Another packaging format is the last thing we need!"
- "What's wrong with apt-get?"
- "Everybody should use Gentoo!"

Slashdotters are highly encouraged to read the autopackage FAQ [autopackage.org]! Our project has existed for over 2 years now, and many people have asked us those questions. In short: autopackage is not meant to replace RPM/DEB/apt-get/etc.

If you have more questions, feel free to come over at #autopackage at irc.freenode.net
We'll be glad to answer your questions

Some FAQ entries

[IamTheRealMike]

So much for sunsite having plenty of bandwidth and fast servers! Well, here are some select pieces from the FAQ:

# What is autopackage?

For users: it makes software installation on Linux easier. If a project provides an autopackage, you know it can work on your distribution. You know it'll integrate nicely with your desktop and you know it'll be up to date, because it's provided by the software developers themselves. You don't have to choose which distro you run based on how many packages are available.

For developers: it's software that lets you create binary packages for Linux that will install on any distribution, can automatically resolve dependencies and can be installed using multiple front ends, for instance from the command line or from a graphical interface. It lets you get your software to your users quicker, easier and more reliably. It immediately increases your userbase by allowing people with no native package to run your software within seconds.

# Is autopackage meant to replace RPM?

No. RPM is good at managing the core software of a distro. It's fast, well understood and supports features like prepatching of sources. What RPM is not good at is non-core packages, ie programs available from the net, from commercial vendors, magazine coverdisks and so on. This is the area that autopackage tackles. Although in theory it'd be possible to build a distro based around it, in reality such a solution would be very suboptimal as we sacrifice speed for flexibility and distro neutrality. For instance, it can take several seconds to verify the presence of all required dependencies, something that RPM can do far quicker.

# Why a new format? Why do the RPMs I find on the net today only work on one distro?

There are a number of reasons, some obvious, some not so obvious. Let's take them one at a time:

• Dependency metadata: RPMs can have several types of dependencies, the most common being file deps and package deps. In file deps, the package depends on some other package providing that file. Depending on /bin/bash for a shell script is easy, as that file is in the same location with the same name on all systems. Other dependencies are not so simple, there is no file that reliably expresses the dependency, or the file could be in multiple locations. That means sometimes package dependencies are preferred. Unfortunately, there is no standard for naming packages, and distros give them different names, as well as splitting them into different sized pieces. Because of that, often dependency information has to be expressed in a distro-dependent way.
• RPM features: because RPM is, at the end of the day, a tool to help distro makers, they sometimes add new macros and features to it and then use them in their specfiles. People want proper integration of course, so they use Mandrake specific macros or whatever, and then that RPM won't work properly on other distros.
• Binary portability: This one affects all binary packaging systems. A more detailed explanation of the problems faced can be found in Chapter 7 [slashdot.org] of the developer guide.
• Bad interactions with source code: because the current versions of RPM don't check the system directly, they only check a database, it makes it hard to install them on distros like Gentoo even when they only use file deps. Similar problems arise if you compile things from source.

There are various reasons why a new format was created rather than changing RPM

Mirror

[anandpur]

Autopackage [nyud.net]

Re:Some FAQ entries

[IamTheRealMike]

Why bother?

# What's wrong with centralized repositories, apt style?

The system of attempting to package everything the user of the distro might ever want is not scalable. By not scalable, we mean the way in which packages are created and stored in a central location, usually by separate people to those who made the software in the first place. There are several problems with this approach:

• Centralisation introduces lag between upstream releases and actually being able to install them, sometimes measured in months or years.
• Packaging as separate from development tends to introduce obscure bugs caused by packagers not always fully understanding what it is they're packaging. It makes no more sense than UI design or artwork being done by the distribution.
• Distro developers end up duplicating effort on a massive scale. 20 distros == the same software packaged 20 times == 20 times the chance a user will receive a buggy package. Broken packages are not rare: see Wine, which has a huge number of incorrect packages in circulation, Mono which suffers from undesired distro packaging, etc
• apt et al require extremely well controlled repos, otherwise they can get confused and ask users to provide solutions manually : this requires an understanding of the technology we can't expect users to have.
• Very hard to avoid the "shopping mall" type user interface, at which point choice becomes unmanagably large: see Synaptic for a pathological example of this. Better UIs are possible but you're covering up a programmatic model which doesn't match the user model esp for migrants.
• Pushes the "appliance" line of thinking, where a distro is not a platform on which third parties can build with a strong commitment to stability but merely an appliance: a collection of bits that happen to work together but may not tomorrow: you can use what's on the CDs but extend or modify it and you void the warranty. Appliance distros have their place: live demo CDs, router distros, maybe even server distros, but not desktops. To compete with Windows for mindshare and acceptance we must be a platform.

# What's wrong with NeXT/MacOSX style appfolders?

One of the more memorable features of NeXT based systems like MacOS X or GNUstep is that applications do not have installers, but are contained within a single "appfolder", a special type of directory that contains everything the application needs. To install apps, you just drag them into a special Applications folder. To uninstall, drag them to the trash can. This is a beguilingly easy way of managing software, and it's a common conception that Linux should also adopt this mechanism. I'd like to explain why this isn't the approach that autopackage takes to software management.

The first reason is the lack of dependency management. Because you are simply moving folders around, there is no logic involved so you cannot check for your apps dependencies. Most operating systems are made up of many different components, that work together to make the computer work. Linux is no different, but due to the way in which it was developed, Linux has far more components and is far more "pluggable" than most other platforms. As such, the number of discrete components that must be managed is huge. Linux is different to what came before not only in this respect, but also because virtually all the components are freely available on the internet. Because of this, software often has large numbers of dependencies which must be satisfied for it to work correctly. Even simple programs often make use of many shar

Re:Aren't there enough

[FooBarWidget]

I knew someone would say that. Read our FAQ. Mike posted a part of it below. Please read our website for the full FAQ once the Slashdotting is over.
And we'll be available at #autopackage at irc.freenode.net if you have questions.

Mirror of the autopackage website

[FooBarWidget]

Mike setup a mirror of the autopackage website! It's here [dur.ac.uk]. The FAQ is also available on that mirror.

Re:Where does everything get autopackaged to?

[FooBarWidget]

By default, autopackage either installs to /usr or to $HOME (depending on what the user wants). If you really want it to use another prefix, you still can. The reason we use /usr instead of /usr/local is because there are many broken distributions that don't setup paths for /usr/local correctly. And /usr is the standard for many/most distributions.

There's a mirror of the autopackage website's information here [dur.ac.uk].

Re:Please let non-root people install

[stefanlasiewski]

The only thing I'd like to see in a package manager is to allow non-root users to install software

Check out the Flash demo. They actually demonstrate that capability-- the application installs packages under $HOME/.local/lib $HOME/.local/share, etc. I dislike cluttering $HOME with $HOME/lib, $HOME/share, $HOME/man, etc. -- installing these pieces under a dot directory to keep them hidden & organized is a great idea.

But I am also curious about allowing groups to install software under /usr/local or /usr/department .

Re:MIRROR IS HERE

[sirReal.83.]

That's not a complete mirror :(

This is missing. [dur.ac.uk]

mirrordot mirrors of article and images

[Nallep]

http://mirrordot.org/stories/16feae672a6b23e1a6bde ee65c18984b/index.html [mirrordot.org]
http://mirrordot.org/stories/4b8dff883e128d08839c1 323793affab/index.html [mirrordot.org]
http://mirrordot.org/stories/5f34bd546afaa065409b5 e4822a1ed01/index.html [mirrordot.org]

Re:Aren't there enough

[diegocgteleline.es]

yes, but that's not the main problem IMO. Package formats are often tought as the number 1 offender when it comes to inter-distro compatibility, but that's not the main problem. The main problem is that package mainteinance happens at distro-level, instead of developer-level. What we need is to move as much mainteinance work to the developers. It's the number 1 cause of problems: A package in debian may require "libfoo", but in fedora it may depend on "foo" or "foo+randomstring". If all those things would be done at developer level, they'd be more coherent, and inter-distro compatibility would be greater. Package format would be irrelevant.

And this is why autopackage is great. It doesn't tries to replace apt/rpm/gentoo, it just tries to be a good way of distributing software, and encourages developers to create their own "autopackage packages" so they work in every distro

Mirrordot of Flash

[vectorian798]

Flash Demo [mirrordot.com] Screenshots [mirrordot.com]

I have to say this is like a godsave for linux. Most layusers will want some easy installation like this instead of using something like Yum (even if it is a GUI front-end to yum like GYUM). This is one giant step towards a viable desktop linux - and I believe that it isn't a replacement for apt/yum/[INSERT YOUR FLAVOR HERE] but uses them under the hood.

Before everyone starts bashing it and says that apt or emerge or whatever they use is the way to go, seriously think about it - one click installation, from a FRIENDLY user-interface, and easy to manage system for installing and uninstalling programs. Now if this were part of the base install on many distributions and some sort of standard was established (seriously, we need standards) I can probably convince my scared-of-Linux-because-it-is-hardcore friends to actually try Linux out.

testing, unstable, experimental and apt-get.org

[free2]

But what do you do if the software you need are not included with you distro?
I usually don't have this problem with Debian (especially with the huge testing and unstable distros), and even the Debian experimental repository is signed.

But if I did, I woud look for some known Debian maintener unofficial packages in apt-get.org .

Re:Conflicts with existing package managers

[FooBarWidget]

Autopackage is meant to be an add-on that coexists with your native package manager. It's not meant to replace anything. And the conflicts you're talking about will only occur if you try to install the same deb and autopackage on purpose.
For post-1.0, native package manager integration is on our todo list.

Re:When it ties in with RPM, I'll take a look

[tajmorton]

That's planned for >1.0 (Integration with Native Package Managers, e.g., rpm, yum, apt-get, etc).

Re:The purpose of autopackage

[FooBarWidget]

Maybe you didn't take a good look at our website. We have had a solution for binary problems for a long time now, and it's called apbuild. If you have any specific questions, feel free to come over at #autopackage at irc.freenode.net

Re:That's right. apt-get works.

[Master of Transhuman]

And exactly how does installing something your RPM database doesn't know about NECESSARILY break your system?

Are you saying you NEVER install anything other than an RPM?

Including third party apps and miscellaneous small utilities? We're not talking OS core here, remember.

Re:What about Java applications?

[FooBarWidget]

Well, I tried packaging a Java app before but I gave up. It's just not possible to provide an easy installation experience, because of the Sun JVM, which is proprietary. I cannot auto-install the JVM because of the license, which prohibits distribution. You *may* be able to get away with this if you compile your app with GCJ but GCJ does not have a stable ABI...

Re:plenty of UI and Usability mistakes

[IamTheRealMike]

Yes we know the bouncing progress bars suck. I yelled at people about that a few days ago. Expect them to disappear as time goes on.

Dialogs same size - not sure there's any point to this. Dialogs that don't fit their content are just weird. Making them all the same window is technically very hard because of the need to run different programs to re-authenticate etc. Yes yes, excuses, but software is compromise.

No cancel button: where is the cancel button? You cannot cancel an install, so there should not be one. The closest I can see is Hide which is different.

Don't ask for passwords: you need the root password for software to be shared, and various things work better when software is installed system-wide. The "No password" button is there for setups where you are not the administrator rather than for home users. Long term authentication-less software installation should happen but it's a big step that neither MacOS X nor Windows have fully taken yet, it requires a lot of thought and technology to ensure it's done correctly.

Progress meters: yes, from a pure UI perspective you are correct. However it's hard to know how long things will take because the packager has complete freedom to do whatever they like. It's not deterministic, in other words. Again, compromise. Long term this whole UI will disappear in favour of drag/drop installs anyway so there's not much point in polishing it too much.

The uninstaller will prompt you for the root password once you hit "Remove". It's just not shown in the Flash demo.

Thanks for your comments!

Re:Please let non-root people install

[hawk]

Err, are there really package systems that *don't* let you do this?

Under the FreeBSD ports system, you simply set $PREFIX to wherever you want things to happen. While this is probably not obvious to grandma, it would just come to a script to

1) add $HOME/.programs to the relevant paths
2) add the "mypackage" script to $HOME/.programs/bin which sets $PREFIX and passess the rest of the argments.

It's been a few years since I've used dpkg, but I'm pretty sure that it had similar options, and I always assumed that rpms could do something similar.

hawk

Re:Yes, we need this!!

[IamTheRealMike]

The autopackage FAQ has "what's wrong with NeXT/MacOSX style appfolders", but it seems to consist mostly of hand-waving and straw men. They don't seem to understand how NeXT/Mac apps work, e.g., w.r.t. linking.

Could you elaborate please? Of course it's possible to build a system that uses appfolders, NeXTStep is one, however:

• Doing that on Linux is hard, because of dependencies and the lack of a large platform. I'm not interested in theoretical systems, I'm interested in what I use which is Linux. I'm also not interested in MacOS X because I don't use that, and because it's proprietary which is totally not the way forward for our society.

• Linux isn't designed to allow appfolders easily. The freedesktop.org specs are oriented around the concept of "drop a file in this directory, update a cache". This works well for package managers, less well for appfolders. It's done that way for valid reasons: the menu system is designed with administrators, windows compatibility and internationalization in mind. The appfolders menu system isn't really designed per se, it's just a convention to put stuff in /Applications (and a shockingly large number of apps break if they aren't in /Applications).

• The issues with linking etc are well known. Yes I understand how MacOS X linking works, that includes how frameworks work and also the weak linkage Apple use. Before suggesting we don't understand linking how about reviewing things like apbuild and relaytool. Nonetheless, new libraries are manufacturered at a quite astonishing rate on Linux: the openness lends itself to massive and extremely fine grained code reuse. That means we can't just rely on having a bigass platform, though one would be nice. Dep resolution is still important simply to prevent your OS becoming totally obsolete within a year.

The only person who has really done serious work with appfolders on Linux is Thomas Leonard, an extremely smart man who was behind the ROX desktop. Note that appfolders aren't a Mac or NeXT invention, they were done by RISC OS back when I was a wee kid.

As Thomas found out, dependencies are kind of a tricky problem with appfolders. His solution was ZeroInstall, a very nice piece of work I must say. However it had issues, as software often does, and now he is designing something called the "Injector". This has quite a few concepts similar to autopackage, namely management of interfaces and dependencies. So in the process of "fixing" appfolders to have all the features people wanted, he ended up with something that vaguely resembles systems like autopackage and apt-get. Note: that doesn't mean to imply that the Injector is useless or anything, it's not and has some good ideas. I encourage people to check it out!

Just be aware that appfolders tend to evolve into installers eventually. They did on DOS, RISC OS, and of course on MacOS X as well (iTunes comes in an installer and it's not the only one ....)

Re:The purpose of autopackage

[TheRaven64]

I like the OS X (and GNUstep) way of handling things, but you are wrong about one thing. Deleting a .app bundle does not delete the configuration files. File associations are stored in a .plist file in each .app bundle, and must be periodically refreshed (fine, the system handles this), but other settings are stored in /Library/ or ~/Library/, and stay around after you have removed the application. This is required due to the way in which applications are upgraded in OS X - delete the old one, drag the new one in - however there is no mechanism provided for removing orphaned config files. Similarly, there is no mechanism provided for uninstalling applications packaged as .pkg bundles (e.g. Apple's X11).