Manage Packages Using Stow

dW writes "This article is about Stow, a software installation management utility for Linux that offers a number of advantages over the tried-and-true Red Hat and Debian package management systems. With Stow, you can package applications in standard tar files and keep application binaries logically arranged for easy access."

Unix Directory Structures

[Boss, Pointy Haired]

Whilst this is some way towards making Linux more user friendly (and ultimately gaining acceptance on the corporate desktop), what are the chances of anything being done about the crazy directory layout of a *nix system?

If the answer is nothing, then a suitable GUI for Linux that has the objective to gain corporate desktop acceptance really has to isolate the [l]user from this - i.e. with something like "My Documents".

I like using Linux, but even as a seasoned IT pro, the directory structure and "what goes where" of a *nix system still bugs me.

Have you tried Gentoo's Emerge

[salimfadhley]

One of the reasons I switched from Redhat 8.1 / 7.3 to Gentoo Linux (Beta) was the amazing Emerge package management tool. It combines simple Tar based package files with cool scripts called eBuilds, that automagically fetch and compile all the components I need.

Of course Gentoo is not for everybody... it takes longer to install than Debian (and that is before you have compiled the entire OS from scratch), but for those who are interested in that sort of thing it can be a refreshing alternative.

What about dependencies?

[jointm1k]

The article does not mention anything about dependencies. In my opinion dependencies are almost as important as keeping track of which file belongs to what application. Maybe they should do some more homework and take a look at Gentoo's Portagepackaging system. This system not only compiles a tar/tar.gz/tar.bz2 package, but also retrieves the needed packages (including the dependencies) from their homepages.

How does Stow handle dependencies?

[Max Romantschuk]

I've got some experience with Debian's package management system, and while hard to use for a novice and somewhat complex there is one great benefit: conflict and dependency handling.

Based on the article I didn't quite understand if Stow provides similar services. There were some hints on this, but could someone with experience shed some light on the subject?

Wow?

[j1mmy]

This has about as much flexibility as distributing binaries in a tarball. You can't include installation/uninstallation scripts (what if my application needs to install a cron job?). Everything is forced into /usr/local/stow/PACKAGEDIR and a mess of symbolic links are used to bump everything into the corresponding bin, lib, include, whatever directories. While it may be easier for the software to manage, it creates countless unnecessary files on your drive.

I don't see the benefit.

Why can't it be more like Windows?

[esanbock]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux? It would greatly improve the user experience in Linux. Instead of having to edit 8 configuration files, the user just starts setup.sh or something and the setup asks questions. This is why I like apt-get - one line setup. But every time I download something that's not part of Debian it turn into a horrible experience I wish I would have never had.

Re:Unix Directory Structures

[Sh0t]

You have it backwards friends. It's not a crazy scheme it's a STANDARD and LOGICAL scheme. Programs installed to certain restrictive locations so you KNOW without trying to guess, where certain binaries go, where config files go and so on. It's a standardization and it works very well YOu just aren't used to it. IF you aren't installl from a system package you can put whatever you want, wherever you want, but to keep things orderly you should follow the scheme windows does the same things actually when you use installshield or msi, some files go to program files/program name, some go to /windows some go to a temp dir for settings, some entries go into the registry etc. I think the standard unix method is much tidyer overall, but it may be a bit confusing at first to those who are migrating.

stow is broken

[Ender Ryan]

Don't attempt to use stow for things such as Gnome or KDE. If you attempt this, things will get horribly broken for a number of reasons.

1. Stow requires you configure all the packages into their own directory, which will cause problems with Gnome and KDE. Some packages are easy to configure into one directory, eg. /usr/local, and then install into another, eg. /usr/local/stow/packagename. Others, not so much...

2. Stow has a serious bug in the way it handles directories. If only one package touches a certain directory, it simply creates a symlink to the directory. And then if another package puts something there, it then removes the symlink and does the normal thing. This is a good idea, however, Stow borks this up sometimes, which is bad.

If you're interested, there's a program similar to stow, called srcpkg, at tempestgames.com/ryan. Yes, I wrote it, sorry for the blatant plug. I thought it relevent because I wrote it after experiencing said problems with Stow. FYI, I use srcpkg to manage all the non-system software on my machine, including Gnome, KDE, mplayer, ogg, SDL, and a very large number of other libs and programs.

There's also a number of similar programs on freshmeat. They're all tailored to slightly different needs, but they're all generally better than Stow.

Stow and problems with "make install"

[Anonymous Coward]

As you may or may not know, Stow relies on "make install". And as you may or may no know, "make install" has many weaknesses, maintainability and readability being two of its most glaring problems.

The reason is readily apparent. There is no clean, high-level way to specify installation details to make. Almost always "make install" invokes a messy ad hoc jumble of Bourne shell commands. Make has its own variables. Bourne shell has its variables. You end up double escaping all kinds of items and end up with $$Variables and a plethora of back ticks. The consequence is that install details in a make file tend to look like Perl's uglier cousin. Throw in line extension by escaping line ends with a reverse solidus, and you have the makings of a maintenance nightmare. Try previewing "make install" with "make -n". Not too helpful is it?

How to fix it? I don't know. Perhaps if all Unix vendors could agree on an "installation specification language" -- ISL. Then each vendor's "make" program could incorporate an interperetter for ISL. Other programs like Linux RPM could benefit for this too and incorporate an ISL interpreter because RPM installation specifications are only slightly better than plain Bourne shell (although definitely a step in the right direction).

Re:Have you tried Gentoo's Emerge

[nautical9]

I agree, Gentoo's portage is the best package management I've come across. Not only doe it make ANY package a one-line command that will automatically "download, untar, [patch], configure, make, make test, make install", but it uses system global optimizations for compiles, takes care of all dependencies, and places binaries, libraries, config files and startup scripts all in standard locations. And their "gentoo-sources" version of the kernel has over 70 high-performance patches it includes to the vanilla kernel.org tree (but of course, you don't have to use them if you don't want to, but why not?)

It even has a great /etc/env system for managing environment variables (both bash and csh flavors), so if it needs to install binaries in a non-standard location, you "PATH" is automatically updated to include it.

I don't use Gentoo as a desktop platform, so I can't comment on its X/KDE/Gnome setups, but I'm sure they're just as complete and easy. And although Gentoo may be rather intimidating for a n00b initially, it does have excellent documentation and a great support community at their site [gentoo.org].

Keeping a system up-to-date with the latest and greatest has never been this easy!

Package Management?

[rampant mac]

I know this will probably come off as flamebait, but what is Linux doing to make program installation / de-installation easier?

Sure, package management is wonderful, but it's not something I would recommend for my parents to use. They have enough trouble setting the time on their VCR, and my mother still can't grasp the concept of tabbed browsing after I set her up with Mozilla.

Will Linux ever mature to the point where applications are bundled like they are in OS X... Where a new user can install a program by dragging one icon to install a program, and then drag that same icon to the trash to uninstall it?

How could this be implemented?

Why Hasn't Stow Caught On?

[4of12]

I remember seeing mention of it a couple of years ago on the GNU site.

Was it just that it was not completely developed, or are there other issues that are inhibiting broadscale adoption of stow?

I'm not deliberately trolling, I just wanted to know.

A few random things I do know are:

• how to go from .tar.gz through configure;make;make install
• that Red Hat's rpm [rpm.org] package manager has a 400 page manual and believe the learning curve looks like Mt Everest
• Debian folks swear by apt-get [debian.org]
• writing autoconf [lanl.gov] macros [gnu.org] makes me weary
• gar [lnx-bbc.org] (of Linux BBC fame) looked like an interesting superpackager

No "package-manager"

[Eivind]

Stow is no replacement for a package-manager. It doesn't even *attempt* to do 99% of what a package manager is for. RPM and deb do all of the following, (sometimes trough frontends like urpmi and apt-get) stow doesn't do any of them:

• Solve dependencies automatically.
• automate configuration and building of packages if you prefer to build yourself. (with stow, building is a manual task)
• Warn you if two packages conflict with eachothers.
• Verify integrity of an installed package.
• Handle updates to installed packages.

Infact, all stow does is let you yourself manually deal with all of the above, only you're supposed to run configure with something like --prefix=/usr/local/stow/mypackage and only when you are finished with all this will stow handle the amazing task of making apropriate symlinks from the application-directory to some directory on your path, a task that would take all of 10 seconds to do by yourself.

Pointless utility made by someone who apparently doesn't even understand which job they're trying to do.

The article is also full of typical misunderstandings like:

On Linux systems, most applications are required to be installed in some specific directory (which is usually /usr/local/), to run and function properly; the requirement comes either from Linux or from the application itself.

There exists *no* requirement "from Linux" that any application reside in any particular directory, except the default kernel excpects /sbin/init to exist.

I'm also not aware of a single package that requires living in /usr/local in order to function.

Another way

[tallniel]

I think more people should take a look at tclkit (http://www.equi4.com/tclkit) and the concept of starkits (http://www.equi4.com/starkit). This is a great concept where an application is delivered as one self-contained file (compressed, with an internal virtual file-system). This gets rid of the problem of "installation" all together.

Very cool stuff.

Re:Why can't it be more like Windows?

[Darth Yoshi]

In windows, I double-click setup.exe, a GUI pops up, I pick the destination and off it goes. Why can't someone make something like this for Linux?

Didn't Loki [lokigames.com] write a graphical installer for Linux? I can't access the Loki site from work to check because it's blocked by websense (ha).

Re:Stow isn't Perfect, alas...

[StormCrow]

A quote from http://cr.yp.to/slashpackage/finding.html

``Software should never ever assume it knows where to get files from,'' someone once wrote. (He says I'm taking his quote out of context, so I won't identify him here.)
Here was my sarcastic response:

Yes, that's a very important principle!

Let's take, for example, csh, which uses /etc/csh.cshrc and /dev/log and /bin/sh and many other files. The reason that all those filenames are listed in /etc/csh.conf is so that they can be changed.

Now, some people want to move /etc/csh.conf itself. That's why csh looks for the /etc/csh.conf filename in a hashed /etc/registry.db file.

Of course, on some machines, we need to move /etc/registry.db. That's why the registry filename is listed in a COMPILEDFREGISTRY environment variable.

There's still the possibility of conflict with previous uses of the COMPILEDFREGISTRY variable. That's why the name of that variable is listed in /etc/fregistry_variable_name.txt.

You say you want to move /etc/fregistry_variable_name.txt? You fool! We have billions of programs that /etc/fregistry_variable_name.txt at the top of main(). Everything _else_ has to be configurable, obviously, but /etc/fregistry_variable_name.txt isn't going anywhere.

Dependencies should use open format ...

[vinod]

It is good to be able to use independent directories for applications that are installed at the site (i.e. not part of the distribution.) And RPM can accomodate such independent directories as well. Within the independent tree, the applications should standardize the directory structure just like in unix: ./etc etc.

Then, putting symbolic links in various directories is bad idea. Instead, users could explicitly 'subscribe' to the directories. A special, user specific ./bin directory can be used to keep the subscriptions to bin directories of subscribed packages.

Bad thing about RPM is that it uses a centralized DB for tracking dependencies, which can't be manupulated by hand. Instead, it could evolve to use 1. open format based on XML 2. Put the dependencies as part of independent directory tree of the package.

In most cases, it is sufficient that dependencies be evaluated dynamically. After all, sysads know what they are doing.

-vgk

Re:Unix Directory Structures

[Permission Denied]

You must be trolling.

Take any file on a modern Linux system. Any file at all. Explain to me its purpose and I'll tell you where you'll find it in the filesystem. A keyboard map? /usr/share. A subprogram to be executed by a program, not by the user? /usr/libexec.

Take any file on a modern Linux system and tell me its full path. I'll tell you what it does, and I'll probably be able to tell you what package it comes from. I can also tell you if you need it or if you can get rid of it. /usr/X11R6/lib/X11/rgb.txt? Rgb color database to define textual names for colors, from XFree86. /usr/local/share/automake/elisp-comp? Support file for automake to integrate with emacs, installed by user after system installation time, from GNU automake. /usr/local/lib/libjpeg.so? Jpeg image library, installed after system installation time, from the JPEG group.

Take any file on a modern Windows system, and you won't be able to do anything with it. C:\winnt\keyacc32.exe? Does that come from MS or from a third party and what is it used for? C:\winnt\system32\getstart.gif? An image that says "Get started with Beta 2" in the Microsoft Arial font. Is this a remnant from a win2k beta and if so, why is still here in a production post-sp3 win2k pro system? Or does it come from some third party? Try figuring out what files are necessary on a Windows system and what's cruft. You certainly won't be able to get a Windows install to fit in less than ten megs because all these files are spread out and undocumented - however, you know it's possible to get a Windows image in only a few megs because Microsoft does it (miniwindows during Windows installation). Getting a FreeBSD install to fit in only a few megs is not a problem - just did it for a compact-flash-based system and it's not hard.

Anyway, this scheme that stow uses is very useful. Djb also has something like this in mind, but his way of doing it is not very elegant IMHO. I've been using something like stow for all my machines for the past three years or so, but it was just a 50-line shell script. Keep all software installed in /opt. Like /opt/emacs/emacs-21.1 with a symlink /opt/emacs/default which points to "current" version (that way, upgrades can be done with just changing a symlink, and downgrades are also just changing a symlink - you don't have to learn some tool's syntax, and, more importantly, the admin that replaces me won't have to learn some esoteric system because you can figure out my system in 30 seconds). All that my shell script does is make symlinks from the /opt/cvs/default/bin to /usr/local/bin, from /opt/python/default/lib to /usr/local/lib, etc. This way, when you run a poorly-written autoconf script, it will always find the requisite packages (because people always assume the required package is in /usr/local) and my users don't have to deal with $PATH. In addition, I can run this:

find /usr/local -type f -print

And this tells shows me all the stuff that I've written locally for the particular system (anything in /usr/local that's not a symlink must have been written by me).

Never run into any dependency problems, upgrades are "ln -s," uninstalls are "rm -rf."

Re:How long

[virtual_mps]

how long is it going to take for something on linux to do what installshield does on windows?

And what would that be? Guess what things are installed and try to remove them properly? This is a tool for a system manager to keep track of what's installed on a system--windows has no advantage whatsoever for that. (Multiple programs overwriting the same dll, registry entries spreading everywhere, lack of seperation of users in many programs, etc., mean that there's a lot of work to do in windows.)

I admit not knowing about linux, and am doing little to change that.

If you don't know and don't care, wouldn't it be easier to just not post?

Re:Have you tried Gentoo's Emerge

[dattaway]

The perceived problems of customized building of an entire may be a strong advantage. It requires all software to be of highest quality or the bugs will show. A community that encounters problems encourages debugging or switching to higher quality packages and not sweeping problems under the rug.

Checkinstall DOES use makepkg

[izto]

And has used it from the very beginning. So you could start recommending it right now ;-).

Checkinstall was initially developed in the time when makepkg had no command line options to answer the questions about symlinks and permissions, so Checkinstall uses by default a makepkg from Slackware 7.1 which was hacked to not ask those questions and instead assume specific answers to them. And it has been updated with the changes and patches introduced in Slackware 8.0 and 8.1 (i.e. the "slack-desc" file with handy ruler and all)

Aditinally, you can still use Slackware's native makepkg if you set the MAKEPKG variable in checkinstall.

Re:Checkinstall DOES use makepkg

[izto]

Yes, I found out about this recently, and already patched checkinstall to use the right tar version if available. The issue was kind of confusing, though: having two different tar binaries in a Slackware system, with "tar" being the wrong one to use when building a Slackware package.

Feel free to contact me with any other issues about the way checkinstall creates the Slackware packages. Building proper packages for the system it is running on (Be it Slackware, Debian or RPM based) is very important for me. The Debian folks are particularly picky about this and we've managed to get checkinstall to build mostly Debian-compatible packages already, for example :-)