Nix fixes dependency hell on all Linux distributions

A next-generation package manager called Nix provides a simple distribution-independent method for deploying a binary or source package on different flavours of Linux, including Ubuntu, Debian, SUSE, Fedora, and Red Hat. Even better, Nix does not interfere with existing package managers. Unlike existing package managers, Nix allows different versions of software to live side by side, and permits sane rollbacks of software upgrades. Nix is a useful system administration tool for heterogeneous environments and developers who write software supported on different libraries, compilers, or interpreters.

Why provide yet another package manager? Because current package managers fall short in the upgrade cycle. Everyone gets burnt by software dependencies, at some point. In particular, with a major release of any given distribution, many people choose not to upgrade until it is time to do a fresh install. With Nix, upgrades are always safe: they don’t overwrite previously installed packages. This means previous versions will continue to work, and you can easily roll back.

Nix started as an academic project at Utrecht University in the Netherlands. The name is tongue in cheek; in Dutch it means “nothing.”

The problems: destructive upgrades, software versioning, heterogenous environments

All popular package managers, including APT, RPM and the FreeBSD Ports Collection, suffer from the problem of destructive upgrades. When you perform an upgrade — whether for a single application or your entire operating system — the package manager will overwrite the files that are currently on your system with newer versions. As long as packages are always perfectly backward-compatible, this is not a problem, but in the real world, packages are anything but perfectly backward-compatible.

Suppose you upgrade Firefox, and your package manager decides that you need a newer version of GTK as well. If the new GTK is not quite backward-compatible, then other applications on your system might suddenly break. In the Windows world a similar problem is known as the DLL hell, but dependency hell is just as much a problem in the Unix world, if not a bigger one, because Unix programs tend to have many external dependencies.

Also, destructive upgrades make it hard to undo, or roll back, an upgrade. Unless you or your package manager makes a backup of all the files that got replaced, you cannot easily undo an upgrade.

Finally, while the package manager is busy overwriting all the files that belong to a package, your system is temporarily in an inconsistent state in which a package may or may not work properly. Hit the power switch on your computer halfway through your next OS upgrade and see if the system will still boot properly!

Usually, an upgrade of a package will make the older version disappear. Sometimes a package manager allows a few versions next to each other — say gcc-3.4 and gcc-4.3. However, this only works if the packager has arranged for this by making sure that the two versions install to different paths. What if you want to test gcc-4.0.3 without disrupting your system? Or if you want to test software using different compiler, library, or interpreter versions and combinations? What if you want to try the latest beta version of an application without risking your existing installation?

Then there is the problem of multi-distribution support. Developers and systems administrators have no way of knowing what combinations of kernel, libraries, and packages a user is running. When a user tries to install some software and complains about a missing library, you can suggest he try to find the appropriate packages, but you have not tested them in the same combination of dependencies. You can ask him to build from source, but he may not be using appropriate versions of libraries and compiler. Even if you are lucky enough to work for an institution that standardizes on a single distribution, you may find users’ libraries are outdated and do not support the software you want to deploy.

Existing package managers do well enough in furnishing stable systems because they depend on a long period of testing by a lot of people. However, when users need more recent software, because of a bug or some missing functionality, they turn to so-called “testing” or even “unstable” packages, which often come with a range of dependencies that also get updated on the users’ systems, potentially introducing instabilities in other software that depends on those components.

Nix has a more graceful approach that lets different versions of software coexist. Rather than installing packages in global locations such as /usr, it instead stores each package in its own directory under /nix/store. The top-level directory for each package contains a cryptographic hash based on all the inputs used to build the package. That creates a unique identifier such that multiple versions of a package don’t interfere with each other, and different packages on your system can use different versions of some dependency without causing a conflict. This means that you can atomically upgrade and roll back packages.

Getting started with Nix

Ironically, the best way to install Nix is to fetch and compile it from source using the normal ./configure ; make ; make install commands, as root, as this will work for any system, including Mac OS X and the BSDs. You’ll need a standard GNU g++ build environment and curl installed to bootstrap Nix. Once Nix is installed it no longer depends on those tools. Nix stores packages and some metadata under the directory /nix, so make sure there is enough space there. If / doesn’t have enough space, either use a special partition for /nix , or bind-mount /nix to a filesystem that has 5 to 50GB of space available. You should also add the file /usr/local/etc/profile.d/nix.sh to your bashrc file. It sets up some environment variables (like $PATH) that Nix needs in order to work properly.

Once Nix itself is installed, you can start using it to install software. You can get Nix Packages (Nixpkgs), a large collection of packages for Nix, by downloading it from the Nix Web site or checking it out from its Subversion repository. However, the easiest way to use it is to subscribe to the Nixpkgs channel, which is just a way to distribute the latest version of Nixpkgs automatically to users:

nix-channel --add http://nixos.org/releases/nixpkgs/channels/nixpkgs-unstable
nix-channel --update

Nixpkgs consists of a large set of Nix expressions, which are Nix’s language for describing how packages are built, similar to Gentoo ebuilds. Nix is essentially a source-based package manager, like Portage, but Nix will automatically download precompiled binaries of packages if it can locate them by their hash values. The Nixpkgs channel contains a lot of precompiled binaries, which speeds up package installation significantly.

Once nix-channel has downloaded Nixpkgs, you can see what packages are available. The nix-env tool acts much like yum and apt-get. You can see a list of packages by running nix-env -qa '*', or to list all the packages named Firefox, nix-env -qa firefox. That should return a list like:

firefox-2.0.0.17
firefox-2.0.0.17-with-plugins
firefox-3.0.4
firefox-3.0.4-with-plugins

You can add the -s flag to see which packages are pre-built in the
Nixpkgs channel. An S in the listing indicates that the package can be downloaded in binary form.

--S firefox-2.0.0.17
--- firefox-2.0.0.17-with-plugins
--S firefox-3.0.4
--- firefox-3.0.4-with-plugins

Installing a package with dependencies

To see the power of Nix, let’s say you want to install Firefox 2, with all its dependencies and a bunch of plugins, such as Flash, without overwriting anything on your existing system. Just type nix-env -i firefox-2.0.0.17-with-plugins and Nix downloads and compiles these packages. Even though firefox-2.0.0.17-with-plugins is not available precompiled in the channel, most of its dependencies are. Nix will download binaries where possible, and compile from source otherwise. Firefox has lots of dependencies, from the GNU C library to GTK. Your distro will already have most of these, but Nix installs its own copies to make sure that you get exactly the right versions and that they don’t interfere with rest of your system. After all, nowadays, why not sacrifice a bit of disk space to get predictability?

So now you should be able to run the Firefox you just downloaded by simply typing firefox on the command line. (You may need to set the FONTCONFIG_FILE environment variable to /etc/fonts/fonts.conf to make sure it can find your system’s fonts.)

Now let’s try upgrading to Firefox 3, either with the command nix-env -i firefox-3.0.4-with-plugins, or nix-env -u firefox, which would have made Nix pick the latest version. After this, you should have Firefox 3 installed, but the old Firefox is not gone — it has not been overwritten. The two versions automatically end up in different paths:

/nix/store/vskr06rlblihz22...-firefox-2.0.0.17-with-plugins
/nix/store/w1i05b7s30zqz...-firefox-3.0.4-with-plugins

You can see from this why packages do not interfere with each other. You can run either version of Firefox cleanly from the above paths.

Outside of Nix, you can still use the installed packages on your native system. The Nix packages are extra and independent.

Profiles

One can invoke a program directly from its path, or by adding paths to the system’s search paths, but the long path names obviously make this ugly. That is why nix-env automatically generates trees of symlinks to the installed packages automatically. To make this work, you would have in your $PATH something like ~/.nix-profile/bin, which points at one of these trees of symlinks. This is incidentally how Nix can do atomic upgrades and rollbacks: it just generates a new tree of symlinks to the new packages, then flips ~/.nix-profile to point at the new one.

You can have more than one of these trees of symlinks, or profiles, each containing different activated packages. This lets you set up a profile for a specific version of any application within a profile. For instance, you could set up a profile to try out the latest version of Firefox by entering the command:

nix-env --profile my-firefox-test -i firefox-3.0.4-with-plugins

then running ./my-firefox-test/bin/firefox (which is a symlink to the actual location of Firefox 3 in /nix/store) — it won’t affect your ‘normal’ Firefox in ~/.nix-profile/bin.

Rollback

Since installing Firefox 3 did not overwrite the old Firefox 2, it is straightforward to roll back changes. If you want to go back to Firefox 2, all you need to do is say nix-env --rollback to return to the previous situation.

Of course, you’ll probably want to get rid of old versions eventually. If you are sure you do not need to roll back any more, you can say nix-collect-garbage -d to delete all unneeded packages from the system — in this case Firefox 2 and any dependencies that are not used by some other (still enabled) package.

Creating your own packages

Finally, Nix comes with a domain-specific language for writing package descriptions. This functional yet straightforward language provides an elegant way of providing and maintaining packages with their assorted dependencies. A package for PHP with dependencies, looks like:

# Nix expression for building PHP. This is a function that given some arguments
# (like flex and libxml2) builds an instance of PHP in the Nix store.

# These are the function arguments...
{stdenv, fetchurl, flex, bison, libxml2, apacheHttpd, postgresql ? null, mysql ? null}:

# and this is the result of the function: a build action.
stdenv.mkDerivation {
  name = "php-5.2.4";
  src = fetchurl {
    url = http://nl3.php.net/distributions/php-5.2.4.tar.bz2;
    sha256 = "1h513j7crz08n7rlh8v7cvxfzisj87mvvyfrkiaa76v1wicm4bsh";
  };

  inherit flex bison libxml2 apacheHttpd;

  builder = ./builder.sh;

  buildInputs = [flex bison libxml2 apacheHttpd];

  patches = [./fix.patch];
}

This Nix expression requires libxml2, among other packages, and allows a choice of SQL back ends on installation. Nix expressions come with a large number of commands for package definition, which usually makes package descriptions compact and easy to understand. At build time the environment is clean and isolated from all other libraries, so dependencies always have to be included explicitly. A missing dependency will always be detected, which is a great feature for software developers.

The ability to pass options to the package configuration allows for flexible package management. For instance, you could build a package with GUI support on desktops and without GUI support on servers. Stripped-down versions of software and even documentation are feasible with Nix. More complex examples can be found in the package source tree.

Conclusion

With Nix, you can use a rock-solid and stable distribution like Debian stable or Red Hat Enterprise Linux as a foundation. When you need to deploy recent software, not provided by the distribution, Nix provides support for multiple versions, complete dependencies, and rollback. Nix allows you to escape dependency hell by creating a predictable system for software managment.