Author: Mayank Sharma
GoboLinux is perhaps best known for its alternate filesystem hierarchy. Unlike Linux’s traditional Filesystem hierarchy, where a program has bits and pieces scattered in several places like /etc, /usr/bin, and /usr/share, each program gets its own directory tree under GoboLinux.
Needless to say, this type of hierarchy aids in package management. Users can install and use multiple versions of the same program or system libraries. In fact, when GoboLinux switched over to the GNU Compiler Collection (GCC) version 3, they still kept older programs running since the filesystem allows for multiple libraries to exist together in peace. Also, uninstallation is just a matter of removing the program directory.
But how does one install applications under such a radical directory structure?
Source is good enough
From the start, GoboLinux’s developers had no intentions of adding another package format like RPM or Debian packages. Furthermore, depending on the popularity of an application it might or might not be available in the RPM or Debian package formats. But all applications will be available as a compressed source tarball.
Hisham H. Muhammad, who developed GoboLinux along with André Detsch, explains that a tarball can simply be unpacked, and then three commands, “configure, make, make install”, should install it.
Dependencies are checked by configure (though not resolved) and the program is linked to the correct libraries present on the system, with no incompatibilities. The developers reasoned that if a source tarball is decently put together, and follows a few GNU conventions, there is no reason why it couldn’t work in any Linux distribution. Therefore, ideally, no distribution formats would be needed at all and a tarball should be good enough. Additionally, if an application is put together using standard build systems, like the GNU buildsystem its installation can be automated.
But not everyone uses the GNU buildsystem to put together their source tarball. This is where GoboLinux’s Compile tool comes into play. Compile works on the basis that there a few standard ways that sources are distributed, and that there are lots of minor variations that can be declaratively specified. Each of these ways is an “execution model”, and support for them is implemented separately. There are three main execution models: ‘compileprogram’ (which covers configure-based files), ‘makefile’ (which covers configure-less make/make-install projects) and ‘xmkmf’ (which covers applications based on the X imake tools).
It’s the job of Compile to download source tarballs, unpack, compile and install them, with a single command. Compile uses a simple file called a “recipe” that describes how an application is to be compiled.
So how does Compile work?
The best thing about Compile is that it uses every project’s own download site to fetch sources. The GoboLinux repository is just used for downloading recipes. Recipes can be downloaded explicitly, using the GetRecipe command, or on-the-fly by Compile, when a compilation is requested.
To download the Joe text editor using Compile, simply do Compile joe. This will first do a case-insensitive search for a recipe named joe. If you want a particular version, you could do a Compile joe 3.1, which will look for a recipe for the 3.1 version of the application. The recipes are available as compressed tarballs and the one for joe is named Joe–3.1-r2–recipe.tar.bz2.
Upon uncompressing the tarballs, Compile finds a file called Recipe and a directory called Resources. The recipe file looks something like this:
# Recipe (MakeRecipe) for Joe by roko, <ro.ko@mcnon.com>, on Wed Oct 27 03:01:32 BRST 2004 # Recipe for version 3.1 by roko, <ro.ko@mcnon.com>, on Wed Oct 27 03:01:32 BRST 2004 compile_version=1.8.0 url="$httpSourceforge/joe-editor/joe-3.1.tar.gz" file_size=381201 file_md5=2a6ef018870fca9b7df85401994fb0e0 recipe_type=configure
Compile follows the URL and downloads the source tarball from the project’s home. The recipe_type option tells Compile the kind of compilation a particular program needs, in this case, the common ./configure, make and make install procedure. The file size and MD5 signature help Compile verify the package after downloading.
Next, Compile turns its attention to the Resources directory. This directory contains two files, one called Description which briefly describes the application and one called Dependencies, which lists all the libraries Joe depends upon. If the Dependencies file lists a library not available on the system, Compile gives you the option to fetch and install the missing library as well.
After downloading and verifying an application and satisfying its dependencies, Compile configures and install the application. Compile has a special flag “–configure-options” where one can supply extra parameters that Compile passes on to the configure script. For example, Compile --configure-options="--enable-debug" joe.
Once the program is compiled, it is placed under its own directory under /Programs. In the case of Joe it’s under /Program/Joe/3.1/. Compile then copies the recipe’s Resources directory inside the target directory at /Programs/Joe/3.1/, and generates four new files inside this directory.
The Architecture file tells which architecture the package was compiled for, BuildInformation stores which dependencies were tracked by ldd (an application that lists dynamic dependencies) after the package was compiled, FileHash stores md5sum for each file in the package and FileHash.sig contains FileHash’s PGP signature. Lucas Villa, one of the main developers of GoboLinux explains that the idea is to have all contributors listed in a “ring” of public keys, so that one can identify who created a given package at the time of its installation.
When all the files and directories have been created and populated, Compile creates all the necessary links throughout the system and the application is ready to use. Villa says that currently GoboLinux’s repositories contain 5,376 recipes for 1,709 different programs.
Write your own recipe
Sitting and watching Compile go about installing Joe, I wondered what it would take to create a recipe. Since Joe’s recipe is for version 3.1, I decided to create one for the latest version, 3.5. I was surprised to find that creating a recipe is pretty straightforward.
There are two methods of creating recipes. The MakeRecipe method, explained later, is for creating recipes for applications that don’t yet have a recipe. The NewVersion method helps create a new recipe based on an existing older version recipe.
NewVersion takes a package’s name and its version number as parameters. NewVersion joe 3.5 will create a template for the new recipe based on the existing recipe which is kept under /Files/Compile/Recipes/. It also modifies the download URL, since generally it only involves replacing the old version with the new one in the URL’s path. In case the location of the application has changed, you can pass the new URL as a parameter when calling NewVersion. For example, NewVersion joe 3.5 http://jaist.dl.sourceforge.net/sourceforge/joe-editor/joe-3.5.tar.gz will use this download URL instead of the existing one in the original recipe. All locally created recipes are kept under /File/Compile/LocalRecipes/.
The new recipe looks like this:
# Recipe for version 3.5 by Mayank Sharma, on Mon Jan 29 16:49:14 IST 2007 # Recipe (MakeRecipe) for Joe by roko, <ro.ko@mcnon.com>, on Wed Oct 27 03:01:32 BRST 2004 compile_version=1.7.1 url="$httpSourceforge/joe-editor/joe-3.5.tar.gz" recipe_type=configure
There’s no better way to check a recipe than installing from it. Once you have the new recipe under /File/Compile/LocalRecipes/, ask compile to install Joe again. This time around, Compile will use the locally available recipe for version 3.5, download the new tarball, which verifies the URL, and then configure and install Joe, in a sandbox. A sandbox allows for testing the application without affecting the base system. After verifying that everything is in order, that all executables are available and all paths map properly, Compile copies the files under the respective /Programs directory and updates system paths. Finally, it also creates a compressed tarball of the new recipe that’s kept under /File/Compile/Store, ready for submission to the GoboLinux repository.
If you want a faster way to verify your recipe, test it with the GenRecipeStore script. This script checks the recipe’s structure, verifies URLs, does basic syntax validation, generates a temporary report under /System/Variable/tmp and creates the compressed recipe tarball in the Store.
Now let’s suppose, that you want to create a recipe for an application Foo, which doesn’t have a recipe. MakeRecipe http://unc.dl.sourceforge.net/sourceforge/foo-app/foo-1.0.tar.gz will not only download the application, but also extract its name and version from the download URL and create a new recipe.-
MakeRecipe will then let you know whether the source was assembled using standard toolchains, like Autoconf, or not. If it’s built with standard tools, it can be compiled with Compile, which will install the application and prepare the compressed recipe.
Resolving dependencies and creating binary packages
If the application Foo, depends on libraries not on the system, Compile will fail to install them. In this case, one will have to create the Dependencies file by hand. To overcome this hurdle, Villa says that the GoboLinux developers have lately been working with a tool called ChrootCompile. “It’s basically a chroot environment which is constructed only by “clean” packages, taken from the recipe store. The base set of packages, which is assumed to exist in all GoboLinux installations, are automatically merged in the chroot, and after the script is finished with its creation, Compile is called from inside that root filesystem. All other dependencies not covered by these basic set of packages needs to be explicitly listed in the Dependencies file, which is parsed by the script responsible of creating the chroot environment.”
Villa also points to the fact that Dependencies can now be specified in a range. For example, GTK+ > 2.8.0 specifies that the application depends upon a version of GTK released after 2.8 and GLib < 1.2.10 means a GLib version earlier than 1.2.10 is required.
They’ve also added support for cross-compilation, to clearly specify which dependencies are only needed when cross-compiling. So Autoconf 2.59 [cross], means that Autoconf needs to be installed in order to cross-compile the package and Xorg [!cross] means that Xorg is only needed if not cross-compiling.
Once you have successfully installed a package, either from an existing recipe or from your own, you can also create a binary package for it. Binary packages install quickly since they are pre-compiled. You can create binary packages for your own use, or distribute them to help users that don’t want to install from source.
To create a binary package, you need to have that application installed on your system. Since we have two versions of the Joe text editor installed, CreatePackage joe 3.5 will create a binary package of version 3.5.
CreatePackage simply compresses the /Programs/Joe/3.5 directory, after scrapping off the settings sub-directory /Programs/Joe/Settings. These settings are replaced by the default settings stored inside /Programs/Joe/3.5/Resources, which is copied from the Recipe tarball and contains the four additional files mentioned earlier.
Installing Binary packages is covered in detail on GoboLinux’s wiki. No matter how you installed a package, either from source or binary, it can be uninstalled using the RemoveProgram tool.
Conclusion
GoboLinux is as much an educational tool as it is an interesting distribution. It took me some time to understand its layout and the various package management tools and how they operate. Once past that hurdle, it hardly took me anytime to contribute my first recipe.
GoboLinux doesn’t have as many packages as other popular distributions, but this will change dramatically as more and more applications are built using standard toolchains. Package management on GoboLinux will become a non-issue as creating recipes will only be a matter of executing the MakeRecipe script.
