Linux/VAX Porting Project: Maybe the last port to a classic computer

You can find Linux running the latest generation of handheld gadgets, transforming game consoles, and it will soon be making winners or suckers out of Las Vegas casino visitors. We’ve become so accustomed to hearing about Linux running non-traditional computing devices that it’s almost a shock to run across a port of Linux for, of all things, an actual computer.
Since September 1998, Andy Phillips, Kenn Humborg, Dave Airlie, and a contributing cast of other developers have been toiling away in their spare time to bring Linux to the DEC VAX architecture.

VAX (Virtual Address eXtension) computers made quite a splash in the computing world of the mid-1970s. The first 32-bit computer with virtual memory to ship from Digital (now owned by Compaq), VAXes gained a well-deserved reputation for being solid, stable computer workhorses. The first VAX commercial model, the 11/780, shipped in 1978.

Digital’s VAX — coupled with its VMS (now OpenVMS) operating system — revolutionized modern computing and networking, contributing a great deal to the expansion and development of what would eventually become the Internet. But this article isn’t about what VAX has done (check out Compaq’s 20th anniversary special, in PDF format, for that); rather, it’s about what VAX will do as the Linux/VAX Porting Project continues along on its quest to port Linux to the VAX hardware.

I was cruising along during my daily NewsForge story-posting schedule, when I came across a link to an email message from Phillips, announcing that Airlie had successfully booted Linux to a shell prompt on his VAXStation 3100m38.

When I saw that announcement, I figured that I could maybe get a nice 500-word piece about this project, so I fired off a few questions by email to Phillips, who graciously provided some fascinating — not to mention comprehensive — information on the project. Phillips sent the message along to fellow contributors Humborg and Airlie, who were kind enough to provide additional information and insight from their perspectives. It seemed that condensing this information into a few paragraphs would have done a great disservice to the Linux/VAX Porting Project, so I decided to reprint their answers in an interview format. Enjoy.

Q: When did work begin on porting Linux to the DEC VAX architecture?

Andy Phillips: The short answer is that the current project started in the summer of 1998. This followed on from an earlier project started by some CS students in Canberra, Australia, in 1996/7, I think. However, by the time I became interested in the port, that project had mostly stalled, so I decided to just start work by myself. That was September 1998. The only substantial item carried forward from the earlier project was the a.out cross development environment, which has been completely superseded now by the newer ELF-based cross compiler. When they needed to shut down their mailing list/webserver, I organized the current setup.

Q: How did this effort get organized; what events led to the creation of the Porting Team?

AP: The Porting Team is not really as formal as the name sounds. Kenn Humborg and Dave Airlie just joined in around the end of 1999, and since they and I have done most of the coding, we have become the “porting team.” Other people have contributed the odd patch here and there, but all the major coding has been done by the three of us.

As is usual with these things, it all sort of just happened. It’s never been a high-priority item for any of us, and certainly it has always been a part-time affair — and we have all taken large stretches of time off at one point or another, as I have spent much time out of the country, working on environmental projects, and both Dave and Kenn go through busy periods at work, and during these times we are not able to work on the port at all.

Kenn Humborg: And I’m getting married real soon now, which reduces my hack time considerably.

AP: This is why it has taken so long from the start. If we had all sat down in a room together at the beginning, I guess we would be where we are now in after about a month or so.

But seeing as this is a just-for-fun project, and unlikely to be of real use to more than a few people, the amount of time it has taken has never really bothered me. For me, it’s been a great learning exercise.

KH: It’s basically fun. It’s architecture that I know fairly well
from my university days 10 years ago and it’s very nice to work
with.

Q: How many other developers are contributing to the port?

AP: Well, things are changing now that we have a shell prompt. Now, people who are happy with application development, or C library work, but not necessarily with kernel development, can join in and lend a hand. I expect that we will gain extra help, as I know that there are several very keen people people out there who have been waiting for this stage of the project.

In terms of kernel code, various people have contributed the odd patch here and there, but most of the work is down to the three of us.

I would, however, like to mention several people who have been a real help. Firstly, Anders Magnusson (Ragge), who is the NetBSD/VAX maintainer. He has been an absolutely invaluable assistance to all of us, on just about all fronts. Without his help and willingness to share his experience in email discussions I think I’d have given up several times over. It’s been very handy to get his perspective on this, as he has already been through what we are doing, and knows the hardware inside out.

We are on friendly terms with the NetBSD people, as is shown by the fact that the membership of port-vax (the NetBSD/VAX list) and linux-vax is very similar, and for a time I was using NetBSD/pmax as my main system for developing Linux/VAX.

Mattias Nordlund, who has been our tester of kernels, has also been very helpful. I’d also like to mention Paul Lamb at MSSL, for his deep knowledge of VAXes, technical discussions, and the discount deals he did to get me my current VAX collection. Plus, of course, all the people on the mailing list, who have offered helpful suggestions, or assisted in the testing of things.

Q: So, who does what in the team? Who’s the release coordinator for the project, and so on?

AP: No formal roles. Kenn and I manage the CVS. The work has split like this, although the areas overlap a lot:

Me — Compiler, toolchain (assembler, linker etc.), mop daemon. General kernel porting, and the special memory management layer we use.

Kenn — Some compiler work, but mostly process and interrupt code in the kernel, general porting, and the SCSI driver for VAXstation 3100s.

Dave — Ethernet Driver, plus process and program loading code. Dave has also put together the root file system we boot from, and ported a simple libc.

That’s not a complete list. Generally, each one of us tackles whatever is interesting, or is next on the list to getting things working. We each have different systems so we work on different hardware.

Q: Getting to the shell prompt was a major accomplishment for the project. Any idea of how many developer hours were spent to reach that milestone?

AP: That’s a hard thing to put a figure to. My best guess, as mentioned above, is that if we had each sat down together and spent a month solid, we would be where we are now — probably. However, I personally have spent more than a man month on the port, over the last couple of years, mainly on the compiler, and writing the first pass, including files for the kernel. When Kenn and Dave joined, this allowed us to bounce ideas off each other, and we each have spurred the other on somewhat, so things did accelerate after the rather slow pace of the first year or so.

Dave Airlie: I was working this [question] out the other day for a laugh. I put down three years for Andy, two years for Kenn, and I’ve been on board for a year from
July, I think. I worked out for myself I probably spent averaging out my
time over the year three to four hours a week at it, (as in, I spend two days
drinking/hacking on Saturday/Sunday, one to two weekends a month). Some months I do
nothing, so in total that has been 24 (eight-hour) days a year, so about five man
months in total… I agree with Andy, if we were all working together a
solid month would have gotten us where we are now …

Also my reasons for doing this, are I needed something to do that
wasn’t my job, I’ve previously been involved with Linux DecStation (mips)
project, and I still listen in on it, but I had gotten my DecStation as
far as I could (had no monitor to finish the graphics card/keyborad
drivers :-), and a friend had a spare VAXstation in his company and I’d
heard from Kenn (both members of the Irish LUG), all about VAX and the
port, and I realised VAX and DECstation had a lot of similar peripheral
hardware, so I jumped on. I’ve learned so much about Linux memory
management, process loading, and filesystems since I’ve started; it has
been an invaluable hands-on learning exercise on parts of the kernel I’d
never looked at before … and hell. I’ve learned VAX assembler 🙂

I’d also like to join Andy in thanking Ragge from the NetBSD/VAX team, he
has explained things to me that my poor brain had stared at for a couple
of days … and Shengchao Li for sending me a copy of the Vax Arch Ref
Manual at a very timely stage in my VAX development.

Q: What were some of the more challenging aspects of this project? Anything in particular that put you at the point of considering abandoning the project?

AP: There were two major show-stoppers. The first was that the Linux kernel after version 2.0 relies heavily on features of the ELF binary format. You could still build a system using the older a.out format, but practically speaking from version 2.0 onwards, the writing was on the wall for a.out. The thing that stopped the early port in its tracks was the use of ELF sections for exception tables. Sections like that just aren’t part of the a.out specification.

So, that forced me to go and rewrite the compiler and binutils to support a ELF binary format for the VAX, which set me back by about nine months. I seriously considered giving up at several points during that. I have intense admiration for people like HJ Lu and Co., who wrote the initial gcc and libc support for Linux/i386, based on that experience. I’d say it’s at least as hard as writing the kernel, and its a shame they don’t get more recognition. There is no agreed ELF definition for the VAX, so I sort of had to make it up as I go along, which is one of the reasons I’m planning on rewriting large sections of that code soon. The other reason is to try and share code with NetBSD/VAX, which has acquired its own, probably superior, ELF implementation in the meantime.

The second show-stopper was the memory management layer, which is still causing problems. There are two reasons why the VAX architecture is “special” in regards to memory management.

The first is that the page size, which is the fundamental unit of memory as far as memory allocation is concerned, is only 512 bytes. Which is very small compared to modern processors. The page size is 4096 bytes (4k) on an Intel processor, and 8192 bytes (8k) on current implementations of the Alpha architecture. The Linux architecture independent memory management code sort of assumes that no system will have a page size of smaller than 4096 bytes, so we faced the decision of either rewriting the Linux MM higher layers and convincing Linus to accept it (unlikely), or faking a 4k page size. We opted for the latter approach, so we have a custom layer that sits between the hardware’s idea of the page size (which we call pagelets) and the Linux MM code’s idea of the page size, which is 4 kb. This in itself causes problems, as the page size is a pretty fundamental property of the architecture, and all the compilers and so forth “know” that a VAX has a page size of 512 bytes. Except that Linux/VAX doesn’t.

The second reason the VAX is special in the memory management department requires a bit of understanding about how processors manage memory. Modern processors use a hierarchical page table. If you access a certain address, the processor looks the address up in a page directory. That gives it the address of a page table containing information about the memory page that holds the address you are interested in. There are two layers of page tables on the Intel architecture, and three on the Alpha. Linux assumes a generic three-level page table scheme by default.

The problem with the VAX is that there is no hierarchical page table. There are three pairs of base and length registers instead. The base register points at the start of the page table entries for that region of memory, and the length register tells you how long the list is. So there is a fundamental mismatch with the memory management models in use by modern processors (and thus in Linux), and the VAX. Practically this means that
memory allocation on the VAX has to be contiguous. That is, you can’t allocate a page at 1k and another at 640k without wasting memory for all the (unallocated) pages in between. On a modern processor, you can allocate memory where ever you want without incurring a memory penalty for the unallocated pages (mostly). So it makes sense to cram the memory allocation into contiguous bits, and avoid holes as much as possible. In fact, it’s pretty much a necessity.

This also places limits on the maximum amount of virtual memory you can allocate to a process. We still don’t have a complete handle on that, and it will probably be a
permanently limiting factor for this port. VAX/VMS (Digital’s own proprietary OS) solved this by adopting a completely different approach to allocating memory to
processes. VMS gave each process a maximum amount of memory it was allowed to use (the working set maximum). To change limits, that usually required a reboot. Unix (and
Linux) in contrast, will quite happily hand over all of the RAM and swap to a single process, and normally don’t place much by way of limits on process virtual memory.

After those negatives, which mostly relate to the fact that the VAX was designed in the 1970s, its nice to note several positive things about the architecture. It has a
very nice interrupt system. The hardware for each model is well defined, and there is no “badly behaved” hardware of the type common on the IBM PC platform.

KH: Well, there is, but not nearly as much. (For example, there’s
a bug in the KA650’s handling of page faults when accessing process page tables, and something funny about early KA650’s Q-Bus interface chipset. Those are the only CPU bugs I know of so far …) And since most of the hardware was designed by
DEC, you tend not to get the sorts of incompatibilities that you find in the x86 world.

AP: In general the instruction set is a joy to use, with complete register
orthogonality (i.e., you can use any access mode on any register, unlike the mess of special cases on the Intel Architecture) that makes it relatively easy to write
assembly code, and compilers for the VAX.

KH: Well, (apart from the VAXstation 3100 Model 76 machines, which
I seem to be able to break too easily :-), the hardware is built to last. It’s all probably over-engineered, but that really helps when you’re dealing with older machines.

Q: What Linux kernel is the Linux/VAX port based on?

AP: We are tracking the 2.4 series, and hope to merge with the 2.5 series. We tend to lag by a revision or so.

Q: I noticed [on the project’s SourceForge page] the nod to Pergamentum Solutions for its help with the Linux/VAX project. What does the company provide?

AP: Pergamentum Solutions is the company run by a friend of mine, Alisdair Davey, based in Bozeman, Mont. He manages the mailing list on mithra, and has donated and hosts the www.linux-vax.org domain name.

In the past he has donated time, web space/storage/machine time,
technical and administrative assistance and other resources to help
the port. Pergamentum also provides me with an email address/Web space/etc.
I can access from anywhere in the world.

More importantly he also pulls me away from the screen and up
mountains in Montana when I need it. Pergamentum Solutions
deserves acknowledgment as much as SourceForge does. If SourceForge
died tomorrow, I’m sure Pergamentum would step in and take over.

Q: How has Digital/Compaq responded to the project; have they provided any kind of support?

AP: We have not really approached them at all. There are several
Digital/Compaq employees, or ex-employees, on the mailing list
I know about. There has been no formal recognition or assistance
from Compaq/Digital of the port. It would be nice if they donated
technical specs or later models of hardware like the model 4000/90
and later to the project. However, I think that since they are
obsoleting the VAX architecture, they are probably not much interested in
promoting it in any way. Chances are, we have just not registered
on their corporate radar at all.

Q: How can interested contributors get involved with the project, and is there any specific area of knowledge or expertise that’s needed at the moment?

AP: Subscribe to the mailing list — linux-vax@mithra.physics.montana.edu. See what’s happening, and lend a hand wherever you feel comfortable. We are still right at the start. If you have a VAX on hand, and want to help out, porting glibc, helping with the compiler or kernel, or even just building applications and testing, it’s all waiting to be done … The Web site needs sorting, too. We have documentation, but no one
likes updating the Web site.

Q: Anything else you’d like to add that this rather slow-witted reporter might have forgotten to ask?

AP: One of the reasons I originally got involved in this port
is that the VAX architecture is historically very important for UNIX. The
VAX was the first major 32-bit architecture to run UNIX, and was the
test bed and “standard system” for the early days of UNIX. TCP/IP was
first implemented on the VAX, and the first hosts on the Internet were
VAXes running UNIX (BSD). The book, “C Programming Language,”
was written and typeset on a VAX 8550. The story of the VAX and modern
architecture independent UNIX is very closely intertwined.

From that point of view, it’s nice to do this port of Linux back
to the “parent architecture” from which its roots sprang. (Purists
will note that the PDP-11 was the original parent system that UNIX booted
on, but the 32-bit VAX architecture is really where the modern
recognizable UNIX was born, IMHO). So this port closes the circle. Given that the
VAX is now obsolete, its also probably going to be the last port of a
major operating system to this architecture, and so I’m glad we got
there in time.