Author: Preston St. Pierre
are advertised but unusable, recommended updates that are thoroughly broken,
failure to update packages with high-impact problems, and a poor software
packaging and distribution policy have me testing other distributions with a
goal of taking my business elsewhere.
This heightened level of aggravation with Red Hat started with their split from
the community version. It should be noted that I think this was a smart
business decision for Red Hat. There was no way they could continue to have a
single distribution that was all things to all people. However (and some of
this is coincidental), after that happened, I started having very large
problems with the commercial version of Red Hat Linux (specifically, Red Hat
Advanced Server). I wanted to standardize on the commercial version of Red Hat
for all of the machines in my department that were running Linux. I also wanted
that done so that we wouldn’t go down the road of maintaining more than one
distribution as more and more production services were deployed on Linux.
Another part of my goal was to reduce the number of packages we build from
source. Building packages from source means that every time the application is
updated, we have to build it from source again, and hope it all works. One or
two packages isn’t so bad, but we build our SSH server, SSL, print server,
database server, web server, PHP, and plenty of other stuff from source. This
adds up to a lot of overhead. Some of this is, I believe, historical in nature,
so I decided to analyze the actual need for this, and I found some success in
one service in particular: CUPS.
The CUPS Print Server Debacle
CUPS is a print server package. The first problem I had with the Red Hat version
was that the version in the package is almost totally unrelated to the
capabilities of the package. What exactly, then, is the purpose of having a
version number on the package? That was an annoyance — luckily, the package,
as it turns out, came with a copy of the ps2ps command that worked as it was
supposed to, which is to say that it properly stripped out PJL code from PS
files printed from Windows machines. Great news, since we could then retire an
aging Perl script we used to do this. Great — we were ready to go into
testing.
We tested printing against this machine for a couple of weeks. The
week before we were to move it into production, we ran up2date on the
machine, and ps2ps broke again. Wonderful. As if that weren’t enough, there’s
another problem on that new server that could result in an unintentional DoS.
The installed (and running by default) authd daemon’s config file
(/etc/xinetd.d/authd) is configured by default with the key
“instances” set to “UNLIMITED”. As a result, every time one of our guys who was
running a fairly strict firewall tried to connect to the box via SSH, the authd
server would spawn hundreds of processes trying to connect back to his machine.
This caused the load to immediately spike, and the entire machine becomes
completely unusable. Thanks, Red Hat.
Lackadaisical
In addition, other machines that were to move into production had also been
updated, only to find that the aacraid driver was broken, and they couldn’t
boot! The report in bugzilla for the RAID driver resulted in Red Hat saying they
would release a fix for this in the next kernel package, which would be
released the next time a security vulnerability was reported and fixed — they
would release them in the same package. I couldn’t believe this, especially
since I knew that the biggest server retailers used hardware that required that
driver. As a result of this, my machines remained running with a
locally-exploitable kernel vulnerability for the sake of a RAID driver. Thanks,
Red Hat.
Up2Date Rollbacks, or not
This makes for a nice segue into the undocumented-but-advertised features area.
After running up2date on these machines and noting their extreme brokenness, I
very much wanted to do a rollback on certain packages. I had enabled rollbacks
in my up2date configuration, only to find when I actually needed to do one that
the process is completely undocumented by Red Hat. It would seem antithetical to
the cause of releasing a product that is, if nothing else, stable and
predictable, to release a package with features enabled that are not only
undocumented, but to the best sources my searches could find, are not
recommended for production use by Red Hat. Unfortunately, I didn’t learn of the
issues with rollbacks until I needed to use the feature, at which time I found
that the best solution was to scrap rollbacks, save my disk space, and just
reinstall the package in question from scratch again. Rollbacks are not a part
of a well-balanced administrative policy, as implemented by Red Hat.
The most recent update also broke the quota program. The feature that was
broken is one that had been noted as being broken several months before the
updated release. This was a recommended update from Red Hat, and it’s not like
just a command line option is broken — the program, as a whole, is unusable.
This was in Red Hat’s bugzilla in May 2004. How it wound up in an update in
December of the same year is beyond me. How it hasn’t been updated since that
time is also beyond me. Thanks, Red Hat.
New Face, Old Packages, Same Price
Finally, due to Red Hat being slow as molasses to implement new and highly
recommended features of software packages, I am forced to completely discount
them as a solution for some of the services I’d like to deploy without building
from source. My prime example is OpenLDAP. I’ve been trying for way too long to
migrate the department from NIS to LDAP for authentication. NIS won’t go away
completely, because it’s useful for some things, but we like LDAP for several
reasons, and for some things it’s far better than NIS, so we’re moving in this
direction.
The current version of OpenLDAP is 2.2.23. The version the Red Hat package is
based on is 2.0.x. That version of OpenLDAP, if I’m not mistaken, didn’t even
fully support LDAPv2, let alone version 3, and used a backend that the OpenLDAP
developers have long ago shunned. For some unknown reason that would seem like
a whole lot of work, Red Hat appears to be applying patches to this old,
decrepid version of OpenLDAP and distributing it, presumably with the
expectation that someone, somewhere will use it. As far as I know, it is the
oldest version of OpenLDAP distributed by default with any Linux distribution.
If I use Red Hat for this service, I would be forced to build not only OpenLDAP,
but the updated version of the Berkley DB backend that is (for the past ~2
years) the standard OpenLDAP backend recommended by the development team. I’d
be doing this by myself, and applying updates by myself, in spite of paying
Red Hat a yearly fee for “updates”. Thanks, Red Hat.
The Point
The point is, what exactly is my motivation for staying with Red Hat? I can’t
seem to find one. I still use Fedora Core on things that don’t matter much
(when I can get it to work — that distro has plenty of issues of its own), but
outside of that, I’m evaluating other options. I’m finding some really nice
tools. I’m finding far more up to date software packages. I’m finding much more
admin-friendly tools and environments. And I’m finding that Red Hat,
comparatively speaking, is not a very big deal in the server space. Overall,
I’m seeing no benefit to using Red Hat’s enterprise line over, say, the Fedora
line. Maybe they started charging for the wrong distribution.
