Work Smarter, Not Harder: Tips for Linux Administration Done Right

Interrupt driven. That’s how a lot of sysadmins define their jobs, and it’s absolutely true. We spend a good portion of every day running from problem to problem. Whether it be a user who can’t get their email, to a jammed printer, to a server that’s got a failed disk, we friendly neighborhood system administrators have plenty to do on a daily basis. However, we’re also some of the most resourceful people in any given organization. It’s time we used that resourcefulness to make our lives easier. I’m going to run through some techniques and ways of thinking that will help you better manage your workload and increase the overall quality of the services you provide to your organization.

Here are a few values that my techniques are built on. They may seem basic, but it’s amazing how you can lose sight of the larger picture and your value to the organization when you’re playing fireman. It’s difficult to think strategically if you are acting tactically all the time.

• Work sucks.

• You want to do as little of this as you can get away with. The job is difficult enough, so figure out ways to make your life easier.

Good Tools™ are like gold.

• Build them, download them, or buy them — they are worth it!

If you have to work, its better to do prep work up front than panic work.

• Panic work will take years off your life, tick off your loved ones, and make you grumpy. Avoid it. Be prepared, like a Boy Scout.

Reuse, Recycle, Reduce.

• Leverage work done before you. Use other code, or pieces from projects. Don’t reinvent the wheel. Document your work.

Think, think, think.

• It wouldn’t be called “Working Smarter” if you didn’t think. So do that.

Tools of the Trade

Hardware is the foundation of what we do as system administrators, so it behooves you to ensure your hardware is up to task. We’ve all got systems we’ve inherited, or systems that are old and dying that are under our care. If you don’t like a particular system, or it’s got issues, quit grumbling about it, get it off the back burner, and fix the darn thing. A system that needs constant hand-holding doesn’t do your organization any favors. Here are a few other hardware-related tips that help out a bunch:

• Lights Out Management

• Goes by iLO if you’re using an HP server, or DRAC if you’re using a Dell, or LOM if you’re on a Sun. This lets you connect remotely to your server and get a console on it, or power cycle it, or do remote diagnostics. It’s invaluable. Insist that all your servers have this. HP servers include it by default.
• If you can’t get a server with iLO, you can fake it. Remote console servers like a Cyclades unit will get you a serial text console on a given server, and a remote power strip like a BayTech or APC unit can get you the ability to power cycle a machine. Alternatively, you can build your own using an old computer and some X10 gear like I did at home — though I don’t recommend this for your business!

RAID

• Not the bug spray, but a Redundant Array of Inexpensive Disks. Your servers should all have RAID-1 (a pair of disks, mirrored) at a minimum, unless you’re doing something like Hadoop. This protects against a disk failure, and will save you time by not having to rebuild a system if a disk fails.

Redundancy

• From redundant power supplies, to redundant network interface cards using ethernet bonding — no one ever got fired for making something too redundant.

Virtualization and the Cloud

• Don’t be afraid of virtualization or the cloud. If your application can tolerate being migrated to a virtual machine or offloaded to a cloud provider, seriously consider it. You may wind up saving a lot of hardware-related headache.
• Virtualization and the cloud are not “one-size-fits-all” solutions, however. Use good judgement and be sure to run many pilot tests before deploying your application on these platforms.

By making the best use of hardware, you’ll be able to focus more on the services you’re providing rather than dealing with hardware failures — or having to drive to the data center at 2 a.m.