Author: Brian Jones
How do I get there from here?
There isn’t a single route to system administration. For every sysadmin I know, I know a different story of how they ended up there. Some actually got degrees in computer science, but didn’t want to be programmers. Others started at help desks and slowly crept into administration. I was a consultant who noticed that, at all the client sites I went to, the people playing with the coolest stuff were the admins. They were also the gatekeepers to every resource I needed access to. They played key roles in growing the IT organization. Most of all, though, they had a different view of everything than anyone else in the IT department, and this intrigued me. I slowly began taking on sysadmin tasks that dealt directly with my role as a database administrator, started playing with even more stuff on my free-but-more-than-capable Linux system at home, and the job which immediately followed my DBA consulting gig was a systems engineer gig for another consulting firm.
It doesn’t matter where you are now. If your personality leads you toward systems administration, you’re likely to end up there. So what personality traits make for a good systems administrator? Well, for starters, the ability to tirelessly research your craft. Systems administration is a field that is never idle. You can’t get a certificate in administration, get a job, and stay put until retirement. You won’t last three years. The job demands that you stay on top of new technologies. I’m not talking about just reading magazines, either. Download or otherwise get your hands on the software. Install it, configure it, run it, break it, lather, rinse, repeat ad nauseum. Figure out where a technology fits, how it could be better, how your environment could benefit from it, and where its deployment would naturally lead. When it breaks, figure out why. Is it a shortcoming of some system library, or is it a real application problem? What kind of hardware will you need to run it effectively? How does it make your life easier? How does it benefit the user community? What are the trade-offs?
Got all that answered? Great. Now figure out how to integrate it into your authentication, logging, backup, and email architectures. Done all that? Now unleash Netcat, tcpdump, ngrep, and anything else that tickles a port on it. Don’t be satisfied that it didn’t crash or cause a denial of service condition on the server. Did it let you know something was going on? Did it crash? Did it fail over? How does it react to simulated malicious behavior?
In practice, systems administrators have, in the back of their minds, the answers to every one of these questions, for every component of their environment. Some answers take more time to remember than others, but they’re there. In addition, we assume that other administrators are the same way, and some take personal offense at those who are lackadaisical in their duty to keep up with the environment. However, in theory, it’s not about answering all of these questions off the top of your head.
What makes an admin tick
Philosophically speaking, systems administration is about your problem-solving mindset. This can be broken down into your ability to ask the right questions, and your ability to effectively process the responses.
Asking the right questions is a matter of understanding what information is available to you on a given system, and what tools are available with which to retrieve it. But it’s more than that. It’s also about when to ask questions, when to take action, and when to do nothing. For example, a newbie admin, upon noticing a lot of zombied processes and a load of over 120, might just ask himself “what do I do?”, and go power-cycle the system. A more experienced admin will note any similarities between the zombied processes. When were they launched? Who owned the processes? What environment settings were the processes running with? Is the load growing? Do I have to power-cycle this machine?
In many cases, administration becomes an investigation into the why and the how of a given situation. In the situation above, an experienced admin would continue on into questions to try to figure out why the processes were zombied in the first place, why the system load is so high, what resources are actually being hogged, whether the zombied processes are actually the cause of the load (most times they are not), and whether this problem could have been avoided.
Of course, deciding what action to take, if any, is dependent on your ability to effectively process the data and responses to your questions. If you make the decision to power-cycle a server based solely on the answer to a single question, you’ve almost certainly failed. Why? Because even when the question is “can I even reach this system,” and the answer is “no” (leaving you with little choice but to power-cycle), you should still be asking why you can’t reach the system, and making sure there isn’t a networking or console software problem before you go interrupting services to your users.
Where do I start?
One question that managers who hire administrators have to address coincides with a question that administrators have to answer for themselves: to specialize or to generalize? Nowadays, it seems administrators are getting more and more specialized. We now have email administrators, backup administrators, directory administrators, Web administrators, and even storage administrators. In addition, we have security officers, hardware guys, and a host of other staff that do what a general system administrator would otherwise do. All of this would lead you to believe that managers would like to hire a specialist, and so that’s the way you should go, but hold on.
There’s nothing wrong with starting out as a specialist. I know of a lot of cases where people got into administration by taking on the one specialized administrative task that nobody else in the office wanted. They started as specialists and evolved into generalists as the years went by and needs arose for different skills. I would caution against painting yourself into a corner. If you’re an email admin, try to expand your skillset. Run a Web server at home. Run your own DNS at home if you can. Be a secondary DNS for a friend. Set up a proxy server. Set up file and printer sharing for every device on your home network. Run at least three operating systems on a regular basis. Go wireless, and secure the wireless network. Set up a homemade home security system.
More important than any of this, make yourself a trivial part of the maintenance of these systems and services. Try to script yourself into the position of an end user. It sounds impossible to set up and maintain all of these services and have them doing productive things without locking yourself in the basement for the rest of your life. The goal of the administrator is to automate oneself out of a job. This involves not only knowing tools, networking, systems, and services, but the scripting languages to make the tools, networking, systems and services take care of themselves.
The human element
Almost everything I’ve said thus far applies not only to systems, but to people as well. You have to be able to ask them the right questions as well. You have to know how to parse their answers. You have to know how to get past what they say they want to get at what it is they really need. I can’t tell you how many users are convinced that “If they only had root…”. Truth is, I’ve never given a user root privileges, because no user I’ve run into has ever needed root privileges to get his work done. I personally avoid the root account if at all possible, and get a tremendous amount of work done, so if a user account is good enough for me, it’s likely to be good enough for anyone else in my environment. Nevertheless, you can’t just say “no” and think that’ll be the end of it. It may not be today, but eventually, your attitude toward the user community will come back to either reward or punish you.
There’s even more to it than this. In the future, I’ll touch on this topic again. However, the important thing to remember is that system administration is more than just a job. It’s a lifelong craft. Technology is such that no one person ever knows everything about it. It forces you to keep learning and evolving along with it or be left behind.
