Troubleshooting in the Command Line: Tips for Linux Beginners

5619

Linux has come a long way in its short life, and it’s more reliable and stable than ever. But things still go wrong, and you can diagnose and fix just about anything.

Desktop Freezes

Compositing window managers are a huge step forward in making graphical environments more stable. But sometimes your nice Linux graphical desktop still locks up, and then what do you do? Drop to a console is what you do, by pressing Ctrl+Alt+F2 on your keyboard. This takes you to a console that is independent of your graphical environment. Go ahead and login.

command line login screen

Now you can do stuff. If you have a notion what locked up your desktop, you can find its process number and kill it. For example, when I’m connected to a remote network share in my Dolphin graphical file manager and the network connection is interrupted, Dolphin locks up my whole desktop. So I drop to a console and run this command to find its process number:

$ ps aux | grep dolphin 
carla 9218 ? Sl 0:00 /usr/bin/dolphin
--icon system-file-manager -caption Dolphin

This shows who owns the process (carla), who should be able to kill it with the kill command:

$ kill 9218

If root or a different user owns the process, then use sudo kill 9218.

Now press the up arrow on your keyboard to scroll back to the previous command, hit the Enter/Return key to re-run it, and see if it worked. If it didn’t then roll out the big gun:

$ kill -9 9218

-9 sends the SIGKILL signal, which is the nuclear option because it cannot be ignored.

What if you see a runaway process that has spawned child processes? You need to kill the parent process because that also takes out the children, and it prevents it from re-spawning any child processes that you kill. (If you’re not comfortable with this terminology, you have company because I’m not comfortable with it either.) Add the -f switch to see parent and child processes in a tree view, like this abbreviated example for the Plex media server:

root 1776 /bin/sh -e /proc/self/fd/9 
plex 1803 _ /bin/sh /usr/sbin/start_pms
plex 1804 _ ./Plex Media Server
plex 1970 _Plex Plug-in [com.plexapp.system]
plex 2645 _ /usr/lib/plexmediaserver
plex 2690 _ Plex Plug-in

So you could use sudo kill 1776 to wipe out the whole works.

On most distros there are 6 consoles, tty1 to tty6tty7 is usually your X session, so you get back to your graphical desktop by pressing Alt+F7. Which means you can get to your console with anything from Ctrl+Alt+F1 to Ctrl+Alt+F6.

Who is the Culprit?

What if you’re not sure which process is causing the problem? Try the good old top command:

$ top  
top - 12:07:33 up 4:13, 7 users, load average: 0.56, 0.38, 0.34
[...]
PID USER VIRT RES %CPU %MEM COMMAND
6399 carla 493m 27m 94.2 0.2 konsole
4386 carla 1937m 819m 2.0 5.1 firefox
1511 root 613m 189m 1.3 1.2 Xorg

This (abbreviated) example points to Konsole as the troublemaker, as it’s sucking up 94.2% CPU. The process ID is right there in the output, so you know what process to kill.

Logging Saves The Day

Most services log their activities. If you look in /var/log you’ll see a set of logs such as CUPS, boot, dmesg, kern.log, syslog, and udev. When you install services they usually have configurable logging, so you can choose the location and amount of detail, from emergency to debug:

 

  • debug
  • info
  • notice
  • warning
  • error
  • critical
  • alert
  • emergency

 

emergency outputs the least amount of information, and debug the most. info is a good everyday logging level, recording a mix of routine activity and warnings and errors. debug can be overwhelming, so a good tactic is to turn on debug when you’re investigating a problem, and put it back to info when it’s resolved. Where do you do this? Look in the /etc directory. Individual services have their own configuration files, like /etc/cups/cupsd.conf. The syslog is configured in /etc/syslog.conf, if your Linux has the old syslogd, or /etc/rsyslog.conf if you have the shiny new rsyslogd.

Logfiles aren’t exactly fun to read, but they’re essential to solving problems. You can home in on error messages with grep:

$ grep -i error /var/log/syslog

Or any text string to help you quickly find the information you want, like this example for seeing what Network Manager has been doing:

$ grep -i networkmanager /var/log/syslog 
Dec 10 14:54:50 studio NetworkManager[1402]:
(eth1): DHCPv4 state changed bound -> renew
Dec 10 14:54:50 studio NetworkManager[1402]:
address 192.168.10.182
Dec 10 14:54:50 studio NetworkManager[1402]:
prefix 24 (255.255.255.0)

Once you home in on the log messages that look helpful, you can reference your documentation to see what’s up, and hit the Googles to find out more.

But Graphical Apps Don’t Have Logs

Most graphical applications don’t generate logfiles, which is sad and unhelpful. But you can still see some command output by running the app from the command line. Like my favorite game, Super TuxKart:

$ supertuxkart  
Irrlicht Engine version 1.8.0 Linux 3.8.0-19-generic
#30-Ubuntu SMP Wed May 1 1
6:35:23 UTC 2013 x86_64 [FileManager] Data files will be fetched from:
'/usr/share/games/supertuxkart'
[FileManager] Addons files will be stored in
'/home/carla/.local/share/supertuxkart/addons'.

Debian requires all programs to have a man page, so if you’re running Debian, Ubuntu, or one of their derivatives you’ll always have man page to reference. If there isn’t a man page or other documentation try the -h switch to see a help menu, like supertuxkart -h. Of course I couldn’t get anything to misbehave so I would have an example to show you, but when you’re having problems with a graphical application this is a good way to see why it’s acting up. You may be missing a library or having a conflict, and chances are the command-line output will tell you.