Linux Know-How: Redirection and file descriptors

As you know from basic shell usage, input and output of a command may be redirected before it is executed, using a special notation - the redirection operators - interpreted by the shell. Redirection may also be used to open and close files for the current shell execution environment.

Redirection can also occur in a script, so that it can receive input from a file, for instance, or send output to a file. Later, the user can review the output file, or it may be used by another script as input.

File input and output are accomplished by integer handles that track all open files for a given process. These numeric values are known as file descriptors. The best known file descriptors are stdin, stdout and stderr, with file descriptor numbers 0, 1 and 2, respectively. These numbers and respective devices are reserved. Bash can take TCP or UDP ports on networked hosts as file descriptors as well.

The output below shows how the reserved file descriptors point to actual devices:

You might want to check info MAKEDEV and info proc for more information about /proc subdirectories and the way your system handles standard file descriptors for each running process.

When you run a script from the command line, nothing much changes because the child shell process will use the same file descriptors as the parent. When no such parent is available, for instance when you run a script using the cron facility, the standard file descriptors are pipes or other (temporary) files, unless some form of redirection is used. This is demonstrated in the example below, which shows output from a simple at script:

Redirection of errors

From the previous examples, it is clear that you can provide input and output files for a script (see Section 8.2.4 for more), but some tend to forget about redirecting errors - output which might be depended upon later on. Also, if you are lucky, errors will be mailed to you and eventual causes of failure might get revealed. If you are not as lucky, errors will cause your script to fail and won't be caught or sent anywhere, so that you can't start to do any worthwhile debugging.

When redirecting errors, note that the order of precedence is significant. For example, this command, issued in /var/spool

ls -l * 2 > /var/tmp/unaccessible-in-spool

will redirect output of the ls command to the file unaccessible-in-spool in /var/tmp. The command

ls -l * > /var/tmp/spoollist 2 >& 1

will direct both standard input and standard error to the file spoollist. The command

ls -l * 2 >& 1 > /var/tmp/spoollist

directs only the standard output to the destination file, because the standard error is copied to standard output before the standard output is redirected.

For convenience, errors are often redirected to /dev/null, if it is sure they will not be needed. Hundreds of examples can be found in the startup scripts for your system.

Bash allows for both standard output and standard error to be redirected to the file whose name is the result of the expansion of FILE with this construct:

&> FILE

This is the equivalent of > FILE 2>&1, the construct used in the previous set of examples. It is also often combined with redirection to /dev/null, for instance when you just want a command to execute, no matter what output or errors it gives.