Bash Redirection

Introduction

We're going to explain a few things about Bash redirection here. It's a perennial oldie-but-goodie topic that in my opinion is usually explained poorly.

Redirection (or reconnection/rerouting/switching) is all about the manipulation, via Bash, of a command's input and output streams. Redirection makes it possible to achieve various effects that are perhaps not possible with the command alone, and to combine commands in various ways.

Operators

Redirection causes Bash to change the standard open file configuration that it normally establishes when it processes a command.

In this standard configuration, a process inherits three open files (more accurately, file structures/objects), with file descriptors 0, 1, and 2, on which the file /dev/tty, i.e. the terminal keyboard/screen, is open for reading, writing, and writing respectively. These three particular files are referred to as stdin (standard input), stdout (standard output), and stderr (standard error) respectively (in a C program, the constants stdin, stdout, and stderr are the analogous file pointers). Conceptually

     program
stdin stdout stderr
  0     1     2
   \    |    /
    /dev/tty

Note that it is Bash, not the command being run, that opens the files, and that it is merely convention that commands write normal output to file descriptor 1 and error output to file descriptor 2. Also, most commands that take a file argument read by convention from file descriptor 0, i.e. stdin, i.e. the terminal keyboard, if no such argument is provided, e.g. cat, sort.

NOTE: On Debian, /dev/stdin and /dev/fd/0 are symlinked to /proc/self/fd/0, which is itself symlinked to /dev/tty or /dev/pts/, i.e. the tty of the process. Similarly for /dev/stdout and /dev/fd/1, and /dev/stderr and /dev/fd/2.

The redirection operators generally consist of a file descriptor, some symbols, and a file name. The key to understanding these operators is to recognize that the whole construct is the operator, and also that the symbols are mnemonic.

0<file        # open *file* for reading on fd 0: read input from *file*
              # the < points towards 0 indicating that fd 0 handles input 
1>file        # open *file* for writing on fd 1: write normal output to *file*
              # the > points away from 1 indicating that fd 1 handles output 
2>file        # open *file* for writing on fd 2: write error output to *file*
              # the > points away from 2 indicating that fd 2 handles output 

1>>file       # open *file* for appended writing on fd 1
2>>file       # open *file* for appended writing on fd 2

1>&2          # open the file open for writing on fd 2 on fd 1 too
              # duplicate fd 2; the & means "and" in the sense of duplication
2>&1          # open the file open for writing on fd 1 on fd 2 too
              # duplicate fd 1; the & means "and" in the sense of duplication

&>file        # open *file* for writing on fd 1 and fd 2
              # shortcut equivalent to 1>file 2>&1
&>>file       # open *file* for appended writing on fd 1 and fd 2
              # shortcut equivalent to 1>>file 2>&1

The classic example is to count the files in the current directory using:

ls 1>temp
wc -l 0<temp

Note that a file opened for writing or appending is created if it doesn't exist - the count above will include the file temp. Also note that redirection is not limited to file descriptors 0-2, although these are most commonly used; any file descriptor can be redirected if it is known that the program uses it.

Usage

Some usage tips:

• always use file descriptors (numbers); however in 0<file the 0 may be omitted; in 1>file, 1>>file, and 1>&2 the 1 may be omitted

• don't use spaces; using spaces masks the fact that the whole construct is the operator, and it also confuses redirection with piping:

ls > file 2>&1   # confusing - how many operators? what goes with what? looks like a pipe

ls 1>file 2>&1   # clear - two operators

• operators may be placed before or after a command; such command lines are clearer when spaces aren't used; these two command lines are equivalent:

ls 1>file

1>file ls

• operators are evaluated left to right and the order of operators is significant when duplicating, i.e. a file descriptor is duplicated as it exists when the operator is evaluated:

1>file 2>&1   # open *file* for writing on fd 1 and fd 2
              # fd 1 duplicated after redirection

And there we are. A few things about Bash redirection.