To understand the principle of redirections and pipes, we need to explain a notion about processes which has not yet been introduced. Each Unix process (this also includes graphical applications) opens a minimum of three file descriptors: standard input, standard output and standard error. Their respective numbers are 0, 1 and 2. In general, these three descriptors are associated with the terminal from which the process was started, with the input being the keyboard. The aim of redirections and pipes is to redirect these descriptors. The examples in this section will help you understand this concept better.
Imagine, for example, that you wanted a list of files ending with .gif[1] in the directory images. This list is very long, so you want to store it in a file so you can look through it at your leisure. You can enter the following command:
$ ls images/*.gif 1>file_list
This means that the standard output of this command (1) is redirected (>) to the file named file_list. The operator > is the output redirection operator. If the redirection file does not exist, it is created, but if it exists its previous contents are overwritten. However, the default descriptor redirected by this operator is the standard output and does not need to be specified on the command line. So you can write more simply:
$ ls images/*.gif >file_list
and the result will be exactly the same. Next, you can look at the file using a text file viewer such as less.
Now imagine that you want to know how many of these files there are. Instead of counting them by hand, you can use the utility called wc (Word Count) with the option -l, which writes on the standard output the number of lines in the file. One solution is as follows:
wc -l 0<file_list
and this gives the desired result. The operator < is the input redirection operator, and the default redirected descriptor is the standard input one, i.e. 0, and you simply need to write the line:
wc -l <file_list
Now suppose that you want to look at this, remove all the file "extensions" and put the result in another file. One tool for doing this is sed, i.e. Stream EDitor. You simply redirect the standard input of sed to the file file_list and redirect its output to the result file, e.g. the_list:
sed -e 's/\.gif$//g' <file_list >the_list
and there is your list created, ready for you to view at your leisure with any viewer.
It can also be useful to redirect standard errors. For example, you want to know which directories in /shared you cannot access: one solution is to list this directory recursively and to redirect the errors to a file, while not displaying the standard output:
ls -R /shared >/dev/null 2>errors
which means that the standard output will be redirected (>) to /dev/null, a special file in which everything you write to it is discarded (i.e. as a side effect the standard output is not displayed) and the standard error channel (2) is redirected (>) to the file errors.
Pipes are in some way a combination of input and output redirections. The principle is that of a physical pipe, hence the name: one process sends data into one end of the pipe and another process reads the data at the other end. The pipe operator is |. Let us go back to the example of the file list above. Suppose you want to find out directly how many corresponding files there are without having to store the list in a temporary file, you would then use the following command:
ls images/*.gif | wc -l
which means that the standard output of the ls command (i.e. the list of files) is redirected to the standard input of the wc command. This then gives you the desired result.
You can also directly put together a list of files "without extensions" using the following command:
ls images/*.gif | sed -e 's/\.gif$//g' >the_list
or, if you want to consult the list directly without storing it in a file:
ls images/*.gif | sed -e 's/\.gif$//g' | less
Pipes and redirections are not restricted solely to text that can be read by human beings. For example, the following command sent from a xterm:
xwd -root | convert - ~/my_desktop.gif
will send a screenshot of your desktop to the my_desktop.gif[2] file in your personal directory.
[1] | You might think it crazy to say "files ending with .gif" rather than "GIF images". However, once again, files under Unix only have an extension by convention: extensions in no way define a file type. A file ending with .gif could perfectly well be a JPEG image, an application file, a text file or any other type of file. |
[2] | Yes, it will indeed be a GIF image :-) |