To understand the principle of redirections and pipes, we need to explain a notion about processes which has not yet been introduced. Most UNIX processes (this also includes graphical applications but excludes most daemons) use a minimum of three file descriptors: standard input, standard output and standard error. Their respective numbers are 0, 1 and 2. In general, these three descriptors are associated with the terminal from which the process was started, with the input being the keyboard. The aim of redirections and pipes is to redirect these descriptors. The examples in this section will help you better understand this concept.
Imagine, for example, that you wanted a list of files ending with .png [6] in the images directory. This list is very long, so you may want to store it in a file in order to look through it at your leisure. You can enter the following command:
$ ls images/*.png 1>file_list |
This means that the standard output of this command (1) is redirected (>) to the file named file_list. The > operator is the output redirection operator. If the redirection file does not exist, it is created, but if it exists, its previous contents are overwritten. However, the default descriptor redirected by this operator is the standard output and does not need to be specified on the command line. So you can write more simply:
$ ls images/*.png >file_list |
and the result will be exactly the same. Then you could look at the file using a text file viewer such as less.
Now, imagine you want to know how many of these files exist. Instead of counting them by hand, you can use the utility called wc (Word Count) with the -l option, which writes on the standard output the number of lines in the file. One solution is as follows:
wc -l 0<file_list |
and this gives the desired result. The < operator is the input redirection operator, and the default redirected descriptor is the standard input one, i.e. 0, and you simply need to write the line:
wc -l <file_list |
Now suppose you want to remove all the file “extensions” and put the result in another file. One tool for doing this is sed (Stream EDitor). You simply redirect the standard input of sed to the file_list file and redirect its output to the result file, i.e. the_list:
sed -e 's/\.png$//g' <file_list >the_list |
and your list is created, ready for you to view at your leisure with any viewer.
It can also be useful to redirect standard errors. For example, you want to know which directories in /shared you cannot access: one solution is to list this directory recursively and to redirect the errors to a file, while not displaying the standard output:
ls -R /shared >/dev/null 2>errors |
which means that the standard output will be redirected (>) to /dev/null, a special file in which everything you write is discarded (i.e. the standard output is not displayed) and the standard error channel (2) is redirected (>) to the errors file.
Pipes are in some way a combination of input and output redirections. The principle is that of a physical pipe, hence the name: one process sends data into one end of the pipe and another process reads the data at the other end. The pipe operator is |. Let us go back to the file list example above. Suppose you want to find out directly how many corresponding files there are without having to store the list in a temporary file, you would then use the following command:
ls images/*.png | wc -l |
which means that the standard output of the ls command (i.e. the list of files) is redirected to the standard input of the wc command. This then gives you the desired result.
You can also directly put together a list of files “without extensions” using the following command:
ls images/*.png | sed -e 's/\.png$//g' >the_list |
or, if you want to consult the list directly without storing it in a file:
ls images/*.png | sed -e 's/\.png$//g' | less |
Pipes and redirections are not restricted solely to text which can be read by human beings. For example, the following command sent from a Terminal:
xwd -root | convert - ~/my_desktop.png |
will send a screenshot of your desktop to the my_desktop.png [7] file in your personal directory.
[6] You might think it is crazy to say “files ending with .png” rather than “PNG images”. However, once again, files under UNIX only have an extension by convention: extensions in no way define a file type. A file ending with .png could perfectly well be a JPEG image, an application file, a text file or any other type of file. The same is true under Windows as well!
[7] Yes, it will indeed be a PNG image (however, the ImageMagick package needs to be installed...).