Process Substitution from the Linux command line by OnWorks

Skip to content

Process Substitution

While they look similar and can both be used to combine streams for redirection, there is an important difference between group commands and subshells. Whereas a group com- mand executes all of its commands in the current shell, a subshell (as the name suggests) executes its commands in a child copy of the current shell. This means that the environ - ment is copied and given to a new instance of the shell. When the subshell exits, the copy of the environment is lost, so any changes made to the subshell’s environment (including variable assignment) is lost as well. Therefore, in most cases, unless a script requires a subshell, group commands are preferable to subshells. Group commands are both faster and require less memory.

We saw an example of the subshell environment problem in Chapter 28, when we discov- ered that a read command in a pipeline does not work as we might intuitively expect. To recap, if we construct a pipeline like this:

echo "foo" | read echo $REPLY

The content of the REPLY variable is always empty because the read command is exe- cuted in a subshell, and its copy of REPLY is destroyed when the subshell terminates.

Because commands in pipelines are always executed in subshells, any command that as- signs variables will encounter this issue. Fortunately, the shell provides an exotic form of expansion called process substitution that can be used to work around this problem.

Process substitution is expressed in two ways: For processes that produce standard output:

<(list)

or, for processes that intake standard input:

>(list)

where list is a list of commands.

To solve our problem with read, we can employ process substitution like this:

read < <(echo "foo") echo $REPLY

Process substitution allows us to treat the output of a subshell as an ordinary file for pur- poses of redirection. In fact, since it is a form of expansion, we can examine its real value:

[me@linuxbox ~]$ echo <(echo "foo")

/dev/fd/63

[me@linuxbox ~]$ echo <(echo "foo")

/dev/fd/63

By using echo to view the result of the expansion, we see that the output of the subshell is being provided by a file named /dev/fd/63.

Process substitution is often used with loops containing read. Here is an example of a

read loop that processes the contents of a directory listing created by a subshell:

#!/bin/bash

# pro-sub: demo of process substitution

while read attr links owner group size date time filename; do cat <<- EOF

Filename: $filename Size: $size

Owner: $owner

Group: $group Modified: $date $time Links: $links Attributes: $attr

EOF

done < <(ls -l | tail -n +2)

#!/bin/bash

# pro-sub: demo of process substitution

while read attr links owner group size date time filename; do cat <<- EOF

Filename: $filename Size: $size

Owner: $owner

Group: $group Modified: $date $time Links: $links Attributes: $attr

EOF

done < <(ls -l | tail -n +2)

The loop executes read for each line of a directory listing. The listing itself is produced on the final line of the script. This line redirects the output of the process substitution into the standard input of the loop. The tail command is included in the process substitution pipeline to eliminate the first line of the listing, which is not needed.

When executed, the script produces output like this:

[me@linuxbox ~]$ pro-sub | head -n 20

Filename: addresses.ldif Size: 14540

Owner: me

Group: me

Modified: 2009-04-02 11:12

[me@linuxbox ~]$ pro-sub | head -n 20

Filename: addresses.ldif Size: 14540

Owner: me

Group: me

Modified: 2009-04-02 11:12

Links: