Command Substitution

By the end of this lesson you can capture the output of any command or pipeline into a variable, nest substitutions, and predict what the captured text will and will not contain. Command substitution is how a script composes programs: run a command, take what it printed, and use that text as a value.

Two syntaxes

stamp=$(date +%Y-%m-%d)       # modern form, always prefer this
echo "Report for $stamp"

stamp=`date +%Y-%m-%d`        # legacy backticks: avoid

$( ... ) is preferred because it is easier to read, it nests without escaping ($(basename "$(pwd)")), and linters expect it. Inside backticks, nesting needs backslashes.

Capturing output

lines=$(wc -l < notes.txt)              # the number of lines in a file
who=$(whoami)                           # the current user name
ext=$(awk -F. '{print $NF}' <<< "$file")      # a field out of a string

Any command works, including a whole pipeline and a function of your own. awk splits each line into fields: by whitespace unless you give another separator with -F, so awk '{print $2}' prints the second field of every line. $NF is the last field, whatever the number of fields. Unlike cut -d' ', which treats every single space as a separator, awk's default treats a whole run of spaces or tabs as one. A substitution can contain another substitution, and the innermost runs first:

echo "Working in $(basename "$(pwd)")"

Quote the inner and outer substitution separately, as above, because each one is a new quoting context.

What you get back

Trailing newlines are removed. Everything the command printed is captured except the newlines at the very end:

x=$(printf 'hello\n\n\n')
echo "${#x}"          # 5: just the word hello

That is what you want 99% of the time, since $(date) would otherwise carry a newline. If you need to know about trailing blank lines, measure them separately (wc -c).

Only stdout is captured. Error messages still reach the screen. To capture both, redirect inside: out=$(some_command 2>&1).

The status comes with it. After a plain assignment such as n=$(false), $? is the exit status of the command inside. That is how scripts find out whether a captured command succeeded; the next lesson covers statuses fully.

Quoting rules

A plain assignment is one of the few places where bash does not split the result into words, so files=$(ls) keeps the whole listing in one variable. The danger is where you use the value. An unquoted $files or an unquoted $(...) in an argument is split on whitespace and glob-expanded:

echo $files        # newlines collapse into spaces
echo "$files"      # newlines preserved

The classic trap is looping over a substitution:

for f in $(ls *.txt); do ...        # breaks on names with spaces or *
for f in *.txt; do ...              # correct: the glob yields each name as one word

Process substitution: a different beast

<(cmd) looks like a file containing cmd's output, which lets you hand commands to tools that expect file names:

diff <(sort old.txt) <(sort new.txt)

Command substitution gives you the output as a string; process substitution gives you a file name that reads as the output. Process substitution is a bash feature, not POSIX sh.

Common mistakes

• Unquoted use. echo $(cmd) splits and globs the output; write echo "$(cmd)" unless you want that.
• Looping over $(ls). Use a glob or while read -r.
• Backticks. Hard to read and hard to nest.
• Expecting trailing newlines. They are always stripped.
• Capturing huge output. The whole text is held in memory; pipe to a file or process line by line instead.

Your exercise

Summarise one line of text by capturing three facts about it, each from its own command substitution. The words are separated by runs of spaces or tabs, which is where simple field-cutting tools go wrong, so check what each tool does with repeated whitespace before you rely on it.