Skip to content
Parsing Redirects
step 1/5

Reading — step 1 of 5

Read

~3 min readTokenizing & Parsing

Parsing Redirects

Redirections are the tokens that aren't arguments. In sort < in.txt > out.txt, the program sort receives zero arguments — the four tokens after it are instructions to the shell about stream plumbing. Parsing redirects means walking a command's tokens and separating the words that belong to the program from the operator+filename pairs that belong to you.

The operator zoo

< file      stdin from file
> file      stdout to file (truncate)
>> file     stdout to file (append)
2> file     stderr to file
2>> file    stderr, append
&> file     both stdout and stderr (bash-ism)
2>&1        stderr to WHEREVER stdout currently points (a dup, not a file!)

Each redirect is conceptually a triple: (which stream, what operation, target). < is fd 0, >/>> are fd 1, the 2-prefixed forms are fd 2 — and the general grammar allows any leading digit (3< file is legal POSIX!), which is worth supporting because it makes the digit a parameter, not a special case.

The parse

Walk the command's tokens left to right:

words = [], redirects = []
for each token:
    if token is a redirect operator:
        the NEXT token is its target — consume it too
        (no next token? syntax error: "expected filename")
    else:
        append to words

Applied to sort -r < in.txt > out.txt:

words:     [sort, -r]
redirects: [(0, read, in.txt), (1, write, out.txt)]

Two facts about redirect placement that surprise everyone and fall out of this parse naturally:

  1. Position doesn't matter. < in.txt sort -r and sort < in.txt -r are both legal and identical — redirects are plucked out of the token stream wherever they appear; the remaining words keep their relative order. Don't special-case position; the loop above already handles it.
  2. Order among redirects DOES matter — they apply left to right. The famous pair: > file 2>&1 (stdout to file, then stderr to where stdout now points → both in file) versus 2>&1 > file (stderr to where stdout points now — the terminal! — then stdout to file). Your parser just preserves the order; the executor applies them sequentially, and the semantics emerge. Keep them in a list, never a map.

2>&1 is not a filename

The &1 target means "duplicate stream 2 onto stream 1" — a dup operation between file descriptors, no file involved. Grammatically it still parses as operator-plus-target; the target just names an fd instead of a path. Represent it distinctly ((2, dup, 1)) so the executor (Applying Redirections, chapter 2) can tell "open a file" from "copy a descriptor" — conflating them is the classic implementation bug, and the > f 2>&1 ordering case above only works if dups are their own kind.

Your exercise: Extract Redirections

Commands in, (words, redirects) out. The test progression that maps the space: single redirect → multiple → redirect before the command name → append forms → fd-prefixed forms → 2>&1 → the missing-filename error. Afterward, notice what you've built: the executor now receives a clean argv (what to run) plus a to-do list of stream operations (how to wire it) — the exact two inputs fork/exec plumbing wants. The stage boundaries keep paying rent.

Discussion

Ask a question, share an insight, or help someone who’s stuck.

Sign in to post a comment or reply.

Loading…