Pipes and Redirection

This is the lesson bash exists for. One idea — every program reads a stream in and writes streams out, and you can plug them into each other — turned a collection of small tools into the most durable software architecture ever shipped. Fifty years later, it's still how professionals answer questions in ten seconds.

The three streams

Every process is born holding three open channels:

• stdin (0) — data in
• stdout (1) — results out
• stderr (2) — errors and chatter out, deliberately separate so problems don't contaminate results

Redirection: streams ↔ files

sort < unsorted.txt          # stdin from a file
ls > listing.txt             # stdout to a file (TRUNCATES it first!)
ls >> listing.txt            # append instead
ls /nope 2> errors.log       # stderr (stream 2) to a file
ls /nope 2> /dev/null        # …or into the void (silence errors)
cmd > all.log 2>&1           # both streams to one file (order matters: file first)

> file wiping the file before the command runs is the classic self-inflicted wound (sort data > data empties data). And /dev/null — the kernel's paper shredder — completes the toolkit.

Pipes: streams ↔ programs

history | grep "git" | wc -l     # how many git commands have I run?

A | B connects A's stdout to B's stdin — no temp files, both running concurrently. Each stage is small and dumb; the pipeline is smart. The workhorse stages worth knowing cold:

grep pattern      # keep matching lines        sort          # order lines
uniq              # collapse adjacent dupes    wc -l/-w      # count lines/words
head -5 / tail -5 # first/last N               tr 'a-z' 'A-Z' # translate chars
cut -d, -f2       # column extraction          sed 's/a/b/'  # stream editing

And the pattern template that answers 80% of "quick question about this data": select → normalize → sort → count:

tr ' ' '\n' < file | sort | uniq -c | sort -rn | head -5   # top 5 words. Ten seconds.

(uniq only collapses adjacent duplicates — the sort before it isn't decoration. Everyone learns this by getting wrong counts once; you just learned it cheaper.)

One conceptual footnote that pays later: each pipeline stage runs in a subshell, so … | while read x; do count=$((count+1)); done loses count when the subshell exits — the famous "my variable vanished after the pipe" mystery. File it away; the fix (< file while-loop, no pipe) is in your Loops lesson.

Your exercise: Count Words

Count the words in the input. The entire program:

wc -w

No, really. wc -w reads stdin (the grader's input), counts words, prints the number — your script inherits its stdin, so a bare filter command IS a complete bash program. If the grader's expected output is a bare number and wc -w prints leading whitespace on your system, pipe it through tr -d ' ' or use echo $(wc -w) to trim. One command, and you've written your first real filter — the shape of half the bash you'll ever write.