Two-Phase Parser Architecture

Every serious Markdown parser splits its work in two passes, and every lesson in this course has been one half of that split. This lesson names it and makes you build the seam between the halves: a block phase that turns lines into tokens, and a render step that hands the right tokens to an inline phase. Your program will parse a small document into tokens first, and only then produce HTML.

1. Block phase. Scan line by line and group lines into block tokens: paragraph, heading, code block, quote, list, table, thematic break.
2. Inline phase. For each block that contains inline content (paragraph text, heading text, list item text), walk its text and produce emphasis, strong, code, links and images.

Why two phases?

Block structure is decided by line context. Whether > a is a quote depends on whether a > prefix continues. Whether - x is a list item depends on its neighbours. Whether --- is a thematic break or a setext H2 depends on what came before it. And a fenced code block consumes its content verbatim, so inline parsing must never run inside it. You cannot decide any of that by looking at one line in isolation; you need state that tracks the block you are in.

Inline structure is decided by character context: delimiter pairing (*open* with *close*) and bracket matching ([text](url)), independent of which line a character is on within a single block.

The data between the phases

The block phase returns a list of tokens. For this exercise there are three kinds:

A worked example. These lines:

# Title
Some *text*
```sh
a * b
```

produce three tokens, in order: heading(1, "Title"), para("Some *text*") and code("sh", ["a * b"]). Only the first two reach the inline phase. The * in the code token never meets an emphasis rule, so it comes out as a * b.

Why code blocks prove the point

Inline parsing inside a code block would be a disaster. Every language has characters Markdown claims: * in C, _ in Python names, backticks in shell. If you ran inline parsing first, you would corrupt code samples. Only the block phase knows that a span of lines is code, which is why it must run first and why the code token carries raw lines, not text.

The Crafting Interpreters parallel

In Crafting Interpreters, a scanner produces a flat token stream and a parser groups tokens into expressions. Markdown does the same thing at two scales: the block phase is the scanner (it groups characters into block tokens by line context), and the inline phase is the expression parser (it matches delimiter pairs inside a token's content).

What the split buys you

The two layers can be tested and reasoned about independently, which is what lets CommonMark have 650 canonical examples and a deterministic spec. Single-pass, regex-based converters collapse on the corner cases: either they run inline rules inside code, or they get setext headings, lazy continuation and tight versus loose lists wrong.

Common mistakes

• Parsing inline inside code. The code token must keep raw lines and skip the inline phase.
• Letting a paragraph run on. A heading line or a fence opener interrupts a paragraph even with no blank line.
• Closing the fence wrongly. An unclosed fence runs to the end of the document.
• Mixing the phases. If block_parse calls the emphasis rules, you have a single-pass parser with extra steps.

Your exercise

Write block_parse (lines to tokens) and render (tokens to HTML lines). The starter provides the inline phase from the emphasis and code lessons, plus the heading and fence recognisers from the headings and code lessons, so the exercise is the architecture itself.