Two-Phase Parser Architecture

Every serious Markdown parser splits its work in two passes:

1. Block phase: scan line-by-line, group lines into block tokens (paragraph, heading, list, code block, blockquote, table, hr).
2. Inline phase: for each block that contains inline content (paragraph text, heading text, list item text), walk the text character-by-character and produce inline tokens (text, emph, strong, code, link, image).

Why two phases?

Phase 1: line context Block-level structure is determined by line context.

• Whether > a is a blockquote depends on whether a > prefix continues.
• Whether - x is a list item depends on whether neighbouring lines also match - .
• Whether --- is a thematic break or a setext H2 depends on what came BEFORE it.
• Fenced code blocks consume content verbatim — inline parsing must NOT run inside a code block.

You cannot decide any of this by looking at one line in isolation. You need a state machine that tracks the current block type.

Phase 2: character context Inline structure (emphasis, links, code spans) is a different beast: it deals with delimiter pairing — *open* matched with *close*, [text](url) matched bracket-paren — independent of which line the characters are on (within a single block).

Inline parsing inside a code block would be a disaster:

`*not italic*`

The * here is literal. Only the block phase knows this span is inside a code block; if you ran inline parsing first, you would corrupt code samples.

The Crafting Interpreters parallel In Crafting Interpreters, the scanner produces a flat token stream and the parser groups tokens into expressions. Markdown does the same thing:

• Block phase = scanner (groups characters into "block tokens")
• Inline phase = expression parser (matches delimiter pairs into nested nodes)

What this means in practice

• Your block scanner returns a list of (block_type, content_text) tuples.
• Your inline parser is called on the content_text of each block.
• Inline parsing never sees raw line boundaries; it just sees a string.
• Block-level state is closed before inline parsing starts.

This separation is what lets CommonMark have 600+ canonical test cases and a deterministic spec — the two layers can be reasoned about independently. Single- pass parsers (often regex-based) collapse under the corner cases.