Browser Rendering Pipeline

HTML Parsing and Tokenization

How Tokenization Fits in the Pipeline

The HTML parser runs synchronously on the main thread. It reads the raw byte stream, resolves character encoding, and emits a sequence of tokens — start tags, end tags, attributes, text, comments — that drive DOM construction one node at a time. Every token emitted immediately narrows the time available for style resolution, layout, and paint within the same frame budget. This is part of Browser Rendering Pipeline Fundamentals; tokenization is the first stage that feeds Render Tree Generation downstream.

Tokenizer to tree construction to DOM, with the preload scanner branch Bytes feed the tokenizer, which emits tokens to tree construction building the DOM; a parallel preload scanner branches off the byte stream to fetch subresources early. Main-thread parse Byte stream Tokenizer Tree construction DOM Preload scanner (parallel) Scan ahead Fetch css / js / fonts

Three things halt tokenization:

  1. Parser-blocking scripts. When the tokenizer encounters a <script> without async or defer, it must pause, wait for the network (if external), and hand control to the JavaScript engine. The DOM is frozen until the script completes.
  2. document.write calls. These inject new HTML at the current parse position, forcing the parser to restart from that point — see how document.write blocks the parser for the full stall mechanism.
  3. Malformed markup. Error-recovery logic can create unexpected subtrees that inflate DOM size and slow subsequent style resolution.

Deeply nested markup does not stall the tokenizer itself, but it increases the volume of nodes the engine must process during CSSOM Construction Rules and downstream style calculation, compounding the total time to first paint. Poor parsing performance cascades into delayed Render Tree Generation.

Trace Analysis

Isolate tokenization cost in Chrome DevTools by recording a Performance trace during page load. In the Main thread lane, filter for Parse HTML and Evaluate Script. A healthy load shows short Parse HTML slices interleaved only with preloaded resources. Prolonged Evaluate Script gaps are where parser-blocking scripts stall tokenization.

Main-thread timeline of a parser-blocking script stall A timeline where a short ParseHTML slice is followed by a long EvaluateScript block that stalls tokenization for roughly two frame budgets before parsing resumes. Main thread lane time ParseHTML 12.4ms EvaluateScript (parser-blocking) 32.7ms — DOM frozen ParseHTML resume UpdateLayoutTree 3.7ms ~2 frame budgets lost to a single script 
[Main Thread]
0.0ms  - 12.4ms: ParseHTML (Tokenize)         — within 16.6ms budget
12.4ms - 45.1ms: EvaluateScript (parser-blocking) — 32.7ms, ~2 frame budgets
45.1ms - 48.3ms: ParseHTML (resume)
48.3ms - 52.0ms: UpdateLayoutTree

The 32.7ms script evaluation consumes the equivalent of almost two frame budgets. Everything downstream — style resolution, layout, FCP — is delayed by the same amount. For a detailed walkthrough of debugging these stalls, see How browsers parse HTML into DOM nodes.

Optimization

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <!-- preload: starts the font fetch at high priority without blocking the parser -->
  <link rel="preload" href="/fonts/critical.woff2" as="font" crossorigin>
  <!-- defer: parser continues uninterrupted; script runs after DOMContentLoaded -->
  <script src="/app-bundle.js" defer></script>
</head>
<body>
  <main id="app-root">
    <section class="hero">...</section>
  </main>
  <!-- async: fetches and executes independently; does not block the parser -->
  <script src="/analytics.js" async></script>
</body>
</html>

What each attribute does:

  • defer — the browser continues parsing the full document; the script executes in document order after parsing is complete, before DOMContentLoaded. Use for scripts that depend on the DOM.
  • async — the script executes as soon as it downloads, potentially before parsing finishes, and out of order relative to other async scripts. Safe for analytics and independent widgets.

Neither attribute helps with inline <script> blocks, which always execute synchronously. Move logic out of inline scripts into deferred external files.

How blocking, defer, and async scripts interleave with parsing Three lanes compare a blocking script that pauses parsing, a deferred script that fetches in parallel and runs after parse, and an async script that runs the moment its fetch completes. parse fetch / execute blocking parse fetch + run (blocks) parse resumes defer parse uninterrupted fetch in parallel run after parse async parse run on fetch done parse resumes

Streaming HTML (chunked transfer encoding) allows the browser to start tokenizing the first chunk while the server is still generating the rest of the document. This overlaps network and parse time, reducing TTFB-to-FCP latency without code changes. Pairing streaming with inlined critical styles — see Critical Rendering Path Optimization — keeps the first paint from waiting on a round-trip for an external sheet, since CSS blocks rendering until the CSSOM is built.

Validation

After applying changes, re-run a Performance trace and verify:

  • No Evaluate Script gap during the ParseHTML phase (confirms scripts are no longer parser-blocking).
  • ParseHTML task duration scales linearly with payload size — a sign that no error-recovery overhead is inflating cost.
  • FCP and TBT improve in Lighthouse CI relative to the pre-optimization baseline.
Validation decision path after re-tracing a load A decision tree checks whether an EvaluateScript gap still interrupts ParseHTML, routing to either a remaining parser-blocking script or a passing linear-cost result. Re-run Performance trace gap inside ParseHTML? Yes — script still blocks add defer / async, retest No — cost is linear FCP + TBT pass in CI yes no loop back to Optimization

Target thresholds:

Metric Target
TBT < 200ms
FCP (4G / mid-tier device) < 1.8s
Main-thread blocking per task < 50ms

Deeper Dives

These focused walkthroughs expand on individual parser behaviours:

The Tokenizer State Machine

The HTML tokenizer is a state machine defined byte-for-byte in the WHATWG standard, and understanding that it is a state machine rather than a regex explains most of its surprising behaviour. It begins in the data state, emitting character tokens until it sees a <, which moves it into the tag-open state; from there the next character decides whether it is a start tag, an end tag, a comment, or a bogus-comment. Inside a tag it transitions through attribute-name and attribute-value states, handling quoting and entity references as it goes. The output is a stream of tokens — start tags, end tags, characters, comments, and the doctype — which the tree-construction stage consumes to build the DOM. Crucially, the tokenizer’s current state changes how following bytes are interpreted, which is why an unclosed attribute quote can silently swallow the rest of a line: the tokenizer is still in the attribute-value state, treating your markup as a value.

Two states matter disproportionately for performance and correctness. The <script> and <style> elements switch the tokenizer into raw-text-like states where markup is not parsed as tags, only scanned for the matching end tag — which is why a literal </script> inside inline script content terminates the block early and dumps the remainder into the document as markup. And RAWTEXT/RCDATA handling for <textarea> and <title> decodes entities but ignores tags. Knowing which state an element forces is the difference between debugging a mysterious “half the page vanished” bug in minutes versus hours. The tree-construction rules that run on top of these tokens are what how browsers parse HTML into DOM nodes walks through node by node.

[HTML tokenizer states — simplified path for a start tag]
data ──"<"──▶ tag-open ──alpha──▶ tag-name ──space──▶ before-attr-name
                                                   │
                                          attr-name ──"="──▶ before-attr-value
                                                   │
                                          attr-value(quoted) ──quote──▶ after-attr-value
                                                   │
                                                  ">" ──▶ emit start-tag, back to data

Where the Parser Yields and Stalls

DOM construction is incremental, but it is not uninterruptible. The single largest stall is a synchronous <script> without defer or async: when the tree builder reaches it, tokenization pauses, the script is fetched (if external) and executed to completion, and only then does parsing resume — because the script might call document.write and change the very bytes still being tokenized. This is the parser-blocking behaviour that the preload scanner exists to mitigate: while the main tree builder is frozen on the script, the scanner reads ahead in the raw byte stream and starts fetches for resources it can see, so the network is not idle during the stall. A stylesheet in the <head> adds a subtler dependency — a following script cannot run until the CSSOM is ready, because the script might query computed style, so a render-blocking stylesheet can transitively block parsing through a script that depends on it.

document.write deserves its own warning because it is the one API that mutates the input stream mid-parse. A call inside an inline script splices new bytes into the tokenizer’s input right after the script, and if those bytes contain another script the whole stall repeats, serialising what could have been parallel fetches. The modern replacement is to build content off-DOM and insert it once, or to inject scripts with async/defer instead of writing tags. The failure mode and its fix are traced in how document.write blocks the parser. Character-encoding discovery is a final, often-forgotten stall: if the charset is not declared in the first 1024 bytes, the parser may have to restart tokenization once it discovers the real encoding in a late <meta>, throwing away the work done so far — always declare <meta charset="utf-8"> as the first thing in the <head>.

Speculative Parsing and Error Recovery

Two properties of the HTML parser make it both fast and forgiving, and both have performance consequences worth understanding. The first is speculative parsing: while the main parser is blocked on a script, the preload scanner does not just fetch resources — some engines also speculatively tokenize ahead so the tree builder has less to do when it resumes. This work is discarded if document.write invalidates it, which is one more reason that API is expensive: it throws away speculative work the browser already did. The second property is error recovery. HTML has no “parse error stops the page” mode; the tokenizer and tree builder have defined recovery rules for almost every malformed construct — an unclosed <p>, a stray </div>, a table cell outside a row. The adoption agency algorithm reconstructs mis-nested formatting elements (<b>, <i>, <code>) by cloning them across the boundary, which is why a single unclosed inline tag can silently wrap far more of the document than you intended, duplicating that element around subsequent content.

For an author the lesson is that “it renders fine” is not the same as “it parses as written.” Malformed markup that the recovery rules paper over still costs parse time and can produce a DOM subtly different from your mental model — extra nodes from reconstructed formatting elements, reparented table content, or an element hoisted out of a context it is not allowed in. Validating markup and closing tags explicitly keeps the DOM the parser builds identical to the one you designed, which in turn keeps style calculation and render tree generation operating on the structure you expect. When a page behaves strangely, comparing the live DOM in the Elements panel against your source is often the fastest way to spot a recovery rule that fired. Server-rendered and template-generated markup is especially prone to this, because a conditional branch can emit an unbalanced tag that only appears for certain data — the kind of defect that passes a quick visual check but ships a subtly wrong DOM to a fraction of requests. Running the built HTML through a validator in CI, rather than eyeballing it, is the reliable guard: it catches the unclosed tag before the parser’s recovery rules quietly paper over it in production. The same discipline applies to any markup assembled from user or CMS content, where a stray angle bracket in a title or comment can open a tag the template author never wrote.

Frequently Asked Questions

Why does an unclosed inline tag wrap more of the page than expected?

Inline formatting elements like <b>, <i>, and <code> are tracked in the parser’s list of active formatting elements. When one is left open across a block boundary, the adoption agency algorithm reconstructs it by cloning the element around the following content so the formatting continues. That reconstruction is why a single missing </code> can wrap headings, lists, or entire sections in a <code> element the source never intended. Closing inline tags explicitly prevents it.

Does the tokenizer parse markup inside a `script` or `style` element?

No. Those elements switch the tokenizer into a raw-text state where it does not recognise tags at all — it only scans for the matching end tag. That is why a literal </script> inside inline script content ends the block early and the remainder spills into the document as markup, and why script and style content must escape or avoid that sequence.

Why does a plain script tag freeze the HTML parser?

A <script> without async or defer is parser-blocking by spec: the tokenizer must stop emitting tokens, wait for the file to download if it is external, then hand the main thread to the JavaScript engine to execute. Because the script could call document.write and change the byte stream, the parser cannot safely continue past it. The DOM is frozen for the full download-plus-execute duration.

Does the preload scanner keep working while a script blocks parsing?

Yes. The preload scanner runs on a separate lightweight pass over the raw bytes and continues discovering href and src resources even while tree construction is paused on a blocking script. It cannot build DOM, but it warms the network so CSS, fonts, and images are already in flight when parsing resumes.

Should I always prefer defer over async for scripts?

Use defer when a script depends on the DOM or on execution order — deferred scripts run in document order after parsing completes, just before DOMContentLoaded. Use async only for independent code such as analytics, because async scripts execute the moment their fetch finishes, in no guaranteed order.

How do I tell if error-recovery is inflating my parse cost?

Record a Performance trace and check whether ParseHTML duration scales linearly with payload size. A superlinear jump, or unexpected DOM nodes in the Elements panel, signals that malformed markup triggered the parser’s error-recovery paths and created extra subtrees that slow later style calculation.