Complete Guide

Your Heading Does Not Travel With Your Paragraph: What Chunking Does Before an AI Reads You

Retrieval systems split your pages into passages before anything reads them, and in at least one documented pipeline the heading above a passage is left behind by default. This guide explains the mechanism inside Floor Two of the four-floor model, what breaks when a page is cut up, and why a token count is not a content rule.

8 min read 1,644 words Published Nov 2026

The Two Ways a Machine Reads Your Page

There are two, and almost everything written about AI visibility assumes the first one.

The first is a fetch. An assistant decides your page is worth reading, requests it, converts the HTML to text and reads it as a whole page, up to whatever its truncation limit happens to be. Your heading structure, your running order and your opening paragraph all arrive together.

The second is a retrieval index. Your page was crawled earlier, parsed, split into passages and stored. When a question arrives, the system returns the passages that match it best. Nothing reads your page. Something reads a fragment of it, selected by a process you never see, and often without the rest.

The second path is where the surprises live. It also explains a problem a lot of people have noticed without being able to name: content that is accurate, well written and perfectly findable, which still gets quoted wrongly, or not at all.

The Photocopy

Somebody photocopies three pages from the middle of a report and hands them to you. No cover. No contents page. The running header did not come out.

You can read every word on the paper. You still cannot say which report it came from, which product it describes, which version, or whether the figure halfway down is a maximum, a minimum or a default. If the first line reads “this setting overrides the default,” you cannot act on it at all, because nothing on the page you are holding says which setting.

That is a chunk. The paragraph arrived. The heading above it did not.

What One Documented Pipeline Actually Does

Most writing on this subject stays at the level of “AI reads in passages,” which is true and useless. Google Cloud’s Agent Search, formerly Vertex AI Search, publishes enough of its behaviour to be specific, and the specifics are more uncomfortable than the generalisation.

  • Chunks follow layout. Headings and lists decide where a chunk starts and ends.
  • Chunk size is counted in tokens, with a default and a maximum of 500. Google estimates 100 tokens at roughly 60 to 80 English words, so a chunk holds something like 300 to 400 words of prose. Code, URLs and tables consume more tokens per word, so a section heavy with those fits less.
  • Ancestor headings are off by default. There is a setting that puts the headings above a chunk inside the chunk. It has to be switched on.
  • Only the layout parser detects headings at all. The default parser treats the page as plain text blocks, so your heading structure helps only if the index was built with the right parser.

The third of those is the one that matters most and gets discussed least. A passage can reach a model carrying nothing but the page title and the URL. Not the section heading that told a human reader what the passage was about.

And none of this is standardised. Which heading levels a chunker splits on is a design decision made by whoever built the pipeline, not a rule anybody publishes.

This Is the Engine Inside Floor Two

The four-floor model puts extractability on Floor Two: if your content cannot be extracted cleanly, it cannot be cited, and if it cannot be cited it cannot be recommended. That floor has always been right. It has also been under-explained, because it described a symptom without naming the mechanism.

Chunking is the mechanism. A passage is extracted, and whether it survives extraction depends on whether it still makes sense alone.

It also explains something about the CITATE framework that has previously been justified by observation rather than by cause. CITATE asks for a specific verifiable data point attributed to a named source, in the same sentence as the claim. Read as a style preference, that is fussy. Read as a chunk-survival rule, it is structural: a figure separated from its subject by a paragraph break is a figure that can arrive alone and mean nothing.

Self-containment was never about elegance. It is about what happens to a sentence when the page around it is taken away.

What Breaks When a Page Is Cut Up

Each of these is a normal, defensible editorial choice that behaves badly under retrieval.

PatternWhat happens to it
A long section under one headingIt splits. The second and third pieces begin partway through an argument, with no heading and no set-up.
Vague headings“Introduction,” “Key features” and “Known limitations” could belong to any page on the internet. “Known limitations of offline caching” identifies itself.
Styled paragraphs used as headingsStyling is stripped in conversion. A paragraph made to look like a heading with CSS arrives as a paragraph, so the chunker never sees a boundary there.
Long reference tablesThe header row may not repeat. Later rows can arrive as bare delimited values, so a column reading “No” could be a default, a requirement or a support flag, and a reader cannot tell which.
Long code samplesSplit in the wrong place and neither half runs. One piece has the imports and setup with no call, the other has the call with nothing defining what it acts on.
Information carried only in an image or diagramWhether it survives depends entirely on the parser. Inline SVG is frequently dropped or flattened into disconnected labels.
Long numbered proceduresA chunk can begin at step eight, without the prerequisites. A warning placed after the list can end up separated from the step it warns about.
Tabbed contentTabs serialise into one long sequence, so a section that looks short on screen can be several times its apparent length, and the labels that said which platform each part referred to may not survive.

Notice what is not on that list: anything about keywords, density or length as a ranking signal. These are structural failures. The content is correct and the container loses it.

The Cover Test

One test, and it takes a minute per section.

Cover the heading with your hand and read the section underneath it. Can you still tell what it is about, which product or service it refers to, and what the numbers in it mean?

If you cannot, a retrieval system holding that passage cannot either, and it has no hand to lift.

The fixes are unglamorous and mostly structural. Keep a section short enough to survive in one piece. Make headings specific enough to identify the page they came from. Use real heading tags. Break a long reference table into smaller tables under headings that name what they cover. Keep a code sample and the sentence introducing it together. Never let a diagram be the only place a fact lives. Break long procedures up and keep each warning inside the step it belongs to.

Almost none of this changes what your content says, which is the practical appeal: structural corrections do not need the review that substantive ones do.

What This Does Not Mean

This is where the subject is about to go wrong in public, so it is worth being exact.

500 tokens is not a rule. It is one pipeline’s default. Another pipeline splits at different heading levels, at a different size, with ancestor headings on. Anyone quoting a token count as a general content guideline has turned one vendor’s configuration into a law.

This is not a description of how ChatGPT, Perplexity or AI Overviews cite the open web. The documented behaviour above comes from retrieval indexes built over documentation, including the ones behind MCP servers. Public assistants citing public pages are a related but separate path, and nobody has published their chunking behaviour. What transfers is the direction, because every retrieval system splits content somehow. What does not transfer is any number.

It does not contradict Google’s guidance either. Google has said site owners do not need to break content into chunks for its AI features to understand a page, and that is accurate for Google Search, which parses a whole page and can surface the relevant part of it. A retrieval index is a different system, and it chunks you whether you participate or not. Both statements are true. They are routinely collapsed into one, and that is the error.

So the honest position is narrow. Writing passages that stand on their own costs nothing, helps human readers, and protects you in any system that splits content. That is the whole argument. It does not need a token count to be worth doing.

This guide was written to its own rules. Cover any heading above and the section below it should still tell you what it is about.

Evidence ledger, last reviewed 2 November 2026
What is documented
Google Cloud Agent Search, formerly Vertex AI Search, offers layout-aware chunking in which chunk size is measured in tokens with a default and maximum of 500; Google estimates 100 tokens at roughly 60 to 80 English words. Including the headings above a chunk inside that chunk is a setting that is off by default. Heading detection requires the layout parser; the default parser treats content as plain text blocks. Google has separately stated that site owners do not need to chunk content for its AI features in Search. Reported and assembled by Tom Johnson in his agent readability series, 31 October 2026, from Google Cloud documentation.
What we infer
That the direction generalises: any system that splits content before retrieval rewards passages that stand alone, and penalises figures separated from their subject. This follows from the mechanism rather than from measurement, and it is the basis for treating self-containment as structural rather than stylistic.
What is not established
That 500 tokens, or any token count, applies outside the pipeline that publishes it. That this describes how public AI assistants chunk open-web pages, which none of them document. That any measured ranking or citation uplift attaches to chunk-aware editing. No controlled study is cited here because none is known to us.

Frequently Asked Questions

What is content chunking?

Chunking is the step where a retrieval system splits a page into passages before storing it in an index. It happens when the index is built, not when a question is asked. When an assistant later searches that index, it receives the matching passages rather than the whole page, so a passage may be all that anything ever reads.

Does my heading get included with the passage?

Not necessarily. In Google Cloud Agent Search, including the headings above a chunk is a setting that is off by default, so a passage can arrive carrying only the page title and URL. Other pipelines make their own choice, and most do not document it.

How long should a section be?

Short enough to survive in one piece in whatever pipeline is indexing you. Where a 500-token limit applies that is roughly 300 to 400 words of prose, and less where the section contains code, URLs or tables. Treat that as an illustration rather than a target, because the limit is a configuration choice and not a standard.

Does this apply to ChatGPT and Google AI Overviews?

The direction does, because any system that splits content before retrieval favours passages that stand alone. The specifics do not. The documented behaviour comes from retrieval indexes built over documentation, and no public assistant publishes how it chunks open-web pages. Anyone quoting a token count as a general rule for AI search is overstating what is known.

How do I check whether my content survives chunking?

Cover the heading and read the section underneath. If you cannot tell what it is about, which product it refers to, or what its figures mean, a retrieval system holding that passage cannot either. Fixes are structural: shorter sections, specific headings, real heading tags, smaller tables, and no fact that exists only inside an image.

Sean Mullins

Founder of SEO Strategy Ltd with 20+ years in SEO, web development and digital marketing. Specialising in healthcare IT, legal services and SaaS — from technical audits to AI-assisted development.

Ready to improve your search visibility?

Book a free 30-minute consultation and let's discuss your SEO strategy.

Get in Touch