1. Home
  2. Insights
  3. What makes a website AI-optimized in 2026
Web build

What makes a website AI-optimized in 2026

AI crawlers now generate roughly 28% of Googlebot request volume and they do not run your JavaScript. What an AI-optimized website means, stripped of the marketing.

The short answer

An AI-optimized website is one where the substance renders on the server, the structure is semantic rather than merely visual, the structured data resolves into one connected entity graph, and the crawler policy is a decision rather than an accident. Everything else marketed under that label is either ordinary technical SEO or decoration.

It is not a design style, and it is not a website that was designed by AI.

The crawler reality

AI crawlers now generate roughly 28% of the request volume Googlebot does. GPTBot alone makes around 569 million requests a month and Claude’s crawler around 370 million. This is no longer a marginal audience for your HTML.

The important part is how they read. They are far less tolerant of client-side rendering than a modern search crawler, and they do not interact with your page. If your headline, your prices, your proof points or your FAQ answers are painted in by JavaScript after load, there is a real chance the systems now recommending suppliers to your customers never see them.

Render the substance on the server

The rule is narrow and easy to test: everything that constitutes a claim should be present in the HTML source. Open your page, view source, and search for your own headline. If it is not there, that is the entire finding.

This does not mean abandoning JavaScript. Interactivity, personalization and application behavior can all stay client-side. What cannot stay client-side is the text that makes your argument. Server-side rendering or static generation for content, progressive enhancement for behavior, is the shape that satisfies both readers.

Semantic structure, not visual structure

A model infers hierarchy from your markup, not from your type scale. A heading that is visually large but marked up as a styled div carries no structural weight at all.

  • One h1 per page, and a heading order that never skips a level.
  • Lists marked up as lists, tables as tables, and quotes as blockquotes.
  • Landmarks: header, nav, main, article, footer.
  • Headings phrased as the question the section answers, because that is how prompts are phrased.
  • Logical taxonomies with parent and child URL paths, and internal links that stay within a topic cluster, which is how a crawler learns that a set of pages is one subject.

One entity graph, not scattered snippets

Most sites that have structured data have it as fragments: an Organization block on the home page, a Product block on product pages, and nothing joining them. A model resolving "who is this company and what do they sell" gets a pile of unconnected assertions.

A graph is different. Every node carries a stable identifier, and nodes reference each other by it, so Organization, Service, Product, Article, Person and Breadcrumb resolve into one description of a single business. Build it from the content model so it cannot drift out of sync with the page, and validate it as part of the build rather than once at launch.

Routes an agent can complete

The next requirement is already arriving: agents do not only read pages, they attempt tasks. Finding a product, comparing two options, requesting a quote, starting a checkout.

Practically that means forms that work without a mouse and without a bespoke JavaScript widget, search that responds to a plain URL parameter, product and availability data exposed in a feed or an API rather than only in a rendered grid, and no critical action locked behind a hover, a drag or a canvas. Most of this is the accessibility work you should be doing anyway, which is the happiest overlap in this field.

Speed is still a ranking of sorts

Core Web Vitals continue to matter for the Google index, and the Google index is what AI Overviews draws from. Beyond ranking, a heavy page is a page a crawler samples less thoroughly and a buyer abandons faster. Agree a performance budget before the build, enforce it during the build, and measure it in the field afterward rather than in a lab.

What AI-optimized does not mean

  • It does not mean the site was designed by AI. Those are unrelated claims that share a word.
  • It does not mean an llms.txt file. Publish one, it costs an hour, and do not confuse it with the work.
  • It does not mean a chatbot in the corner. A support widget is a support feature, not a visibility feature.
  • It does not mean stripping the design back. Nothing above requires a plain page. It requires the substance to exist in the source, which is a build decision rather than an aesthetic one.

The checklist

Ten things to check on your own site this afternoon. Each one is a yes or a no.

  • Your headline and main copy appear in view-source.
  • Exactly one h1, and no skipped heading levels.
  • Structured data present, valid, and cross-linked by identifier rather than isolated.
  • A visible last-updated date on anything time-sensitive.
  • robots.txt states a deliberate position on GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot and Google-Extended.
  • An llms.txt file exists at the root.
  • Your key facts are written as short standalone sentences, not buried in compound paragraphs.
  • Every form can be completed with a keyboard alone.
  • Core Web Vitals pass in field data, not just in a lab test.
  • Internal links stay inside topic clusters rather than pointing everywhere equally.

Any no on that list is a finding. Most sites we audit produce four or five.

Questions

Asked about this piece.

Does an AI-optimized website need to be built in a specific framework?

No. What matters is that content renders on the server, not which tool does it. Next.js and Astro do it well, WordPress does it by default, and Shopify does it for the parts that matter. The failure mode is a client-rendered single-page application that paints in its text after load, and that can happen in any framework.

Will making my site AI-optimized hurt the design?

It should not. Nothing in the requirement list is about visual restraint: it is about where the text lives in the source, how the markup is structured and whether the structured data connects. A rich, animated, visually ambitious site can satisfy every item on the checklist, and this one does.

How do I know if AI can read my site today?

Two quick tests. View source and search for your own headline, which tells you whether the content is server-rendered. Then read your robots.txt and check what it says about GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot and Google-Extended. Those two checks find most of the problem on most sites, and both take five minutes.

The service behind this

We do this for a living.

Find out where AI sends your customers today.

We run your brand through the answer engines your customers use, then send you the transcript of what they say about you. Free, and yours to keep whether or not we work together.