← Back to Blog

AI crawlers don't run your JavaScript. Check what they actually see.

If your pages render in the browser, ChatGPT, Claude and Perplexity may read an empty shell. How to tell, and how to fix it in Next.js, React and other frameworks.

Most AI crawlers read the HTML your server sends and nothing more. If your product page builds its content with JavaScript after the page loads, a crawler like GPTBot or ClaudeBot can receive a page with a header, a footer and a loading spinner. It cannot recommend a product it cannot read.

Who renders and who doesn't

Googlebot runs JavaScript. It queues pages for rendering and indexes what the browser would show, which is why client-rendered sites can still rank in Google. Gemini and Google AI Overviews draw on Google's index, so they inherit that.

The crawlers behind ChatGPT, Claude and Perplexity are different. A 2024 analysis of crawler traffic by Vercel and MERJ found that OpenAI's and Anthropic's crawlers downloaded JavaScript files but did not execute them. Until a vendor documents otherwise, assume these crawlers see only your raw HTML.

That gap explains a common puzzle: a site that ranks well in Google but never shows up in ChatGPT answers about its own category.

How to check your own pages

Open a product page and use "View page source", not the element inspector. The inspector shows the page after JavaScript has run. The source shows what a non-rendering crawler gets.

Search the source for a sentence you can see on the page, such as your headline or a price. If it is there, crawlers can read it. If the source contains only a root <div> and script tags, your content is rendered in the browser.

You can also fetch the page from a terminal:

curl -s https://yourdomain.com/pricing | grep -i "per month"

No match means the pricing text never reaches a crawler that skips JavaScript.

Promptvane's free AI readiness audit does this for your homepage and a sample of sitemap pages, and flags any page whose HTML carries almost no text.

Fixes by framework

The goal is the same everywhere: send the content in the first HTML response.

In Next.js with the App Router, pages are Server Components by default and render on the server. Problems start when a whole page is marked 'use client' and fetches its data in useEffect. Move the data fetching into the Server Component (or generateStaticParams for static pages) and keep 'use client' for the interactive pieces only.

In the Next.js Pages Router, use getStaticProps or getServerSideProps for content pages instead of fetching on the client.

For a single-page React or Vue app built with Vite, add prerendering for public pages, or move marketing pages to a framework that renders on the server such as Next.js, Nuxt, Astro or Remix.

WordPress, Webflow and most hosted site builders render on the server already. Check page builders and plugins that load content through JavaScript widgets, such as review carousels and pricing tables.

Content that hides in other ways

Server rendering is not the only trap. Content inside tabs and accordions is fine as long as it is in the HTML; content fetched only when the tab is clicked is not. Text inside images is invisible to crawlers that do not run OCR. Prices injected by a third-party checkout script often never appear in your source at all.

FAQ

Does Googlebot rendering JavaScript mean I'm fine?

For Google Search, mostly. For ChatGPT, Claude and Perplexity, no. Their crawlers have not been shown to execute JavaScript, so content that exists only after rendering is likely invisible to them.

Is it enough to server-render the homepage?

No. Assistants usually cite the page that answers the question: a pricing page, a comparison, a feature page. Every page you want cited needs its content in the HTML.

Will server rendering hurt performance?

Usually the opposite. Server-rendered and static pages show content sooner, which also helps Core Web Vitals.

GEO notes, every other week

What we learn tracking thousands of AI answers. No fluff.