Tools: Why AI Crawlers Can't See Your React App

How AI Crawlers Actually Work ## Why This Is Different from Google ## The Traffic Impact Is Real ## Which Frameworks Are Affected ## How to Check If You're Affected ## How to Fix It ## The Bottom Line Google has been rendering JavaScript since 2019. It's not perfect, but it works. So if you're building with React, Vue, or Next.js, you might assume search engines can see your content just fine. That assumption is about to cost you traffic. Because Google isn't the only crawler that matters anymore. AI-powered search engines are growing fast — ChatGPT, Perplexity, Google's AI Overviews, Microsoft Copilot. They're changing how people find information online. And almost none of their crawlers render JavaScript. When GPTBot (OpenAI's crawler), PerplexityBot, or ClaudeBot visit your site, they don't fire up a headless browser. They don't execute your JavaScript. They don't wait for your React components to mount or your API calls to resolve. They send a simple HTTP request, download whatever HTML your server returns, and move on. If your server returns a fully rendered page with all your content in the HTML, great — the AI crawler sees everything. If your server returns an empty shell with a script tag that loads your app, the AI crawler sees nothing. There's no rendering queue. There's no second pass. What your server sends in that first response is all they'll ever see. Google's approach to JavaScript has three steps. First, Googlebot fetches your raw HTML and extracts whatever it can. Second, it puts your page in a rendering queue. Third, Google's Web Rendering Service executes your JavaScript in a headless Chromium browser and indexes the final result. It's not instant — that rendering queue can take hours or days — but eventually Google sees your fully rendered page. AI crawlers skip steps two and three entirely. They only do step one. Fetch the HTML, read it, done. This means a client-side rendered React app that Google eventually indexes correctly might be completely invisible to every AI crawler on the internet. AI-powered search is growing fast. Perplexity reports millions of daily queries. ChatGPT's browsing feature is used by hundreds of millions of users. Google's own AI Overviews now appear on a significant percentage of search results. When these tools answer questions, they pull from content their crawlers have indexed. If your content isn't in their index because their crawlers couldn't see it, you won't be cited. You won't be recommended. You won't get that traffic. This isn't a future problem. It's happening right now. Not all JavaScript apps have this problem. It depends on how your framework renders pages. Client-side rendering is the most affected. This is the default for Create React App and vanilla single-page applications. The server sends an empty HTML shell, and the browser builds the page entirely with JavaScript. AI crawlers see nothing but an empty div and a script tag. Server-side rendering is safe. Frameworks like Next.js (with getServerSideProps or Server Components), Nuxt.js, and SvelteKit can render your pages on the server before sending them to the browser. The AI crawler gets a fully rendered HTML page on the first request. Static site generation is also safe. If you pre-build your pages at deploy time (Next.js static export, Astro, Gatsby), the HTML files already contain all your content. No JavaScript execution needed. The tricky case is hybrid rendering. Many Next.js and Nuxt apps use a mix — some pages are server-rendered, others are client-rendered. You might have SSR on your landing page but client-side rendering on your blog or product pages. This creates a situation where some pages are visible to AI crawlers and others aren't, which is hard to diagnose without testing each page individually. The fastest test takes 30 seconds. Open your site in Chrome, right-click anywhere, and select "View Page Source" — not "Inspect Element." View Page Source shows you the raw HTML your server sends before any JavaScript executes. This is exactly what AI crawlers see. If your main content, headings, meta descriptions, and navigation links are all in that source HTML, you're fine. If you see mostly empty divs, script tags, and a loading spinner, AI crawlers are seeing the same empty page. For a more thorough check, you can compare how your page looks to a regular user versus how it looks to different crawlers. Tools like JSVisible render your pages from multiple perspectives — user browser, Googlebot, and raw HTML — so you can see exactly what each crawler sees and where the gaps are. If you're on a framework that supports server-side rendering, the fix is straightforward. Switch your critical pages from client-side to server-side rendering. In Next.js, this means using Server Components (the default in the App Router) or getServerSideProps in the Pages Router. In Nuxt, it means using universal rendering mode. You don't have to server-render everything. Focus on the pages that matter for discovery — your homepage, landing pages, blog posts, product pages, and any page you want search engines and AI tools to find. If you can't switch to SSR, prioritize getting critical SEO elements into your server-rendered HTML at minimum. Your page title, meta description, canonical URL, and H1 heading should all be in the initial HTML response, not injected by JavaScript after the page loads. Many frameworks have head management utilities that handle this even in client-rendered apps. For internal links, always use proper anchor tags with href attributes instead of JavaScript click handlers. AI crawlers follow links by reading href values from the HTML. If your navigation uses onClick handlers or client-side routing without real anchor tags, crawlers can't discover your other pages. The web runs on JavaScript, but AI crawlers don't. Google's ability to render JavaScript gave developers a false sense of security. The new generation of AI-powered search tools has exposed a gap that was always there — the gap between what a browser sees and what a simple HTTP request returns. If your content depends on JavaScript to appear, you're visible to Google (eventually) but invisible to the fastest-growing discovery channel on the internet. The fix isn't complicated. Server-side render your important pages, put critical SEO elements in your initial HTML, and use real links. The first step is knowing whether you have a problem. Check your page source. If it's empty, so is your presence in AI search. Templates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse