SEO · 10 min read
What Google does with your JavaScript, and where it stops working
Google crawls, queues the page for rendering, runs the scripts, and only then indexes what is there. Almost every JavaScript search problem is a consequence of that sequence. Here is the pipeline, the failures it genuinely causes, and the ones it gets blamed for unfairly.
Written by Zubair Afzal, FounderUpdated
What it looks like
Four symptoms that point at rendering rather than anything else
Rendering faults are unusual in that the site looks entirely healthy from the inside. Everything works in a browser, so the investigation tends to start somewhere else and stay there for weeks.
- Search results show the page title and a description that is not on the page.
- A page whose body arrives from scripts can be indexed as a shell with a title and very little else. The snippet gets composed from whatever text was available, often a navigation label or a fallback message. If a set of your results reads oddly generic, look at what is in the response before the scripts run.
- The number of indexed pages is a fraction of the number published.
- Where routing produces states rather than addresses, or where listings only load on scroll, the individual pages may never be discovered. The site appears complete to anybody browsing it, and a crawler simply never encounters most of it because there was never a link to follow.
- A section that used to work stopped after a front-end change.
- The most common single cause we find is a robots.txt rule blocking script or data paths, added at some point to reduce crawler requests on assets. Google will not render JavaScript from blocked files or on blocked pages, so the effect is a set of pages that fetch successfully and contain nothing.
- Pages that no longer exist return a success status.
- An application that handles missing content in the browser typically responds 200 and then shows an error message. To a crawler that is a working page with unhelpful content, which is what a soft not-found is. On a catalogue with turnover, this accumulates into thousands of addresses that all look valid and say nothing.
Diagnosis
Three questions that locate the fault
Work down these in order. Each answer either ends the investigation or tells you which of the two documents to look at next.
Is the content in the raw HTML response?
Yes: Not a rendering fault. Look at indexing, duplication and whether the page is worth storing.
No: The content depends on rendering. Carry on down.
Are the scripts and data endpoints crawlable?
Yes: Rendering can happen. Check what it actually produces.
No: Blocked resources cannot render. Remove the robots.txt rules first and retest.
Does the rendered HTML contain the content?
Yes: Rendering works. The cause is elsewhere — links, directives or page value.
No: The content never reaches the index. Move it into the server response.
The mechanism
Crawl, queue, render, index — and what each stage can refuse
Google documents processing a JavaScript site in three phases: crawling, rendering and indexing. Googlebot fetches the address and parses the response for links, and the page is queued for rendering, where the scripts run. Only then is the resulting content indexed. The word queued is the important one — rendering happens, but not necessarily at the moment of the fetch.
Two conditions in the documentation cause rendering to be skipped altogether. A non-200 status may skip it, which matters for anything erroring intermittently. And where the original HTML already carries a noindex instruction, Google may skip rendering and script execution, which is why a template that ships noindex by default and corrects it in the browser is an unreliable arrangement rather than a clever one.
A third condition removes the possibility earlier still. Google will not render JavaScript from blocked files or on blocked pages, so a robots.txt rule covering a scripts directory, a bundle path or the API the page fetches its content from produces a page that fetches successfully and is empty.
The routing implication is separate and just as consequential. The documentation asks for the History API rather than fragments for routing between views, because a fragment-based address is not reliably resolvable. If your product pages exist only as states after a hash, there is nothing for a crawler to request.
The two models
What changes between rendering on the server and in the browser
Neither is correct in the abstract. The question is what has to be discoverable, and how much you are prepared to depend on a stage you do not control.
| Dimension | Rendered on the server | Rendered in the browser |
|---|---|---|
| What a crawler receives first | The content and the links, in the response, with no further work required. | A shell. The content exists only after the scripts have been fetched and executed. |
| If something fails | A script failure degrades the interactive layer. The content is still there. | A script failure, a blocked file or a slow data call produces an empty page. |
| Effect on crawling | Ordinary. One request returns everything needed to understand the page. | Heavier. The page plus its scripts and data must be fetched and executed. |
| Effect on loading metrics | Usually better, because the largest visible element is in the first response. | Usually worse on the first visit, and dependent on the visitor’s device. |
| Routing | Every view is a real address by construction. | Real addresses only if the History API is used deliberately. |
| Where it is the right choice | Anything that has to be found: articles, products, categories, locations, services. | Anything behind a login, or highly interactive views nobody needs to find in search. |
| The usual mistake | Rendering everything on the server including views nobody will ever search for. | Building the public marketing site the same way as the logged-in application. |
Checks
How to find out what a search engine actually receives
None of this requires specialist tooling, and it settles arguments faster than any amount of discussion about frameworks.
- Read the raw response rather than the browser inspector. The inspector shows the document after scripts have run; the raw response shows what arrived.
- Load the page with JavaScript disabled and note what survives. Anything that disappears is content that depends on a stage you do not control.
- Run a live URL inspection in Search Console and read the rendered HTML it returns, plus the list of resources it could not load.
- Check robots.txt for rules covering script paths, bundle directories, or the API endpoints the page fetches its content from.
- Take a distinctive sentence from a page and search for it in quotation marks with a site restriction. If the page is indexed but that sentence is not findable, it did not survive to the index.
- Click through the site with scripts disabled and see how far you can get. Navigation that stops working is navigation a crawler cannot follow either.
- Request a deliberately invalid address and read the status code. A 200 with an error message on it is a soft not-found, and at scale it is a real problem.
- Compare the count of published pages against the count indexed in Search Console. A large unexplained gap on a JavaScript site is usually discovery rather than quality.
The honest part
Most sites do not have a JavaScript SEO problem
The mainstream frameworks now render on the server by default, so the population with a genuine rendering fault has narrowed to a specific and identifiable list. Assuming your site is on it, without checking, is how a diagnosis gets aimed at the wrong thing for a quarter.
The list is short. Applications built before server rendering became the default. Content that only appears after a click, a tab change or a scroll. Listings with no paginated addresses behind them. Routing that produces states rather than addresses. Sites where directives are applied by scripts. Sites with an inherited robots.txt rule blocking their own assets.
If none of those describes your site, the two-document check above will take ten minutes and let you rule the whole subject out. That is worth doing before commissioning a rendering project, because a server-rendering migration is a substantial engineering commitment, and buying one to solve a problem you did not have is an expensive way to be reassured.
It is also worth being precise about what server rendering does. It makes content reliably available and usually improves loading measurements. It does not make a page more useful, more authoritative, or more likely to be the best answer to a search. Those remain separate problems with separate solutions.
Questions
What development teams ask about this
Can Google index content that JavaScript creates?
Yes, generally. Google documents processing JavaScript applications in three phases — crawling, rendering and indexing — and the rendering phase executes scripts before the content is indexed.
The qualifications matter more than the headline. Rendering is queued rather than immediate, it will not happen for pages or files blocked in robots.txt, and it may be skipped where the initial response is a non-200 status or already carries a noindex instruction.
Do we have to server-render everything?
No, and treating it as an all-or-nothing decision is usually what makes the project unaffordable. What matters is that the content and links defining the page exist in the response, and that anything a search engine needs to find is reachable through real addresses.
A reasonable split is to render the parts of the page that define what it is on the server, and leave genuinely interactive parts to the browser. A dashboard behind a login has no reason to be server-rendered; a product listing does.
Why is our content in the browser but not in search?
Four causes account for most of it. The scripts or the data they fetch are blocked in robots.txt. The content only appears after an interaction such as a click or a tab change. The route has no real address because it is fragment-based. Or the initial response already says not to index, and the correction happens too late.
Each of these is checkable in a few minutes. Request the page as a crawler would and read the raw response, then compare that against the rendered result in a live inspection tool. The gap between the two is your answer.
Is infinite scroll a problem?
It is if it is the only route to the content. A crawler does not scroll, so anything that only loads in response to scrolling may never be discovered unless there is another path to it.
The usual remedy is not to remove the interaction but to add real paginated addresses behind it, linked properly, so the same content is reachable both ways. Keep the experience for people and give crawlers an ordinary set of links.
Does rendering cost us crawl budget?
It costs resources, and Google’s crawl budget documentation is explicit that resource-heavy sites can consume crawl capacity faster. On a small site this is irrelevant. On a large one it compounds with everything else.
The practical effect is that a site producing hundreds of thousands of addresses that each require rendering will be crawled less thoroughly than an equivalent site serving the same content in the initial response.
Can we set the canonical or the robots tag with JavaScript?
It is a fragile arrangement and worth avoiding. The documentation notes that a noindex present in the original HTML may cause rendering to be skipped, which means the tag you intended to remove in the browser is the one that takes effect.
Treat directives as something the server states. Anything deciding whether a page is indexed, or which version is canonical, should be in the response rather than applied afterwards by a script that may not run in time.
Not sure whether rendering is your problem
It is answerable in an afternoon rather than a project. Send us the site and the pages that are missing, and we will tell you whether the content survives to the index and where it stops if it does not.
Last updated