SEO Part 2 — Making Dynamic Sites Friendlier to Google's Crawler
How Dynamic Sites Affect SEO
Today's dynamic sites lean heavily on JavaScript and Ajax-related techniques, but those techniques aren't very friendly to Google's crawler. According to Google's official documentation, the Googlebot crawler understands HTML structure very well, but still has trouble crawling JavaScript and Ajax. To improve crawler-friendliness, Google still recommends pointing content at HTML.
Link to Google's official guide
Key Takeaways from the Official Docs
- Avoid putting SEO text inside a canvas or image — Googlebot can't recognize those either.
- Avoid iframes, or link individually to their content — content shown via iframe may not be indexable.
- Avoid serving different content to users vs. Googlebot. Sneaky redirects
- Text should still be visible when JavaScript is disabled.
- Links with parameters aren't crawler-friendly either; pages meant for crawlers should still use static URLs.
Solutions for SEO on Dynamic Sites
As mentioned, dynamic sites aren't very crawler-friendly. As frontend engineers, we can use the following rendering techniques to improve SEO on dynamic sites.
1. Server-side rendering (SSR)
The page data is fully rendered on the server before being returned to the client.
Common frontend SSR frameworks: Next.js in the React ecosystem Nuxt.js in the Vue ecosystem Universal in the Angular ecosystem
Pros
- When a crawler visits the site, it can immediately get the latest rendered page.
- Content is delivered to the client faster, especially over slow networks or on slow devices (countries with weaker network conditions). Users don't have to wait for all the JavaScript to download and execute before they see the server-rendered markup, so they see a fully rendered page faster — better user experience.
Cons
- Rendering is offloaded to the server, which can lead to high server load.
- Deployment has higher requirements; unlike a single-page app (SPA), an SSR app needs to run on a Node.js server.
2. Page prerendering
2-1. Custom crawler prerendering (Prerender, Playwright)
When a crawler visits the site, the web server or app inspects the crawler's User-Agent and returns a pre-rendered page.
Pros
- Reduces server load — the crawler can get results without waiting for rendering.
Cons
- Crawlers can't get the most up-to-date dynamic content.
- The technical bar is fairly high. You may need to write a job to render or crawl your own site, plus code to detect Googlebot and serve the matching static content.
2-2. Webpack prerendering (webpack prerender-spa-plugin)
Pros:
- Simple configuration — render HTML based on routes.
Cons:
- Prerendered code is mixed in with the existing site, so existing code needs adjustments.
- Doesn't support scroll-based lazy loading or virtual scrolling.
- You can't operate on the prerender internals, which lacks flexibility.
3. Dynamic Rendering for SEO
Rendertron is a project from the Google Chrome team built with Node.js and Puppeteer. It can solve the problem of dynamic site content that search engines can't crawl.
How it works The web server inspects the search engine's bot user-agent and routes the request to your self-hosted Rendertron server. Rendertron uses its cache if there's a hit; otherwise it spins up Puppeteer to crawl your site and bring the content back.
Pros
- Performance
- Both freshness and immediacy
- Less server load
Cons
- You need to host an additional dynamic rendering service.
P.S. Advanced scenarios — virtualized lists, lazy load, and finer-grained browser control — aren't supported, but the source is open and easy to modify, so you can extend it yourself.
Test
https://render-tron.appspot.com/render/欲動態渲染的網址
Comments