How to check what a crawler actually sees on your page

Before you start

• A browser
• Optionally a terminal with curl
• Search Console access for the definitive check

The steps

1. 1

   Step 1: Open view-source

   Right-click the page and choose View Page Source, or prefix the URL with view-source:.

2. 2

   Step 2: Look for your content

   Search the source for a distinctive sentence from the page. If it is not there, it does not exist until JavaScript runs.

3. 3

   Step 3: Count the words

   Use the curl command below to get a number. Compare it against what you see on screen.

4. 4

   Step 4: Check in Search Console

   URL Inspection → Test Live URL → View Crawled Page. This is Google telling you what it actually received.

5. 5

   Step 5: Test with JavaScript disabled

   Disable JavaScript in DevTools settings and reload. Whatever remains is what a non-rendering consumer gets.

Count the words a non-rendering crawler receives

curl -s https://example.com/ \
  | sed -e 's/<script[^>]*>.*<\/script>//g' -e 's/<[^>]*>/ /g' \
  | tr -s ' \n' ' ' | wc -w

What a large gap means

It means your content is client-rendered. Google will usually render it eventually through a deferred queue; AI crawlers generally will not render at all, so for them the page is empty. The fix is static generation or server rendering.

The number that made this concrete

The previous version of this site returned 73 words in view-source against more than 800 on screen. Everything a buyer needed to read lived inside a JavaScript bundle, and everything that could not execute it saw an apology for requiring JavaScript.

What to do about a large gap

Move the content into the initial HTML response. On a framework site that means server-side rendering or static generation; on a WordPress site it usually means removing whatever is deferring the content into JavaScript. There is no configuration flag or plugin that makes client-rendered content reliably visible to a non-rendering crawler, which is why the architecture decision matters more than any optimisation after it.

The background behind this

JavaScript SEO: the difference between what you see and what gets indexed covers the reasoning in about 3 minutes — free, ungated, written from client work.

Who wrote this

Anas Bin Masud builds e-commerce sites and does technical SEO for businesses in the UK, Canada, Ireland and Pakistan. These procedures are the ones I run on client work, written down rather than invented — including the ones where the honest answer is that the fix is not available on your current hosting.