What is GPTBot?

Why the three are different

GPTBot

Broad crawling for training and corpus building.

OAI-SearchBot

Builds the search index used to answer with citations. Does not train.

ChatGPT-User

Fires when a real person asks about your specific URL. The highest-intent fetch there is.

The block most sites do not know about

Many shared hosts return HTTP 403 to these user-agents at their edge proxy, before your server or robots.txt is consulted. Your robots.txt can say Allow: / to everyone and be entirely irrelevant, and nothing in your control panel mentions it.

Test by fetching your home page with each user-agent and reading the status code. Anything other than 200 means you are invisible to that engine regardless of what you publish.

Should you allow it?

For a publisher whose revenue depends on people arriving to read, there is a genuine argument for blocking. For a business selling services, blocking removes you from the answer at exactly the moment a buyer is asking who to hire.

Where this is covered in depth

A definition can only go so far. How to check whether AI crawlers can actually reach your site covers this properly — 3 minutes, free, no email required.

Who wrote this

Anas Bin Masud builds e-commerce sites and does technical SEO for businesses in the UK, Canada and Pakistan. These definitions come from client work rather than from a content brief — where an entry describes a mistake, it is usually one found on a real site. More about how I work.