Technical SEO
How to create an XML sitemap
Generate it from your build or CMS rather than writing it by hand, include only canonical indexable URLs that return 200, split it into a sitemap index if it exceeds 50,000 URLs, and reference it from robots.txt so any crawler finds it unprompted.
Before you start
- Access to the build or CMS
- A list of which pages should be indexed
The steps
-
Step 1: Generate rather than hand-write
A build plugin or CMS feature keeps it accurate as pages are added and removed.
-
Step 2: Include only indexable URLs
Canonical, 200-returning, not noindex, not blocked in robots.txt. Anything else sends contradictory signals.
-
Step 3: Set lastmod honestly
From actual content change, not from deploy time. A lastmod that moves on every deploy teaches crawlers to ignore the field.
-
Step 4: Skip priority and changefreq
Google ignores both. They add noise and imply a control you do not have.
-
Step 5: Split if it is large
Above 50,000 URLs or 50MB uncompressed, split into several files under a sitemap index.
-
Step 6: Reference it from robots.txt
A Sitemap: line with the absolute URL, so crawlers find it without being told separately.
-
Step 7: Verify it loads and validates
Open it in a browser, then submit it in Search Console and Bing Webmaster Tools.
The three ways sitemaps go wrong
Listing URLs that redirect, listing URLs that are noindex, and omitting pages that should be indexed. All three are contradictions between what the sitemap claims and what the page says, and all three are worth failing a build over rather than discovering months later.
What a sitemap does not do
It does not make anything rank, and it will not rescue a page nothing internally links to. A sitemap-only page typically sits at "Discovered — currently not indexed", which is the engine saying it found the URL and was not persuaded it mattered.
The background behind this
XML sitemaps: useful, misunderstood, and no substitute for links covers the reasoning in about 2 minutes — free, ungated, written from client work.
Who wrote this
Anas Bin Masud builds e-commerce sites and does technical SEO for businesses in the UK, Canada, Ireland and Pakistan. These procedures are the ones I run on client work, written down rather than invented — including the ones where the honest answer is that the fix is not available on your current hosting.