The short version
- A sitemap aids discovery. It is not a ranking signal and it does not guarantee indexing.
- It matters most on large sites, brand-new sites, and sites where internal linking is weak. On a well-linked ten-page site it barely moves anything.
- Only include URLs you actually want indexed: 200 status, canonical version, not blocked, not
noindex. - The limits: 50,000 URLs and 50 MB uncompressed per file. Past that you need a sitemap index.
- Check yours free — status, URL count, index detection and structural problems in one fetch.
What a sitemap is actually for
An XML sitemap is a machine-readable list of the URLs on your site that you would like crawled. That is the whole feature. It does not describe the pages, argue for their quality, or influence where they rank.
It helps with one specific problem: discovery. A crawler finds pages by following links. If a page is linked from nowhere, or linked only from a page that itself is rarely crawled, it can take a long time to be found — or never be found. A sitemap is a direct route around that.
Which also explains its limits. Listing a page in a sitemap does not oblige Google to index it. If the page is thin, duplicated, or judged not worth the space, it will be crawled and declined, and Search Console will report exactly that: discovered, not indexed. The sitemap did its job; the page did not.
A sitemap answers “does this page exist?” It has nothing to say about “is it worth anything?”
When it matters and when it does not
It matters a great deal
- Large sites. Thousands of URLs, where crawl budget is genuinely finite and discovery order affects what gets seen this month.
- Brand-new sites. Few or no inbound links, so there is nothing pointing crawlers at you at all.
- Sites with weak internal linking. Orphan pages reachable only through a search box or a filter.
- Sites that publish frequently. A fresh
lastmodis a useful hint about what has changed since the last crawl. - After a migration. A correct sitemap is one of the fastest ways to get a new URL structure processed.
It barely matters
- A well-linked ten-page site. Everything is two clicks from the homepage. The sitemap is a formality — have one, but do not expect it to change anything.
- As a fix for pages that are not being indexed. If Google has crawled a page and declined to index it, adding it to a sitemap again changes nothing. The page is the problem.
What belongs in one — and what never does
The rule is simple and frequently broken: a sitemap should contain exactly the URLs you want indexed, in their canonical form. Every entry is a statement that this URL matters. Contradicting that statement elsewhere on your site makes both signals weaker.
Include:
- Pages returning 200, not a redirect and not a 404
- The canonical version of each URL — matching protocol, subdomain and trailing slash exactly
- Pages that are crawlable and not marked
noindex
Never include:
- URLs that redirect — put the destination in instead
- Pages blocked in robots.txt, which is a direct contradiction
- Pages carrying
noindex— same contradiction, opposite direction - Non-canonical variants, parameterised URLs, tagged campaign links
httpURLs on anhttpssite- URLs on a different domain, which are simply ignored
Free tool · See what crawlers see
Fetch your sitemap the way a crawler would
Enter the sitemap address and get the HTTP status, the URL count, whether it is an index, and any structural problems — protocol mismatches, foreign domains, malformed entries, size limits.
The limits and the index file
Two hard limits, both worth knowing before you meet them:
- 50,000 URLs per sitemap file
- 50 MB uncompressed per file
Past either, you need a sitemap index: a small file listing several
sitemap files, each within the limits. This is what
/sitemap_index.xml is on most WordPress installations.
Splitting is also useful well below the limits, because it turns Search Console's coverage reporting into a diagnostic. Separate sitemaps for products, blog posts and category pages let you see indexing rates per type — and “92% of blog posts indexed, 31% of product pages” is a finding. One aggregate number is not.
About lastmod
Google has said it uses lastmod when it is consistently accurate, and ignores
it when it is not. A CMS that stamps today's date on every URL at every regeneration
teaches Google to disregard the field entirely. Either make it truthful or leave it out;
an inaccurate one is worse than an absent one.
About priority and changefreq
Google ignores both. They remain in the specification and in every generator's output, and they influence nothing. Setting every page to priority 1.0 is not a strategy; it is a no-op.
Finding, submitting and maintaining it
Finding yours
Try /sitemap.xml first, then /sitemap_index.xml —
WordPress and Yoast use the latter. If neither resolves, read
/robots.txt, which usually declares the location on a
Sitemap: line.
Submitting it
Two places, and do both. Declare it in robots.txt so any crawler finds it,
and submit it in Google Search Console, which additionally gives you per-sitemap indexing
reporting. Bing Webmaster Tools takes about a minute and is worth the minute.
Keeping it honest
The failures are mundane and silent. The sitemap 404s after a migration. It lists
http URLs on an https site, so every entry redirects. It was
generated once, two years ago, and points at pages that no longer exist. None of these
produce an error anywhere you would normally look, which is the argument for checking it
on a schedule rather than on suspicion.
Use these together
- Robots.txt Generator — Build a valid robots.txt with the right allow, disallow and sitemap directives — and understand each one.
- SEO Page Checker — Check any live page for title, description, headings, canonical, robots, alt text and internal links.
- Canonical Tag Generator — Generate a correct canonical link tag and learn when a page needs one.
- The full set of twelve — free, no signup, and for nine of them nothing you type leaves your browser.
Why check it rather than assume
Most people have never opened their own sitemap. It was generated by a plugin, it is presumed to work, and the presumption is usually right — until a migration, a domain change, a plugin update or an HTTPS switch quietly breaks it.
The sitemap checker fetches the file the way a crawler would and reports the HTTP status, the URL count, whether it is a sitemap index, the child sitemaps if so, protocol and domain consistency, where you stand against the 50,000-URL and 50 MB limits, and any malformed entries. One fetch, plain findings.
It has to fetch server-side, because a browser will not let a page on this domain read an XML file from yours — the same-origin policy blocks it and sitemaps essentially never send the CORS header that would permit it. Any tool claiming to analyse your sitemap purely in the browser is using a proxy or not reading the file. The fetch here applies the same protections as the growth score: public addresses only, redirects re-validated on every hop, a hard timeout and a size cap, and nothing retained.
What it deliberately does not do is fetch every URL inside your sitemap to check each one. Requesting thousands of pages is not something a free browser-based tool can do responsibly, and pretending otherwise would mean showing you numbers that were never measured. It validates the sitemap: structure, protocol, domain consistency and size. For per-URL status, the Pages report in Search Console is the right instrument, and it is free too.
FAQ
Does an XML sitemap improve rankings?
No. It helps crawlers discover URLs, which is a precondition for ranking rather than a contributor to it. A page in a sitemap that nobody links to and nothing else supports will be found faster and rank no better.
How many URLs can a sitemap contain?
50,000 URLs and 50 MB uncompressed per file. Beyond either limit you need a sitemap index pointing at several files. Splitting earlier than that is often worth it anyway, because per-type indexing rates in Search Console are far more useful than one aggregate figure.
Should I include noindex pages in my sitemap?
No. A sitemap entry says “this URL matters” and a noindex tag says “do not list this”. Sending both makes your site look inconsistent and wastes crawl attention. The same applies to blocked URLs and anything that redirects.
Do priority and changefreq do anything?
Google ignores both. They remain valid in the specification and every generator emits
them, but they influence nothing. lastmod is the one optional field
Google does use — and only when it is consistently accurate.
Where do I submit my sitemap?
Declare it in robots.txt with a full absolute URL, and submit it in
Google Search Console for the per-sitemap indexing reporting. Bing Webmaster Tools
takes a minute and is worth doing while you are there.
Thejas Acharya
Digital growth consultant based in Bangalore, working with founders and marketing teams across India and the US. I build websites, landing pages, Meta Ads campaigns and AI automation that are judged on one thing: whether they generate leads. The free tools on this site are the checks I run on client work, opened up.