XML sitemaps that stay true

Spencer Thursfield · Published 2026-09 · Last reviewed 2026-09-12 · Evidence: platform documentation

An XML sitemap is a file that lists the pages on your site you want search engines to find. Each entry is a small promise: this address is live, it is the version you want indexed, and it last changed on this date.

Keep the promise. A sitemap that lists redirects, deleted pages or dates that change every day teaches crawlers to stop believing it.

What is an XML sitemap?

Google defines a sitemap as “a file where you provide information about the pages, videos, and other files on your site, and the relationships between them”. The format is set out at sitemaps.org: an XML file with one <url> entry per page, holding the full address in <loc> and, optionally, the date of last modification in <lastmod>.

Two entries from this site’s own sitemap, fetched on 2026-09-12:

<url>
  <loc>https://www.content-is-everything.com/cookies/</loc>
  <lastmod>2026-08-07</lastmod>
  <priority>0.8</priority>
</url>
<url>
  <loc>https://www.content-is-everything.com/structured-data/</loc>
  <lastmod>2026-09-12</lastmod>
  <priority>0.8</priority>
</url>

The dates differ because the pages changed on different days, which is exactly what the field is for. One file may hold at most 50,000 URLs or 50MB uncompressed. A larger site splits its list and publishes a sitemap index that points to each part.

What does a sitemap do, and what does it not do?

• It helps discovery. In Google’s words, “A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” Official recommendation
• It is optional for many small sites. Google says a well-linked site of “about 500 pages or fewer” might not need one.
• It is not a ranking lever. Google’s sitemap documentation describes crawling, not position, and says Google ignores <priority> and <changefreq>. The 0.8 above carries no weight with it.
• It is a weak canonical signal. Google counts sitemap inclusion as “a weak signal that helps the URLs that are included in a sitemap become canonical”.

What should be in a sitemap?

Only pages you would be glad to see in a search result. Each entry should be:

• Live. The address returns status 200, with no redirect on the way.
• Canonical. The one address you want indexed, matching the page’s own canonical tag. Google’s instruction is to “pick a canonical URL for each of your pages and submit them in a sitemap”. The choice itself is covered in canonical URLs.
• Indexable. No noindex on the page.
• Absolute, on the right host. Google asks for “fully-qualified, absolute URLs”, on HTTPS rather than HTTP.

What should not be in a sitemap?

Anything a crawler would fetch and then throw away. Required

• Redirected addresses. Google’s page indexing report says “Redirect URLs are not indexed, only the redirect target.” List the destination, not the old address.
• Pages marked noindex. The sitemap asks for indexing, the page refuses, and the page wins.
• Non-canonical duplicates. Filtered and sorted variants, tracking parameters, print versions. Google’s guidance is explicit: “don’t specify one URL in a sitemap, but a different URL for that same page using rel="canonical"”.
• Deleted pages. A URL that now returns 404 should leave the sitemap on the day the page goes.

Why lastmod dates must be honest

Google uses <lastmod> “if it’s consistently and verifiably (for example by comparing to the last modification of the page) accurate”, and says it should reflect “the last significant update to the page”. A change to the main content, the structured data or the links counts. A new copyright year does not.

The common failure is a generator that stamps every entry with the date the file was built. Bing studied this and found the most common lastmod error was identical dates, usually “set to the date of generation of the sitemap, rather than the date of content modification”. Its warning: if dates are “consistently set to the current date, we will suspect the validity of those dates and may disregard them”. Google’s version is blunter. Claim a page changed yesterday when it changed seven years ago, and “eventually we’re not going to believe you anymore”.

If you do not know a page’s real date, leave the field out for that page. Google says that is fine.

How do you submit a sitemap to Google?

1. In Search Console. Open the Sitemaps report, paste the sitemap’s address into “Add a new sitemap” and click Submit. You need owner permission on the property. The report then gives a status for the last fetch: Success, Couldn’t fetch, or a count of errors.
2. In robots.txt. Add one line anywhere in the file, such as Sitemap: https://www.example.com/sitemap.xml. Google picks it up the next time it crawls robots.txt. This site does exactly that; see robots.txt in practice.

Do both: the Sitemaps report shows only sitemaps submitted through it or the API, so robots.txt alone gives you no status to check. The old anonymous “ping” address is gone: Google deprecated it in June 2023, and requests to it now return a 404.

What about Bing and IndexNow?

Bing reads the same format and, in its post on lastmod, recommends IndexNow for telling engines the moment something changes. Your site sends a list of changed URLs, up to 10,000 per request, and proves ownership with a key file on its own host. Participating engines share each submission with one another. The IndexNow FAQ says to use it alongside a sitemap, which stays “a complete inventory of all URLs on your website”.

How do you check that a sitemap stays true?

A sitemap drifts whenever a page is deleted or redirected and nobody regenerates the file. Check it after any restructure:

• Read the status. The Sitemaps report in Search Console should say Success.
• Sample the entries. Each should return 200 directly, with no redirect, no noindex and a canonical tag pointing at itself. The reachability checks are in can AI reach your site at all?
• Look at the dates. If every lastmod matches the day the file was generated, the dates are telling crawlers nothing.
• Read the page indexing report. It gives Google’s reason for each URL it did not index, including redirects, noindex and pages not found.

llms.txt is a separate file with a separate purpose, and nothing on this page depends on it. Google says its search does not use one; the evidence is in the folklore ledger.

Common questions

Does a small website need a sitemap?

Not strictly. Google says a site of about 500 pages or fewer that is well linked internally may not need one. It is still worth having: it costs little and is the one place to give every page an honest last-modified date.

How do I submit a sitemap to Google?

In Search Console, open the Sitemaps report, paste your sitemap’s address and click Submit. Also add a Sitemap line to robots.txt so every crawler can find it.

Does a sitemap guarantee my pages get indexed?

No. Google says a sitemap helps it discover URLs but does not guarantee they will be crawled or indexed. Each page still has to be reachable, canonical and worth indexing.

What date should lastmod show?

The date of the page’s last significant change to its main text, structured data or links. Never the date the sitemap was generated. If you do not know the real date, leave the field out.