What is XML Sitemap?
A machine-readable list of canonical URLs worth crawling, optionally with last-modified dates. One file holds 50,000 URLs or 50 MB uncompressed; larger sites segment by template or locale.
Related terms
Robots.txt
A plain-text file at the domain root telling compliant crawlers which paths they may fetch. It controls crawling, not indexing: blocked URLs can still be indexed by URL if linked.
HTML Sitemap
A human-readable page listing important site sections for users and crawlers. Secondary to XML sitemaps and navigation, but useful on large or deep sites as a discovery safety net.
Sitemap Generator
A tool emitting the XML sitemap from a site’s real URL inventory. Generators must read canonical targets and indexation state, not just crawl output, or they advertise URLs engines should ignore.
Technical SEO
The crawling, indexing, rendering, and performance work that lets search engines reach, understand, and serve your pages. It covers directives, architecture, speed, and structured data rather than words on the page.
Crawling
How search-engine bots discover pages by following links, sitemaps, and directives. If a URL is never crawled, it cannot be indexed no matter how good its content is.
Crawl Budget
The number of pages a search engine will crawl on a site in a given period, driven by server capacity and demand signals. Large or slow sites must spend it on indexable, valuable URLs.