Documentation Index
Fetch the complete documentation index at: https://wiki.platelunchcollective.com/llms.txt
Use this file to discover all available pages before exploring further.
Technical implementation · Technical SEO
Definition
A web crawl is the automated process by which search engines and AI systems systematically browse the web to discover, fetch, and index web pages. Crawlers follow links from known pages to discover new pages, download content, and add it to their indexes.
Why It Matters for AI Search
Web crawls are how AI systems discover and index content. For brands publishing new content — glossary entries, service pages, blog posts — ensuring that content is linked from already-crawled pages and included in XML sitemaps accelerates crawl discovery. Understanding crawl behavior — which pages are crawled frequently versus infrequently — helps prioritize which pages receive the strongest internal link support for citation-critical content.
Relevant Plate Lunch Collective Services
AI SEO
