> ## Documentation Index
> Fetch the complete documentation index at: https://wiki.platelunchcollective.com/llms.txt
> Use this file to discover all available pages before exploring further.

# TF-IDF

> TF-IDF is a weighting scheme that scores a term by how often it appears in a document against how rare it is across the whole collection.

*Core concept* · *AI Search Infrastructure*

## Definition

TF-IDF, term frequency-inverse document frequency, is a weighting scheme that scores a term by how often it appears in a document against how rare it is across the whole collection. It gives high weight to words that are frequent in one document and uncommon elsewhere, marking them as distinctive. It is the idea underneath most keyword retrieval, including BM25.

## Why It Matters for AI Search

TF-IDF formalizes an intuition that matters for content: the terms that identify you are the ones you use and others do not. Generic language scores low because everyone uses it. Specific, distinctive terms score high, and they are what a lexical system keys on. Writing with the precise vocabulary of a niche is what makes a page legible to these methods, and it is why vague, interchangeable copy is invisible to the keyword half of retrieval.

## Related Terms

<CardGroup cols={2}>
  <Card title="BM25" href="/ai-search-glossary/bm25">See also</Card>
  <Card title="Sparse retrieval" href="/ai-search-glossary/sparse-retrieval">See also</Card>
  <Card title="Keyword search" href="/ai-search-glossary/keyword-search">See also</Card>
  <Card title="Semantic search" href="/ai-search-glossary/semantic-search">See also</Card>
  <Card title="Hybrid retrieval" href="/ai-search-glossary/hybrid-retrieval">See also</Card>
</CardGroup>

## Relevant Plate Lunch Collective Services

[AI SEO](https://www.platelunchcollective.com/services/ai-seo)  [Citation-Ready Content](https://www.platelunchcollective.com/services/citation-ready-content)