Definition
TF-IDF, term frequency-inverse document frequency, is a weighting scheme that scores a term by how often it appears in a document against how rare it is across the whole collection. It gives high weight to words that are frequent in one document and uncommon elsewhere, marking them as distinctive. It is the idea underneath most keyword retrieval, including BM25.Why It Matters for AI Search
TF-IDF formalizes an intuition that matters for content: the terms that identify you are the ones you use and others do not. Generic language scores low because everyone uses it. Specific, distinctive terms score high, and they are what a lexical system keys on. Writing with the precise vocabulary of a niche is what makes a page legible to these methods, and it is why vague, interchangeable copy is invisible to the keyword half of retrieval.Related Terms
BM25
See also
Sparse retrieval
See also
Keyword search
See also
Semantic search
See also
Hybrid retrieval
See also