Semantic Retrieval

Level 3 of Retrieval Maturity Levels: retrieve by meaning rather than exact keyword. Documents are chunked, each chunk is run through an embeddings model that places it in a high-dimensional space where nearness ≈ semantic similarity, and a query is matched against similar chunks (X is similar to X, Y, Z) instead of literal string matches (X = X). A "smart lookup" for feedback then surfaces notes about evaluations and test results, not just the ones containing the word. Tools: Qdrant, Pinecone, Supabase; you can go deep on chunking, embedding, hybrid search, and re-ranking.

The load-bearing corrective is that vector search is not magic. Because it retrieves chunks, it is excellent when you need a pinpoint answer over a lot of text — "what was rule 17 of these 1,000 rules?" pulls the right snippet without reading the whole file — but it fails questions that need the whole document. "Which week had the highest sales?" grabs one plausible chunk and confidently returns a local max while higher values sit in chunks it never pulled; "summarize the March 5th meeting" summarizes five retrieved chunks, not the meeting. When you need full context, don't chunk — have the agent read the whole markdown file. So semantic retrieval is a per-folder choice under Query-Shaped Storage, not a universal upgrade: use it for lookup-over-scale, not for aggregation or completeness.

Claims

Semantic search retrieves by meaning, not keyword — X is similar to Y instead of X = X. observation — a factual property of embedding-based retrieval; it finds related content the exact-match search misses.
Vector search is not magic — chunk-level retrieval misses questions that need the whole document. principle — durable: it's a structural limit, not a tuning problem; aggregation and completeness questions are outside what similarity-over-chunks can guarantee.
Use vectors for pinpoint lookup over lots of text; use a whole markdown file when you need full context. best practice — context: the retrieval-style choice per folder; "best" flips on whether the question is a needle-lookup (vectors win: cheap on tokens) or a whole-document need (a plain file wins: accurate).
Metadata and hybrid/re-ranking can improve results but don't remove the chunk-context limit. observation — honest boundary: tuning helps at the margin, it doesn't make chunk retrieval see the whole document.
Semantic search is unreliable on literal strings and exact identifiers (error codes, function names like useState). principle — durable: retrieval-by-meaning does not privilege surface tokens, so exact-match queries can miss; this failure shape is why Hybrid Retrieval pairs it with keyword search.
Documents are chunked before embedding, and the chunk carries its source line range. observation — factual mechanics (OpenClaw: ~400-token chunks, 80-token overlap); the line-range metadata is what enables citation and targeted fetch (Search-Then-Get).
At scale, vector search uses an approximate nearest-neighbor (ANN) index — trading a little accuracy for speed, with a negligible practical difference. observation — factual: exact one-at-a-time distance works for small sets; ANN is what makes it fast over millions.

Retrieval Maturity Levels — level 3 on the ladder.
Knowledge Graph Retrieval — level 4; typed relationships where semantic similarity isn't enough.
Hybrid Retrieval — the practical pairing that patches semantic's literal-string weakness.
Rank Fusion — how a semantic result list is merged with a keyword one.
Reranking — the model-scored pass that can refine semantic (+keyword) results.
Incremental Indexing — how the vector store stays current without re-embedding everything.
Query-Shaped Storage — semantic retrieval is chosen (or rejected) by the query shape.
AI Second Brain — one retrieval style a folder in the brain can use.
Distillate: Every Level of a Claude Second Brain Explained
Distillate: How AI Agents Search Their Memory — Hybrid Retrieval, in Practice (OpenClaw)

Semantic Retrieval

Claims

Related

Linked from