Views: 1
Ask a retrieval-augmented generation (RAG) system a question containing an ambiguous word, and every so often, it confidently retrieves the wrong kind of document — a finance policy when you asked about riverbank erosion, a sports term when you meant something musical. The instinct is to blame the retrieval logic, the chunking strategy, or the prompt. Often, the actual cause sits one layer deeper: the embedding model itself, and a limitation baked into how many embeddings are built in the first place.
One word, one vector — even when it shouldn’t be
Classic word embedding techniques — word2vec, GloVe, fastText — all share a foundational assumption: every distinct word gets exactly one vector, computed once, used everywhere. For a word like “bank,” that vector has no choice but to be a mathematical compromise between every sense the word has ever been used in across the training data — part financial institution, part riverbank, blended into a single point in vector space that doesn’t cleanly represent either meaning. Tested directly: a raw word-level embedding for “bank” often shows only weak, roughly similar association with both “money” and “river” — genuinely ambiguous, exactly as you’d expect from an average taken across every use of the word.
The moment that same word gets used in an actual sentence — “I deposited money at the bank” — the correct meaning is obvious to any human reader instantly. A static embedding has no mechanism to use that surrounding context at all. It returns the same blurry, compromised vector whether the sentence is about finance or geography.
Why this matters specifically for retrieval
In a vector-search or RAG system, this isn’t an academic quibble — it directly determines which documents get retrieved. If a query embedding and a document embedding are both built on this same one-vector-per-word assumption, ambiguous terms in either the query or the corpus can quietly pull in irrelevant results, or fail to distinguish between genuinely different intents, with no error message anywhere in the pipeline — the system just returns a plausible-looking wrong answer.
The actual fix: embeddings that use context, not just the word
The structural solution is contextual embeddings — representations computed fresh for each specific sentence, using the words actually surrounding the ambiguous term, rather than one fixed vector looked up from a table. Even a simple version of this — blending a word’s own embedding with its immediate neighbors’ embeddings for that specific occurrence — measurably sharpens which sense gets represented: the same word can land clearly closer to “finance”-related vectors in one sentence and clearly closer to “geography”-related vectors in another, purely from mixing in the actual local context. This is the underlying idea behind the attention mechanism inside every modern transformer-based embedding model (the kind actually used by most production RAG systems today) — and it’s precisely why using a genuinely contextual embedding model, rather than an older static one, is one of the highest-leverage fixes available for retrieval quality issues involving ambiguous language.
The practical takeaway
If a RAG or semantic search system is producing occasional confidently-wrong retrievals on queries involving ambiguous or multi-sense terms, it’s worth checking whether the embedding model in use actually contextualizes per-occurrence, or whether it’s still working — even indirectly — off a one-vector-per-word assumption underneath. The failure mode is subtle precisely because it never throws an error; it just quietly returns the wrong kind of “right answer.”
Learn these concepts in depth in our dedicated blog posts series on the topic “From Zero to Agents”.
Leave a Reply