Embedding Cache

An embedding cache stores the vector embeddings you've already computed so you don't pay to generate them again. Because turning text into an embedding costs an API call (money and latency) and the result is deterministic for a given input and model, caching by a hash of the text is an easy, high-leverage optimization. For SaaS builders running RAG or semantic search, this matters in two places. On the ingestion side, re-indexing a document set shouldn't re-embed chunks that haven't changed — cache by content hash and skip the unchanged ones. On the query side, popular or repeated searches can reuse a cached query embedding instead of hitting the model every time. Practical note: the cache key must include the model name and version, because embeddings from different models aren't comparable — swapping models silently poisons a cache keyed only on text. Set a sensible eviction policy (LRU or TTL) so it doesn't grow unbounded, and remember to invalidate when you re-chunk or re-embed. Done right, an embedding cache cuts both your model bill and your p95 latency.