Redis launches LangCache semantic caching service, says it can cut LLM API costs by up to 90%
Redis has introduced LangCache, a managed semantic caching service now in public preview on Redis Cloud. Instead of checking whether prompts are identical at the text level, the service uses semantic matching to identify similar requests. If a match is found, the cached response is returned and the application can skip a fresh large language model call. Redis said this can reduce API costs by as much as 90% and improve response speed by up to 15x on cache hits. LangCache works in a two-call loop. Before sending a prompt to an LLM, the application first sends it to LangCache for vector similarity search. If a semantically similar cached entry exists, LangCache returns that response directly. If not, the application proceeds with the LLM call, then stores the prompt and response pair for future reuse. The service includes built-in embedding model generation and also supports user-supplied models. Redis said LangCache is available through a REST API and comes with Python and JavaScript SDKs. Users can configure similarity thresholds, TTL, and eviction policies, while cache hit rates and cost savings can be tracked in the Redis Cloud console. Customer Mangoes.ai reported a 70% cache hit rate in its patient care application.

