# Request caching

**Source:** https://promtable.com/glossary/request-caching

> Request caching is the LLM-gateway technique of storing exact-match (or semantic-match) prompt → response pairs to skip the model call on cache hits — cuts cost + latency on common queries.

---
Request caching is the LLM-gateway technique of storing exact-match (or semantic-match) prompt → response pairs to skip the model call on cache hits — cuts cost + latency on common queries.

Two cache flavors: exact-match (hash the request, return identical responses for identical inputs — safe + simple) and semantic match (vector-embedding similarity above a threshold reuses prior responses — more hits but quality risk). Common in: customer support FAQ (the same question gets cached answer), classification pipelines (repeated inputs get cached labels), system-prompt prefix caching (the prompt prefix is cached server-side by the provider — Anthropic [[prompt-caching]], OpenAI). Trade-offs: exact-match misses minor wording variations; semantic match needs careful threshold tuning. Production patterns: TTL caches per use case, manual invalidation hooks, cache-miss observability to detect drift. Most LLM gateways (Portkey, LiteLLM, Cloudflare AI Gateway, Helicone) ship caching out of the box.

## When to use

- High-volume repeated queries (FAQ, classification).
- Customer-facing chat with common first messages.

## Common mistakes

- Caching across users without isolation — privacy leak between tenants.
- Semantic match threshold too low — returns wrong answers to similar-looking different questions.

## Related terms

- [semantic-cache](https://promtable.com/glossary/semantic-cache)
- [prompt-caching](https://promtable.com/glossary/prompt-caching)
- [llm-gateway](https://promtable.com/glossary/llm-gateway)

*Last updated: 2026-06-01*
---

Original page: https://promtable.com/glossary/request-caching
Maintained by Promtable (https://promtable.com). Content: CC BY 4.0. Cite as "Promtable — https://promtable.com/glossary/request-caching".
Contact: info@vibecodingturkey.com.