Prompt caching is the highest-leverage LLM cost cut available right now

Every major model provider now supports prompt caching, and the savings are large enough that it's worth checking whether your prompts are actually structured to take advantage of it. Anthropic's prefix caching cuts input token costs by up to 90%, with cache reads priced at roughly $0.30 per million tokens versus $3.00 per million for a fresh read. OpenAI's automatic caching is enabled by default and delivers up to 50% savings once a prompt crosses 1,024 tokens, with no code changes required at all.

Why most teams under-use it

Caching works by reusing the processed representation of a prompt's static prefix — system instructions, tool definitions, retrieved context that doesn't change turn to turn — instead of reprocessing it on every call. The catch is that anything appended before that static content breaks the cache. Teams that put per-request variables (timestamps, user IDs, dynamic context) at the front of the prompt, ahead of the stable instructions, often get near-zero cache hit rates without realizing it.

Reordering a prompt so static content comes first and variable content comes last is usually the entire fix. One reported case, from the security tooling company ProjectDiscovery, raised its cache hit rate from 7% to 84% this way, cutting total LLM spend by 59–70%. At enterprise scale, that kind of improvement on Anthropic's pricing can mean roughly $7,200 a month — about $86,400 a year — from a configuration change rather than a model downgrade.

The practical takeaway: before reaching for a cheaper model to cut costs, check whether your existing prompts are cache-friendly. It's very often the larger and easier win.

Source: AI Cost Check, TECHSY