LLM Cost Engineering in 2026: Caching, Batch, Service Tiers, Effort and Routing

Most LLM bills can fall by half without touching quality. The five levers on Amazon Bedrock in 2026: prompt caching (now with a 1-hour TTL), 50%-off batch inference, Flex and Priority tiers, reasoning effort, and routing each task to the smallest model that passes your evals.

Measure before you cut

Start with where tokens go, per feature: input versus output, how much of each prompt repeats, and which requests need an answer in seconds versus by tomorrow. Bedrock's CloudWatch metrics break usage down by model and, since service tiers, by tier; an LLM observability tool gives the per-feature view. Almost every team finds one feature using most of the budget.

1. Prompt caching

Long system prompts, tool definitions and documents that repeat across requests can be cached, and cached reads are billed at a fraction of the normal input price. On current Claude models on Bedrock a cache checkpoint needs at least 512 tokens on Sonnet 5.5 (1,024 on older models), up to four checkpoints per request, in the system prompt, messages and tools — and there is now a 1-hour TTL alongside the 5-minute default. Put stable content first and variable content last; a timestamp at the top of a system prompt silently defeats the whole cache.

2. Batch inference

Anything that does not need an answer now — classification of yesterday's tickets, embeddings, evaluations, summaries for a report — can run as a Bedrock batch job at 50% of the on-demand price for supported models. Check the model card: batch support varies by model and Region.

3. Service tiers

Bedrock now has four tiers. Standard is the default. Flex trades longer processing times for a discount — good for agentic background work and evaluations. Priority costs a premium and is served ahead of the others, for customer-facing paths under load. Reserved buys dedicated tokens-per-minute for a month or three, with overflow to Standard. Availability varies by model: at the time of writing, Claude Sonnet 5.5's card lists Standard only, so check before you plan around a tier.

4. Reasoning effort

Models with adaptive reasoning let you set how hard they think. Sonnet 5.5 defaults to "high". For extraction, classification and short rewrites, a lower effort is usually just as accurate and far cheaper in output tokens; keep high and above for planning, debugging and multi-step reasoning. Set it per use case, not globally.

5. Routing

The biggest saving is not using a frontier model for work a small one does as well. With an evaluation set per task, try the smallest model first — Haiku-class, Nova, a small open-weight model — and move up only when it fails. A router that sends each request to the cheapest model that passes for that kind of request often cuts spend by more than every other lever combined.