Claude Sonnet 5.5 and GPT-6 Astra both take a million tokens of input. So why build a retrieval pipeline at all? When stuffing the context wins, when retrieval still wins, and the hybrid most production systems end up with.
A million-token context window (Claude Sonnet 5.5 on Bedrock has one, and GPT-6 Astra accepts up to a million input tokens) holds several thousand pages. For many corpora that is everything: a product's documentation, a contract set, a codebase. The case for retrieval-augmented generation used to be that the model could not see enough; for small and medium corpora, that case is gone.
Book a free 30-minute call. You'll leave knowing what we'd do first and roughly what it costs, whether or not you hire us.
The corpus fits and changes rarely, so a cached prompt carries it cheaply across many questions.
Questions need reasoning across the whole set — "where do these contracts disagree" — which retrieval of a few chunks does badly.
You need something working this week, without an ingestion pipeline, chunking strategy and vector store to maintain.
The corpus is larger than any window, or grows every day.
Different users may see different documents. Retrieval with access filters enforces that; a shared giant prompt cannot.
Cost and latency at volume: sending a million tokens per question, even cached, is slower and dearer than sending the twenty relevant passages.
You need citations a reviewer can check against a specific passage.
Retrieve generously — fifty or a hundred passages rather than the five that small windows forced — filtered by what the user may see, then let a long-context model reason over all of them. Retrieval becomes a coarse, permission-aware filter rather than a precision instrument, which makes it far more forgiving, and the model does the precise part. S3 Vectors made the retrieval half cheap; caching makes the long-context half affordable.
Once a month: what changed in tech that month and what it means for you, from cloud and AI to security and the tools you build with. Every item links to its source. No sales pitch, unsubscribe in one click.
Book a free 30-minute call about it. Bring your setup and your bill; you'll leave knowing what we'd do first and roughly what it costs, whether or not you hire us.
Or keep reading
AI Search Over Specs, RFIs and Drawings for Engineering Firms: What Works and What to Check First
Engineering firms sit on decades of specifications, RFIs, submittals, calculations and drawings that nobody can search well. Retrieval with a model on Bedrock makes them answerable, with citations. What it is good at, the licensing and client-data questions to settle first, and why the engineer of record stays responsible.
Read
LLM Cost Engineering in 2026: Caching, Batch, Service Tiers, Effort and Routing
Most LLM bills can fall by half without touching quality. The five levers on Amazon Bedrock in 2026: prompt caching (now with a 1-hour TTL), 50%-off batch inference, Flex and Priority tiers, reasoning effort, and routing each task to the smallest model that passes your evals.
Read
As of October 25, 2026. Provided as is, for general information, and accurate to the best of our knowledge on the date shown. Not legal, compliance, tax, financial or other professional advice. Prices, limits and vendor services change; check current vendor documentation before relying on anything here. Terms of use.