Technology
In October 2026 the published rate for frontier model input sits near two dollars per million tokens, and a cached read of those same tokens sits near twenty cents. Verify both before you budget anything, because they move monthly. The ratio is the durable part, and it has held for two years: reading something you already sent costs roughly a tenth of sending it fresh.
Most teams treat the context window as a capacity. It is a meter.
A context window gets described as a size. Two hundred thousand tokens, a million tokens, as though it were a suitcase and the only question is whether your thing fits. That framing is why the bills surprise people. The window is not a suitcase. It is a turnstile, and you pay at it on every pass.
Here is the part nobody internalizes until the invoice arrives. The model has no memory of your conversation. Every turn re-sends the entire history. Turn one costs you one unit. Turn ten costs you ten. Ten turns of a growing thread do not cost ten times turn one, they cost fifty five times, because you are paying again for the same early tokens on every single pass. Agent loops do this to themselves dozens of times before anyone looks at the shape of the curve.
So the first move is not a cheaper model. It is deciding what gets re-sent.
A context window is a budget with four line items: the instructions, the reference material, the conversation so far, and the thing you actually want answered. Only the last one is new. The other three are rent, and most systems pay full price on all of it forever because nobody ever separated them.
Solomon got there three thousand years before the first price sheet: the eye is not satisfied with seeing. Give a system more window and it will fill the window. Capacity does not discipline itself.
What the price sheet is actually telling you
Published pricing has stopped being a single number, and that is the signal worth reading. Some providers now charge the standard rate across the full million token window. Others apply a surcharge past a threshold, two hundred thousand tokens on one, two hundred and seventy two thousand on another, where input roughly doubles the moment you cross it. Treat all of these as published rates and check them yourself before you commit.
A threshold is not a slope. It is a cliff. A system averaging a hundred and ninety thousand tokens of context looks fine on a spreadsheet and detonates on the days it runs long. Design for the ninety fifth percentile prompt, not the median one.
The other number on every sheet is output, and it runs about five times input. Verbosity is the expensive end of the transaction, not the reading. A prompt that asks the model to think out loud and then discards the thinking is the most expensive habit in production AI, and it is usually there because somebody copied it from a demo.
Cache discounts reward stability, not cleverness. Nothing else in this business gives you ninety percent off for being boring.
Design the prefix
Cache reads are cheap because the provider only has to recognize that it has seen this exact opening before. Exact is the operative word. Byte identical, same order, from the first token forward. Put a timestamp at the top of your prompt and you have invalidated everything behind it, then paid full price on all of it while wondering why the discount never arrived.
Which gives you an architecture instead of a guess. Static first: instructions, schemas, the reference documents that do not change this week. Volatile last: the question, today’s numbers, whatever is different about this call. Append, never insert. The order of a prompt is a cost decision wearing the costume of a formatting decision.
And anything that does not need an answer in the next second does not belong on the interactive endpoint at all. Batch tiers run near half price on the same models. A nightly summarizer, a backfill, a classification sweep across an archive: those are not conversations. Paying conversation rates for them is a billing accident, not a strategy.
The constraint was never how much the model can read. It is how much you can afford to send it twice.
Cheap context changes what you are willing to build. When a retrieval pass costs a tenth of what it used to, you stop rationing it, and you start asking better questions of bigger piles of material. That is the compounding part, and it has nothing to do with which model is on top this month.
Every model named on today’s price sheet will be deprecated. The structure underneath it will not. Input is cheaper than output. Repetition is cheaper than novelty. Thresholds are cliffs, not slopes. Stable prefixes are discounts. Those four survived two years of price cuts and they will survive the next two.
Design the prompt the way you would design a bill. The cheapest token is the one you never had to send.
