The problem
Two teams build the same AI feature on the same model. One gets 93% accuracy. One gets 71% and cannot work out why.
The difference is almost never the model, and it is usually not the prompt either. It is what the model was looking at when it answered.
I see a lot of companies with a capable model, a carefully written prompt, and output they cannot trust. After 4 years building production AI systems, this is the layer I spend most of my time on, and it is the one most teams do not know they are missing.
The number that makes the case
Shopify took an internal AI tool from 71% to 93% accuracy by changing how context was assembled. Rewriting the prompt alone moved it from 71% to 74%.
Twenty two points versus three. Same model, same task.
If you are spending your optimization time on the prompt, you are working on the three point lever.
What context engineering actually is
It is deciding, for every call your system makes, what information the model sees, in what order, in what form, and how fresh it is.
Prompt engineering is a subset of this. The prompt is one slice of the token budget. Context engineering owns the entire budget and how it changes over time.
It is a systems design discipline, not a writing one. The questions are about data pipelines, retrieval, state and freshness, not phrasing.
The five decisions
This is the framework I use on every system I build.
1. Relevance: what changes the answer
For every candidate piece of information, I ask one question. If this were absent, would the correct output change?
If no, it does not go in. Not "it might help." Not "we have it anyway." Out.
This is the hardest discipline to hold, because including things feels safe. It is not safe. Every irrelevant token competes for attention with the relevant ones and pushes real signal toward the middle of the window, where recall drops 20 to 30%.
2. Freshness: how old is too old
Every piece of retrieved data needs a known age and an explicit policy.
Pricing from an hour ago is fine. Pricing from last quarter will generate a confident, wrong, possibly binding answer to a customer.
I put timestamps into the context itself and tell the model what to do when data is older than a threshold. Usually: say so and escalate, rather than answer.
Stale context is the most common cause of confident hallucination I see in the wild, and it is entirely preventable.
3. Form: how it is shaped
The same information helps or hurts depending on its shape.
A 3,000 token CRM record dump is worse than eight labeled fields pulled from it. Raw HTML is worse than extracted text. A transcript is worse than a structured summary of what was established in it.
Structure the data before it enters the window. The model should spend its capacity on the decision, not on parsing your payload.
4. Position: where it sits
Given the position effect, order is a real design decision.
Task instruction and the most decision relevant facts go at the edges. Bulk reference material goes in the middle, because the middle is where you can afford degradation.
5. Budget: what it costs
Every call gets a token ceiling that I set deliberately. When the content exceeds it, something is cut according to a priority I defined in advance.
The alternative is that a truncation function makes that choice at runtime, and it will cut whatever happens to be at the end, which is frequently the thing that mattered most.
Memory is a context problem
For anything that runs longer than a single call, the question becomes what carries forward.
Not the transcript. A structured state object: what has been established, what has been ruled out, what is still unresolved, what the user actually wants.
Rebuild that object on a schedule and drop the raw history. A few hundred tokens of good state beats ten thousand tokens of accumulated conversation, and it does not decay the way raw history does.
Agents fail differently from chatbots. Their failures are state management failures, not prompt failures, and this is why.
How to tell if this is your problem
Some quick diagnostics.
Why this is the job now
Models got good enough that the bottleneck moved. The constraint on most production AI systems is no longer reasoning capability. It is the quality of the information environment the reasoning happens inside.
That environment does not build itself. Someone has to decide what gets retrieved, how it is shaped, how fresh it must be, what carries forward and what gets dropped. That work is invisible when it is done well, which is exactly why it gets skipped.
It is also the difference between a demo and a system.
This is the layer I own on every build, and it is where I would start on any AI system that is technically running and still cannot be trusted.