Context Engineering: Techniques for Building Reliable LLM Agents

Context engineering is the craft of filling the LLM's context window with just the right information for the next step. Here are the failure modes and the techniques that fix them.

Rahul Bisht

Founder, CrawlPilot

·
Jul 5, 2026
·AI & Agents·
9 min read
·
Context Engineering: Techniques for Building Reliable LLM Agents
Key takeaway
Context engineering is filling the model's window with just the right information for the next step, and keeping everything else out. Most agent bugs that aren't plain code errors are one of four failures (poisoning, overload, bloat, drift), and each has a matching lever.

Prompt engineering was about wording a single question well. Context engineering is the harder, more important discipline underneath it: deciding what goes into the model's limited context window at each step of a task — and, just as importantly, what stays out.

The one-line definition worth memorising: context engineering is the art and science of filling the context window with just the right information for the next step. Not everything you have. Not the whole conversation. The right slice, in the right order, at the right moment.

This post covers why it matters, the four ways it goes wrong, and the techniques that keep agents reliable as tasks get long.


Why the model can't just "have everything"

A context window is finite, and it is not free. Every token you put in costs money and adds latency, and — less obviously — more context does not mean better answers. Past a point, extra material actively hurts: the signal the model needs gets buried, and its attention spreads thin across things that don't matter for the current step.

So the job isn't to stuff the window. It's to curate it. That curation is where reliability is won or lost.


The four failure modes

Almost every agent bug that isn't a plain code error traces back to one of these:

  • Context poisoning — a wrong fact (a hallucinated value, a stale tool result) enters the context early and gets carried forward, contaminating every subsequent step.
  • Context overload — too much is packed in, and the relevant detail is lost in the noise. The model has the answer in front of it and still misses it.
  • Token bloat — the window fills with redundant history, verbose tool output, and repeated boilerplate. Cost and latency climb; quality doesn't.
  • Distraction / goal drift — deep into a long task, the original objective slips out of the active window and the agent wanders.

Name the failure and the fix usually names itself. Overload wants pruning. Bloat wants compaction. Poisoning wants validation and isolation. Drift wants the goal re-anchored.


The techniques

Think of context assembly as a pipeline. On each step you gather candidate material, select from it, order it, and hand the model exactly what it needs:

Here are the levers that pipeline gives you.

See how fast a window fills. Push the history or the tool count up and watch which lever the budget points you to:

200,000 tokens
2,000 tokens
40
300
8
800
120
900
Tokens used
128,400
Left for the answer
71,600
Of the window
64%
System prompt2,000 tokens
Tool definitions12,000 tokens
Retrieved docs6,400 tokens
Conversation history108,000 tokens

Tight. Biggest block is conversation history: compact the stale middle of the history into a summary.

Token counts are estimates you set; measure real prompts with your provider's token counter. The model's answer needs room in the window too.

What fills an agent's context window, and which lever to pull. Illustrative token counts; set your own.

1. Retrieval (RAG)

Don't pre-load a knowledge base into the prompt — fetch the few passages relevant to this step and inject only those. Retrieval keeps the fixed context small and pulls detail in on demand. It's the difference between handing the model a library and handing it the one paragraph it needs. (Vector search is the usual engine here — see our vector databases explainer.)

2. Memory — short-term and long-term

Not everything belongs in the live transcript. Split it:

  • Short-term / working memory lives in the current window — the last few turns, the active plan.
  • Long-term memory lives outside it — a file, a store, a scratchpad the agent reads from and writes to across sessions.

Give the agent a place to jot down what it learns and a habit of consulting it, and long tasks stop re-deriving the same facts. Store one lesson per note, and update rather than duplicate.

3. Dynamic tool loadout

Every tool definition you expose costs tokens and adds a decision the model has to make. Loading all your tools on every call bloats the window and dilutes tool selection. Instead, surface only the tools relevant to the current query — dynamic tool selection has been shown to sharply improve function-calling accuracy in benchmarks, because the model chooses from a focused set rather than a crowded one.

4. Compaction and summarisation

When history grows past what's useful, summarise the old turns into a compact form and drop the raw transcript. Compaction keeps the meaning of the conversation while shedding the token weight — the antidote to bloat on genuinely long-running agents. The trick is to compact the stale middle while preserving the goal and the most recent, most relevant turns verbatim.

5. Isolation with sub-agents

One giant context thread trying to hold every sub-task is a recipe for overload and drift. A better shape: a coordinator that spawns focused sub-agents, each with its own clean, isolated context for one piece of the work. Anthropic's research on multi-agent systems found this can outperform a single-agent setup precisely because contexts stay separated — no cross-contamination, no one window doing everything. (More on these shapes in agentic design patterns.)

6. Deliberate ordering

Finally, order matters. Put stable, foundational context first (system instructions, task definition) and volatile, step-specific material last. This isn't just tidiness — it's what makes prompt caching work, and it keeps the model's attention on what changed rather than re-reading a shifting preamble.


Check that the failure-to-fix mapping stuck:

Check yourself

Q1Deep into a long task, your agent starts working on something that wasn't the goal. Which failure is it, and what's the fix?

Q2A hallucinated value from an early tool call keeps showing up in later answers. What's going on?

Q3You expose 80 tools on every call and the model keeps picking the wrong one. Which lever helps most?

A rule of thumb

Start simple and add structure only when a task demands it. Most agents don't need a memory system, sub-agent orchestration, and dynamic retrieval on day one — they need a clean, well-ordered prompt and one good retrieval step. Add each technique when a specific failure mode shows up: reach for compaction when you hit bloat, isolation when one context is doing too much, memory when the agent keeps forgetting.

Context engineering isn't a framework you install. It's a discipline you apply — deciding, at every step, what the model needs to see and nothing more.

Cheat sheet
Context engineering

The core idea

  • Fill the window with just the right information for the next step, and keep everything else out.
  • Every token costs money and latency, and past a point extra context buries the signal.

Four failure modes

  • Poisoning: a wrong fact enters early and is carried forward. Fix with validation and isolation.
  • Overload: the relevant detail is lost in the noise. Fix with pruning.
  • Token bloat: redundant history and verbose tool output. Fix with compaction.
  • Goal drift: the objective slips out of the active window. Fix by re-anchoring the goal.

Retrieval and memory

  • Fetch only the passages relevant to this step instead of pre-loading a knowledge base.
  • Short-term memory lives in the window; long-term memory lives in a file or store outside it.
  • Store one lesson per note, and update rather than duplicate.

Tools, compaction, isolation

  • Expose only the tools relevant to the current query, not every tool on every call.
  • Summarise the stale middle of the history; keep the goal and the most recent turns verbatim.
  • Hand sub-tasks to sub-agents, each with its own clean, isolated context.

Ordering

  • Put stable context first (system instructions, task definition) and volatile, step-specific material last.
  • Stable-first ordering is what makes prompt caching work.

Rule of thumb

  • Start with a clean, well-ordered prompt and one good retrieval step.
  • Add compaction for bloat, isolation when one context does too much, memory when the agent forgets.
The failure modes and the levers that fix them, on one page.

If you're building agents on top of web data, the quality of what you feed them starts with the quality of what you extract. CrawlPilot turns messy pages into clean, structured rows — the kind of well-formed context an agent can actually use. The getting started guide shows the extraction flow end to end, and agentic design patterns covers the loops this context feeds.