Portfolio Blog

Technical Blog

Context Engineering

Context AI Agent

How can an AI coding agent work with a codebase larger than its context window?

You've probably heard about prompt engineering already. But how about context engineering?

Large language models (LLMs) have become remarkably capable at specific tasks: writing code, debugging software, reviewing pull requests, and analyzing documents. However, even a capable model needs the right information to do a task well.

Imagine asking Claude Code to fix a bug in an existing application. You describe the problem in a prompt. The coding agent then explores the repository, searches for relevant functions, reads source files, and runs tests before making changes.

The prompt is part of the context. So are the relevant source files, tool results, previous conversation messages, project instructions, and test failures the model receives. Together, these form the information available to the model at a given step.

That raises an interesting question: who decides what enters the context window, and when?

This is where context engineering comes in.

Context engineering is the practice of selecting, organizing, updating, and preserving the information an LLM receives so it can perform a task effectively.

Unlike prompt engineering, which primarily focuses on the instructions we write, context engineering considers the entire information flow surrounding an LLM or agent. The two overlap: a well-designed prompt is itself part of good context engineering.

1. Why context engineering matters

Every LLM has a context window: a limit on how many tokens it can process in a particular model call. Those tokens may include instructions, conversation history, retrieved documents, code, tool outputs, and space allocated for the model's response. Exact accounting depends on the model and API.

Let's use a hypothetical model with a 1-million-token context window and a codebase that would take 2 million tokens to represent as text. The model cannot ingest the whole codebase in a single request.

But even if the entire codebase did fit, sending all of it might not be a good idea:

  • Relevance: Much of the code may have nothing to do with the bug.

  • Cost and latency: Processing more tokens usually costs more and can take longer.

  • Noise: Unrelated files or old messages can distract the model from the immediate task.

  • Conflicts: Outdated instructions and contradictory tool outputs can create confusion.

  • Limited attention to detail: Long-context support does not guarantee that every detail will be used equally well.

The goal is not to maximize the amount of context. It is to maximize the usefulness of the context available at each step.

Think of the context window as working memory, not as an archive of everything the agent has ever seen.

2. The coding-agent example: retrieve information when needed

Suppose a user reports that login fails after refreshing an authentication token. Instead of reading every file in the repository, the agent can investigate incrementally.

A possible workflow:

  1. Read the task and repository-specific instructions.

  2. Use filename discovery (such as glob) to find authentication-related files.

  3. Use search (such as grep) to locate refreshToken, token validation, and relevant error messages.

  4. Use read to inspect a few specific code regions, with a line range or offset when supported.

  5. Run a failing test to obtain concrete evidence.

  6. Read additional code only if the results make it necessary.

  7. Implement a fix and verify it with tests.

flowchart TD A[User reports bug] --> B[Read task and project instructions] B --> C[Find files with glob / search] C --> D[Read relevant code ranges] D --> E[Run tests and inspect errors] E --> F{Enough information?} F -- No --> C F -- Yes --> G[Edit code] G --> H[Run verification]

The important point is that the codebase remains outside the model's context window until the agent selectively reads parts of it. Files on disk can be much larger than the model's current working context.

This is closely related to retrieval-augmented generation (RAG). Both approaches aim to bring useful external information into a model's context. A coding agent, however, can often actively choose among search, file-reading, execution, and other tools, refining its search based on intermediate findings. It may use lexical search, semantic retrieval, or both.

Tool outputs also need management. Printing a 30,000-line log into the conversation may technically work, but filtering it to the relevant stack trace and nearby errors is usually more useful.

3. Three context engineering techniques I find particularly useful

So far, we've seen how a coding agent can selectively retrieve the information it needs instead of reading an entire codebase. But retrieval is only part of the challenge. What happens when the conversation keeps growing, the task needs to be divided, or the agent has to remember progress across sessions?

There are three context engineering techniques I find especially useful for these situations:

  1. Compaction: Summarize accumulated context so the agent can continue working without carrying every previous message and tool output.
  2. Sub-agents: Delegate focused tasks to agents with separate context windows and return only the relevant findings to the main agent.
  3. Note-taking: Save important decisions, progress, and evidence outside the context window so the agent can retrieve them later.

These techniques solve different problems. Compaction reduces context, sub-agents isolate context, and note-taking preserves context. They can also work together in the same agent workflow.

Technique 1: Compaction — keeping long-running agents on track

A coding agent can accumulate a surprising amount of context during one task: every user message, search result, test output, and explanation consumes tokens if retained in the conversation sent to the model.

Compaction addresses this by replacing part of that history with a shorter representation of the important information. An orchestration layer might trigger it when estimated token usage crosses a configured budget, at a milestone, or through an explicit action. The specific mechanism depends on the agent system.

For example, imagine a hypothetical agent using a context budget of 100,000 tokens. After a long debugging session, the conversation approaches a chosen safety threshold. Rather than keep every intermediate tool result, the system could summarize the important state and continue from the summary.

flowchart TD A[Conversation and tool outputs accumulate] --> B{Near context budget?} B -- No --> C[Continue task] B -- Yes --> D[Summarize important state] D --> E[Preserve plan, decisions, changed files, bugs, tests] E --> F[Replace or trim older history] F --> G[Continue with compressed context] G --> A

What should a good summary preserve?

For software development, I would explicitly ask a summarization step to preserve:

  • The user's original goal and acceptance criteria.

  • The current plan and what has already been completed.

  • Key architecture and design decisions, including why they were made.

  • Files changed, relevant symbols, and important locations in the repository.

  • Bugs fixed, bugs still open, and hypotheses not yet verified.

  • Test commands, results, and known failures.

  • Constraints: APIs, compatibility requirements, security considerations, and things that must not change.

  • The next concrete action the agent should take.

For example, a compacted project-state file could look like this:


task: Fix intermittent refresh-token failure

status: implementation_in_progress

completed:

  - Reproduced the failure in auth integration tests

  - Located refresh logic in src/auth/token_service.py

  - Added regression test for expired refresh token

open_issues:

  - Check concurrent refresh behavior

  - Rerun the full authentication test suite

files_changed:

  - src/auth/token_service.py

  - tests/test_token_refresh.py

constraints:

  - Do not change the public authentication API

next_step: Run targeted tests and inspect remaining failure

This is only an illustrative structure, not a universal format. A real system should preserve source-of-truth artifacts (files, commits, logs, test reports) outside the summary, and link back to them where possible.

The trade-off: summarization can lose important details

Compaction is lossy. A detail that looked unimportant during summarization might become essential later. A summary may also inadvertently turn an unverified hypothesis into an apparent fact.

That is why it helps to separate confirmed facts, assumptions, failed approaches, and next actions. Where possible, persist durable state in files or a structured store and retrieve the original evidence again when needed.

The objective is not the smallest possible summary. It is the smallest summary that lets the agent continue correctly.

Technique 2: Sub-agents — isolating context for focused work

Another technique I find particularly interesting is sub-agent delegation.

Instead of having one agent read every file, run every test, implement every change, and perform every review within the same growing context, a main agent can delegate a focused task to another agent with its own context window.

For example, on a feature implementation:

  • A testing agent reproduces the bug and reports failing tests.

  • An implementation agent investigates and applies the code change.

  • A review agent inspects the patch for mistakes and edge cases.

  • The main agent coordinates the workflow and decides what to do next.

flowchart TD M[Main agent: plan and coordinate] M --> T[Testing agent\nIsolated context] M --> I[Implementation agent\nIsolated context] M --> R[Review agent\nIsolated context] T --> TS[Findings and evidence] I --> IS[Patch and change summary] R --> RS[Review findings] TS --> M IS --> M RS --> M M --> V[Integrate and verify]

Each agent can conduct a detailed investigation but return a much shorter result to the coordinator. That helps the main agent avoid carrying every subordinate's search history and tool outputs.

Parallel does not always mean better

Sub-agents are not automatically more accurate or efficient. Their value depends on how well the work can be divided.

They can introduce:

  • Extra cost: Several agents may repeat searches, read overlapping files, and use more tokens overall.

  • Coordination overhead: The main agent must reconcile findings that may disagree.

  • Merge conflicts: Multiple agents editing the same files can overwrite or interfere with each other's changes.

  • Lost information at handoff: A short report may omit an important caveat.

  • Dependency problems: Some tasks cannot safely run in parallel because one depends on another.

In many cases, it is better for a testing agent to investigate first, an implementation agent to act second, and a review agent to inspect the resulting patch third. This is sequential delegation, not parallel execution, but still benefits from context isolation.

I would use parallel sub-agents when tasks are genuinely independent, such as investigating different hypotheses or auditing separate modules. For code changes, clear file ownership, isolated branches or worktrees, and an integration step can reduce conflicts.

Technique 3: Note-taking — preserving context outside the conversation

Compaction helps with information already inside a conversation, but what about information the agent needs tomorrow, in a fresh session, or after its context is reset?

One solution is to write important state outside the model's current context. Examples include a task checklist, an architecture decision record, a repository instruction file, a structured state object, or an experiment-results table.

Crucially, saving information is not the same as keeping it permanently in the context window. The agent can retrieve the relevant parts later.

This is the difference between:

  • The context window: What the model sees in the present call.

  • External state or memory: What the system stores and may selectively supply in future calls.

A robust design also considers trust. Retrieved code comments, web pages, and tool outputs are data, not necessarily authoritative instructions. Agents should not blindly follow instructions embedded in untrusted material.

There is no single universal ordering. The goal is to ensure the agent has the right information at the moment it needs it, without forcing all information into every model call.

As LLMs get better at reasoning and tool use, the bottleneck is not always the model's raw capability. It is often the quality of the information available to it at each step.

Prompt engineering still matters. But for agents operating across large codebases, long conversations, and multi-step workflows, a clever prompt alone is rarely enough.

Context engineering is not about giving an LLM all the information. It is about giving it the right information, at the right time, in the right form — and preserving what matters as the work continues.

References and further reading