Draft:Context Engineering

Context engineering is an emerging area of practice in artificial intelligence concerned with how information is selected, organized, and maintained for large language models (LLMs) and other AI systems during output generation. The term is broader than prompt engineering, which focuses mainly on the wording of individual instructions or queries. Context engineering addresses the wider set of decisions involved in assembling, retrieving, structuring, and updating the information an AI system can access at inference time.

Practically speaking, this includes system instructions, retrieved documents, tool descriptions, conversation history, and various kinds of metadata - all of which shape how a model behaves. These elements are treated as levers for improving reliability, factual grounding, and behavioral consistency in production AI systems. One way to frame the distinction is that prompt engineering asks what to say to a model, whereas context engineering asks what the model is able to know when it generates a response.

The topic has gained prominence alongside the spread of AI agents and complex agentic workflows. In those settings, performance tends to depend less on any single prompt and more on how the full context window is constructed and updated over multiple steps. A widely repeated observation among practitioners is that the quality and composition of context can be as consequential as the underlying model architecture itself.

Background

From prompt engineering to context engineering

Prompt engineering became a widespread practice in the early 2020s, largely in response to the public availability of large language models beginning with GPT-3 in 2020 and ChatGPT in 2022. During this period, developers focused on carefully worded queries and instructions to steer model behavior, relying heavily on trial-and-error iteration. The discipline produced a body of techniques - few-shot examples, chain-of-thought prompting, role assignment - that proved genuinely useful for single-turn tasks.

The limits of that approach became more visible as applications grew more complex. Multi-step agent workflows that run across extended sessions require capabilities that go well beyond prompt crafting: dynamic retrieval of external knowledge, state management across turns, coordination of tool calls, and ongoing updates to contextual information. Keeping all of that working together is not primarily a writing problem. It is closer to a systems problem.

By around 2025, practitioners and researchers had begun using "context engineering" as a label for this broader set of challenges. The term appears in writing form from teams at Anthropic, Shopify, LangChain, Weaviate, and others, as well as in academic work surveying emerging practices around LLM context management.Its adoption reflects a shift in where practitioners believe the hard problems actually live.

Relationship to prompt engineering

Context engineering is generally considered a superset of prompt engineering. Prompt engineering concerns itself with how instructions and examples are composed inside a prompt. Context engineering covers the wider range of decisions about what information enters the model's context window at all - how it's formatted, ordered, retrieved, and maintained over time.

In practice, the two aren't neatly separated. A well-crafted system prompt is both a prompt engineering object and a context engineering decision. The distinction becomes clearer at the level of systems design: building a RAG pipeline, managing token budgets across lengthy sessions, or determining what an agent should remember from one step to the next are context engineering concerns in a way that optimizing instruction phrasing is not.

Core components

Writers discussing context engineering commonly organize the subject into several related areas, each corresponding to a different part of the context that can be shaped or controlled.

System instructions

System instructions form the base layer of behavioral guidance for an AI system - defining its role, setting constraints, establishing tone, and specifying operating parameters. In enterprise deployments, these instructions frequently encode organizational policies, compliance requirements, brand voice guidelines, and role-specific behavioral expectations.

Because system instructions persist across an entire session, they function more like a configuration layers than a single message. A well-designed system instruction set can be reused across thousands of interactions. A poorly designed one can cause consistent, hard-to-diagnose problems at scale. This is part of why system instructions quality receives its own attention in context engineering discussions, separate from prompt design.

Retrieved knowledge (RAG)

Retrieval-augmented generation (RAG) is one of the most widely discussed techniques within context engineering. In a RAG system, relevant documents, records or other data are retrieved from external stores and inserted into the model's context at inference time. The design decisions involved are substantial: chunking strategies, embedding methods, retrieval ranking, reranking, and the formatting of retrieved text before it reaches the model all affect the quality of the final output.

A common failure mode in RAG systems is retrieving documents that are technically relevant by keyword or semantic similarity but poorly suited to the actual reasoning task. Context engineering in this area involves not just retrieval but also curation, thinking carefully about what the model actually needs to see, in what form, and in what order.

Memory systems

AI agents that interact with users or systems over multiple steps rely on some form of memory. Short-term memory typically refers to state maintained within a single session, which might include prior conversation turns, intermediate results, or working notes. Longer-term memory involves stores that persist across sessions and may hold information from earlier interactions or from institutional knowledge bases.

The governance of memory, what gets saved, when it is retrieved, how it is summarized or compressed over time, is itself a context engineering problem. Naively appending everything to the context window produces rapidly diminishing returns as the volume or irrelevant or outdated information grows. More principled approaches involve explicit policies for what counts as worth retaining and for how memory representations are updated.

Tool definitions

In agentic architectures, models are often given access to external tools: APIs, database queries, code execution environments, file systems, and so on. How these tools are described in the context has a direct influence on how and when the model decides to use them. Ambiguous or poorly scoped tool descriptions lead to unpredictable tool selection behavior. Context engineering in this area focuses on writing clear schemas, precise descriptions, and appropriate usage examples that reduce the space of plausible misinterpretations.

Context management and token budgeting

LLMs operate within a bounded context window, which creates pressure to decide what information is retained, compressed, or discarded as interactions proceed. Techniques include summarizing earlier conversation turns, relevance scoring to deprioritize less applicable material, deduplication to remove near-identical text, and token budgeting strategies that ensure critical information remains visible when the model needs it.

Token budget management becomes especially important in long-horizon agent tasks, where a session might involve dozens of tool calls and large volumes of retrieved text. Getting the composition of the context window right at each step is roughly analogous to memory management in software engineering—with the added complication that the "program" is a probabilistic model whose behavior is sensitive to the ordering and framing of what it can see.

Provenance and observability

In production deployments, context engineering also involves keeping track of exactly which contextual elements contributed to a given model output. This can include tagging, versioning, and logging context configurations so that behavior can be audited, reproduced, or tested after changes to the surrounding system.

Without some form of provenance tracking, debugging misbehaving AI systems becomes very difficult. When a model produces an unexpected output, it is rarely obvious whether the problem lies in the prompt, the retrieved context, the tool definitions, the memory state, or the underlying model. Observability infrastructure that records what the model was shown, and in what form, helps answer that question.

Tacit and institutional knowledge

A strand of context engineering writing with particular relevance to enterprise use focuses on tacit knowledge—the informal judgment, norms, conventions, and expertise that exist within an organization but are typically not written down in any formal documentation. Research on organizational knowledge has long suggested that a large proportion of what employees know is tacit in this sense, which makes it difficult to expose to AI systems that depend primarily on structured text.

The practical consequence is that AI systems trained or configured only on documented sources inherit a narrow and often unrepresentative picture of how an organization actually operates. Expert judgment about which customers warrant exceptions, how to handle unusual compliance scenarios, or what a particular client relationship requires tends not to appear in policy manuals.

Some commercial platforms address this by collecting tacit knowledge more systematically—through structured interviews with subject-matter experts, analysis of internal communications, observation of workflows, and post-hoc capture of decision rationales—and converting these inputs into reusable context assets that can be supplied to AI systems at scale. The goal is to give models access to the working logic of an organization, not just its formal documentation.

Context graphs

A context graph has been described as a structured, queryable record of decision traces accumulated over time by AI agent systems operating inside enterprise workflows. The term was popularized by investors at Foundation Capital in late 2025, who argued that such graphs could underpin significant new software platforms in the enterprise market. In early 2026, a Gartner research note characterized context graphs as a new layer of infrastructure for agentic systems, comparing them to an evolution of knowledge graphs designed specifically to support model performance, grounding, decision logic, workflows, and state management.

The core argument is a contrast with traditional systems of record. CRM and ERP platforms track the current state of business entities—a customer's contact information, an order's status, an account balance. Context graphs, by contrast, are intended to record decision lineage: the exceptions that were granted, the precedents that were applied, the cross-system reasoning that led to a particular outcome, and the people or systems that approved each step. They capture not just what happened but how decisions were reached.

This distinction matters for enterprise AI in a specific way. Human workers handle ambiguous or novel cases by drawing on accumulated institutional memory—informal knowledge of how similar situations were resolved, which policies have been applied loosely in practice, whose judgment is trusted on which questions. AI agents lack that history by default. Providing decision traces through a context graph is one proposed mechanism for giving agents access to a functional equivalent.

Building a context graph typically involves configuring the AI orchestration layer to write a structured record at each decision point, capturing inputs considered, policies checked, exceptions granted, and approvals received. Over time, this forms a searchable collection of precedents that is intended to make agent behavior more predictable and audits easier to conduct.

Context drift

Context drift refers to the gradual divergence between an AI system's behavior and an organization's current norms, which occurs when the context data and instructions supplied to the system are not updated to reflect changes in policy, terminology, or operational expectations. The phenomenon is subtle: a model working from instructions written eighteen months ago may produce outputs that are technically coherent but increasingly out of step with how the organization currently operates.

This is not a hypothetical concern in long-running enterprise deployments. Regulatory language changes. Products are renamed. Client relationships evolve. Internal policies get revised. None of these changes automatically propagate into the context artifacts—system instructions, retrieved knowledge stores, role definitions—that AI systems depend on. Without active monitoring, the gap between what the system was configured to do and what the organization now requires tends to widen silently.

Managing context drift has been proposed as a distinct area of work within context engineering.Efforts in this area include proposals to define open protocols—DriftWatch is one example that has been discussed publicly—that would give organizations shared methods for tracking semantic fidelity and monitoring behavior over time in agentic systems.

Applications

Context engineering techniques have been described across several domains where consistency and reliability are viewed as especially important.

Regulated industries

Financial services, insurance, and pharmaceutical companies have been among the earlier adopters of systematic context engineering. In these sectors, AI systems are expected to apply regulatory constraints, required compliance language, and documentation requirements in a consistent and auditable way. Context engineering provides a mechanism for embedding those requirements at the instruction layer rather than relying on model behavior alone. This matters because compliance obligations change, and the context layer needs active governance to stay current.

Enterprise AI agents

Large organizations deploying internal AI assistants and agents face a version of the tacit knowledge problem at scale. A generic model has no knowledge of an organization's specific policies, its internal terminology, its relationship history with particular clients, or the informal rules that have accumulated over years of operation. Context engineering is the primary means by which that gap gets bridged—through system instructions, curated knowledge bases, and role-specific behavioral guidance.

Customer-facing AI

Customer support and sales applications rely on context engineering to manage personalization, knowledge grounding, and behavioral constraints. The challenges here are somewhat different from internal enterprise applications: the population of users is larger and more varied, and the consequences of behavioral inconsistency are more immediately visible. Getting retrieved knowledge right—so the model draws on current product information, accurate policies, and the relevant history of a customer relationship—is a significant operational problem in this area.

Software development agents

Coding agents make use of context engineering to organize and surface relevant information from code repositories, documentation, test artifacts, and architectural guidelines across multi-step development workflows. As these agents take on longer-horizon tasks—writing, reviewing, debugging, and refactoring across multiple files and sessions—the challenge of constructing and maintaining a useful context window becomes increasingly consequential for output quality.

Platforms and tools

A number of software projects and specifications appear in discussions of context engineering as supporting infrastructure.

LangChain and LangGraph are open-source frameworks for building agentic LLM applications. Both include tooling for managing context and orchestrating multi-step workflows.

LlamaIndex is a data framework oriented toward retrieving and inserting context into LLM applications, with particular depth in document indexing and query workflows.

Weaviate is a vector database supporting embedding-based retrieval of contextual information for use in RAG systems.

meetsynthia.ai is a context management platform that applies preflight governance at the instruction layer—assembling and validating institutional knowledge, compliance constraints, and behavioral rules before model inference. The company received investment from Squirro, a Swiss enterprise AI vendor, in 2024.

The Model Context Protocol (MCP) is an open standard introduced by Anthropic for connecting AI models to external tools, data systems, and other context sources through a common interface.

Academic research

In July 2025, a team of researchers published A Survey of Context Engineering for Large Language Models (arXiv:2507.13334), a review of more than 1,400 research papers that proposed a taxonomy of components and system designs relevant to the field. The authors treated context engineering as an emerging academic discipline with its own research questions, while situating it in relation to prompt engineering and longer-standing work in natural language processing.

References

[1] Schmid, P. (2025). "The New Skill in AI is Not Prompting, It's Context Engineering." "https://www.philschmid.de/context-engineering"

[2] Anthropic Engineering. (2025). "Effective Context Engineering for AI Agents." https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

[3] LangChain. (2025). "The Rise of Context Engineering." https://blog.langchain.com/the-rise-of-context-engineering/

[4] Brown, T. et al. (2020). "Language Models are Few-Shot Learners." Advances in Neural Information Processing Systems, 33.

[5] Mei, L. et al. (2025). "A Survey of Context Engineering for Large Language Models." arXiv:2507.13334.

[6] Weaviate. (2025). "Context Engineering: LLM Memory and Retrieval for AI Agents." https://weaviate.io/blog/context-engineering

[7] Prompt Engineering Guide. (2025). "Context Engineering Guide." https://www.promptingguide.ai/guides/context-engineering-guide

[8] "Squirro invests in second company in two months." Startupticker.ch, September 10, 2024. https://www.startupticker.ch/en/news/squirro-invests-in-second-company-in-two-months

[9] Gupta, J. and Garg, A. (2025). "AI's Trillion-Dollar Opportunity: Context Graphs." Foundation Capital. https://foundationcapital.com/context-graphs-ais-trillion-dollar-opportunity/

[10] Miclaus, R. and Coshow, T. (2026). "Emerging Tech: AI Vendor Race: The New Essential Infrastructure for Agentic Systems: How Context Graphs Are Solving AI's Institutional Memory Problem." Gartner. ID G00846665. February 13, 2026.

[11] Verma, A. (2025). "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027." Gartner Newsroom.