Delegation Requires Trust: On the four ways AI coding agents break that trust, and what it would take to restore it.

Martin Kleppmann opens the reliability chapter of Designing Data-Intensive Applications with a distinction that sounds obvious until you actually sit with it: the difference between a fault and a failure.

A fault is when one component deviates from its specification. A failure is when the whole system stops providing the required service.

I kept returning to this distinction while building software with AI coding agents over the past year. The agents were not exactly faulty. The code they produced was often correct, tests passed, pull requests opened cleanly.

But somewhere in the workflow, the system was failing me in ways that were quieter and harder to name than a broken build.

Microsoft Research put a name to part of it in April 2026. Their DELEGATE-52 paper studied what happens when you delegate repeated editing tasks to language models across 52 professional domains.

The finding was stark. Even frontier models, Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4, corrupt an average of 25% of document content over 20 delegated interactions. The errors are not gradual either. They are sparse and severe, what the paper calls silent corruption that compounds over long interaction.

The paper opens with a line I have not been able to stop thinking about: delegation requires trust.

That is not an observation about capability. It is a statement about system design.

Trust is not something you feel toward a model. It is something you engineer into the workflow around it.

The workflows most developers use today, Claude Code, Cursor, Codex, Devin, do not engineer it. They assume it.

Andrej Karpathy described the shift to Claude Tag as the third major redesign of LLM UI. A persistent, asynchronous entity with org-wide context working alongside teams.

He is right that this is a genuine paradigm change. But a warning circulating in the same conversation is equally worth sitting with. The moment your AI vendor becomes the place where work is interpreted, remembered, routed, and executed, that is not model lock-in. That is context lock-in.

Models can be swapped. The memory of how your codebase actually works is much harder to move.

Before getting into what a better workflow looks like, I want to describe the four specific ways the current paradigm breaks trust.

Problem One: The Agent Decided for You

You give Claude Code or Codex a task. The agent runs, reads files, reasons about the problem, chooses an approach, and writes code.

By the time you see the diff, the approach has already been chosen and the write has already happened.

You open the PR. The code is probably fine. But somewhere in that sequence, a decision was made. Which pattern to follow. Whether to add a new abstraction or extend an existing one. Whether to handle the edge case inline or extract it to a utility.

These are not trivial choices. In a real codebase, an architectural decision made in the wrong direction costs hours or days to unwind.

You are now in a different cognitive mode. You are auditing an outcome, looking for problems with a choice that has already been made.

The psychological research on this is consistent. Reviewing a completed action carries different cognitive load than participating in a pending decision. You are more likely to accept what is in front of you.

You can give Claude Code an MCP tool that posts a checkpoint before writing. But the write still happens inside the loop, after the approach is chosen. The review is still downstream of the decision.

Problem Two: Nobody Remembered the First Solution

I solved a state machine problem in one project. The agent and I worked through the approach together, settled on a pattern, and implemented it.

Six weeks later I was working on a different project. A similar problem came up and I described it to the agent.

The agent had no memory of the first project. It approached the problem fresh, chose a different pattern, and introduced a subtle bug the first solution had explicitly handled.

I did not catch it in review because I did not remember the connection either. The bug made it to production.

CLAUDE.md, AGENTS.md, and third-party tools like agentmemory all exist to address cross-session context. But memory is not a built-in property of the default workflow. It is an opt-in layer that requires deliberate setup separate from the actual coding work.

The cross-agent survey from vexp.dev in March 2026 confirmed the baseline: Claude Code has no long-term session memory across runs. Each new session requires re-exploration of the codebase.

Chip Huyen describes the analogous failure in ML systems as distribution shift. The world changes after training, and predictions degrade because they reflect a past state.

An agent working on your codebase without memory of past decisions is the same problem applied to software development.

A memory MCP server addresses this in theory. In practice, the agent queries it only if you prompt it to. It is a habit, not a guarantee. The fix needs to be structural: retrieval as a mandatory step before any approach is chosen.

Problem Three: The Reasoning Disappeared

Charles Petzold spends the first chapters of Code building from the simplest possible observation. The machine does exactly what you tell it, not what you mean.

An AI agent that produces a PR with no context is a precise instance of this problem.

You open the pull request. The code is correct, tests pass, the implementation matches the requirements.

But you have no idea why this approach was chosen, what alternatives were considered, or whether this is consistent with similar decisions elsewhere in the codebase.

So you merge it, because the tests pass and the diff looks fine.

Three months later, a new developer needs to modify that code. They cannot tell if the structure is intentional or arbitrary. They make a change that violates an assumption the original implementation depended on. Something breaks in a way that takes hours to debug.

Brian Christian calls this specification gaming in The Alignment Problem. The system achieves the stated objective while violating the intended one.

The knowledge of why the code is the way it is existed for exactly one moment. That moment passed without being captured.

A skill that appends reasoning to a PR description helps if the developer remembers to invoke it. The fix is to make reasoning a required output field, gate the implementation behind it.

Problem Four: Every Session Starts from Zero, By Default

Every time you start a new AI coding session, you spend the first ten to twenty messages re-establishing context.

The service uses this queue library, not that one. Postgres for this concern, MongoDB for that one. The auth middleware expects a specific header format. The job deduplication logic depends on a convention nobody wrote down.

A mid-2026 comparison of the four major agentic coding CLIs listed persistent memory as a shared gap across all of them. All four operate within one repository at a time. Working across a monorepo plus three sibling repos still requires you to be the coordinator.

A CLAUDE.md file captures conventions, not the history of decisions. It tells the agent how you work, not what you have already done.

The reasoning behind a specific architectural choice made three months ago lives nowhere retrievable, unless someone deliberately wrote it down.

Vendor-hosted memory tools, like Anthropic's memory layer, address the retrieval problem while deepening the context lock-in problem. The memory of how your codebase works now lives in their infrastructure, not yours.

Skills are composable additions to a flow that was never designed for review. Every one of these problems requires the fix to be structural, not optional.

The Third Paradigm and the Context Lock-in Problem

Andrej Karpathy described Claude Tag as the third major redesign of LLM UI. A persistent, asynchronous entity with org-wide tools and context, working alongside teams of humans.

He is right that this is a genuine paradigm shift. It is also worth reading alongside a concern that surfaced in the same conversation.

Once your AI vendor becomes a shared coworker, it stops being just a model provider. It starts becoming the place where work is interpreted, remembered, routed, and eventually executed. That is not model lock-in. That is context lock-in.

Models can be swapped. Claude today, something else next month. What is nearly impossible to move is the memory of how your organisation actually works.

Once that memory lives inside the vendor's infrastructure, switching costs grow every week you keep using it.

The answer is an architecture that lets you have the third paradigm without the trap of the second. Rent the intelligence, switchable per task, no lock-in. Own the context, in infrastructure you control.

The context should live in infrastructure you control, queryable independently of any AI vendor.

Your company memory should be inspectable, permissioned, portable, and model-neutral. It should not be buried inside the same vendor selling you the intelligence and the workflow surface.

The four problems described here are specific and addressable.

They are not going to be solved by better models. They are structural properties of how the current paradigm is designed.

Moving review upstream. Making past work retrievable. Capturing reasoning as a required output rather than a side effect. Treating project context as persistent infrastructure rather than something you reconstruct every session. None of this requires a new tool.

It requires treating the workflow itself as a design problem. Most people building with AI agents today are not doing that.