When I write C++, I send my teammates a pull request containing C++. I don't compile it into assembly and ask them to review the assembly.
The C++ code expresses implementation decisions in a language developers understand. The compiler translates those decisions into executable instructions, and we use tests and other verification tools to establish confidence in the result.
I've been thinking about how this compares to the way we're building software with AI agents. I write a prompt describing what I want, an agent interprets that prompt and generates an implementation, and another developer reviews the generated code to determine whether the agent did what I asked.
As agents become more capable, I expect the gap between code generation capacity and human review capacity to grow. I suspect this mismatch is contributing to some of the friction around agentic software development.
That raises a question: if agents are increasingly responsible for implementation, what should humans spend their time reviewing?
The Cost of Understanding Generated Code
Consider a relatively simple change: adding rate limiting to a login endpoint. I could ask an agent to limit failed login attempts to five per account within a 15-minute window. The agent might modify the authentication service, add Redis counters, introduce configuration, and write tests.
That short prompt could produce a pull request spanning several files and hundreds of lines of code. A reviewer now has to understand how the rate limit works, whether concurrent requests can bypass it, and what happens when Redis becomes unavailable.
Some of these details might have been specified in the original prompt. Others were decisions the agent made during implementation, leaving the reviewer to reconstruct those decisions from the code. The person requesting the change knows what they wanted, and the agent had context about its implementation choices, but much of that information may be missing from the pull request.
Code review has always involved understanding someone else's implementation. Agentic development changes the economics of that process because generating another implementation is becoming increasingly inexpensive while human attention remains relatively expensive.
I wonder how much review effort we're spending reconstructing information that the agent could provide directly.
Let the Agent Explain Its Work
We can introduce an intermediate step using the workflows we already have. After generating code, the agent produces a structured walkthrough describing the behavior it implemented, the decisions it made, and the evidence supporting its claims.
The walkthrough describes the completed implementation in a format reviewers can compare against the original requirements. It provides context for understanding the code while keeping the implementation and its verification evidence available for inspection.
For the rate-limiting example, the walkthrough might look something like this.
Implemented Behavior
- Each account may accumulate up to five failed login attempts within a rolling 15-minute window.
- After the fifth failure, additional login attempts return HTTP 429 until the restriction expires.
- Counters are shared across application instances.
- Concurrent failed attempts count toward the same limit.
Implementation Decisions
- Redis stores failed-attempt counters.
- Atomic operations handle concurrent requests.
- The existing authentication API remains unchanged apart from the new rate-limit response.
- Rate-limit settings use the application's existing configuration system.
Verification Evidence
| Behavior | Verification |
|---|---|
| Five failed attempts are permitted | Integration test |
| The next attempt is rejected | Integration test |
| Restrictions expire after 15 minutes | Time-controlled test |
| Concurrent failures respect the limit | Concurrency test |
| Existing authentication behavior is preserved for unrestricted requests | Regression tests |
Open Questions
- Should limits apply to accounts, IP addresses, or both?
- What should happen when Redis is unavailable?
- Should successful authentication reset the failure counter?
- Could an attacker use this behavior to lock out another account?
This is an illustrative example, but it captures the information I'd want when reviewing an agent-generated change. The reviewer starts with a description of the behavior and the decisions behind it, then follows references into the implementation where additional context is needed.
I especially like the idea of surfacing open questions. Agents regularly encounter decisions that weren't covered in the original request, and those decisions can have important consequences for security, architecture, or product behavior. Making them explicit gives reviewers an opportunity to evaluate the choices the agent made.
The walkthrough also needs to be verifiable. Each behavioral claim should reference relevant source locations, and test results should come from actual executions against the commit being reviewed. An agent can produce a convincing explanation of incorrect code, so the supporting evidence is essential to establishing confidence in the walkthrough.
That evidence also needs to cover the claims being made. A passing integration test establishes something about the scenarios it exercises, but it cannot establish that the implementation handles every possible input or failure condition.
A structured walkthrough could reduce the effort required to understand generated changes. It also gives us a starting point for developing a common language to describe software behavior.
From Implementation Walkthroughs to Specifications
Most agent prompts are written in natural language, and their structure varies considerably. One developer might provide detailed acceptance criteria, while another writes a short request and allows the agent to work out the implementation details.
We could begin developing a common language by standardizing how agents describe completed implementations. Every generated pull request could include the same categories of information: implemented behavior, constraints, architectural decisions, verification evidence, and unresolved questions.
As teams use these walkthroughs, they'll discover which information helps reviewers understand changes and which details are consistently missing. The format could evolve through actual development work, giving us a structure grounded in how engineers review software.
Once that structure becomes useful and reliable, we can start moving it earlier in the development process. The agent could generate a specification from the developer's initial request, allowing a human to review the requirements, clarify ambiguities, and approve the intended behavior before implementation begins.
The agent would then implement the approved specification and produce evidence showing how the resulting code satisfies it.
In effect, the output specification becomes the input specification.
I can imagine this evolving through three stages, each building on what we learn from the previous one.
Stage 1: Code-First, Explanation-Assisted Review
The first stage looks much like today's development workflow. A developer writes a prompt, an agent generates code, and the resulting pull request includes a structured implementation walkthrough.
Reviewers use the walkthrough to understand the behavior and architectural decisions, then inspect relevant code and verification evidence. The implementation remains the primary artifact, with the walkthrough providing additional context for reviewing it.
This stage gives us a way to measure whether structured explanations make code review more efficient. It also helps establish which information reviewers need and whether agents can describe their implementations accurately.
The distinction between a walkthrough and a specification matters here. A walkthrough generated from completed code describes what the implementation appears to do. It doesn't establish whether that behavior matches the original intent.
Stage 2: Specification-Assisted Development
Once the walkthrough format becomes useful and consistent, we can introduce it before implementation. The agent drafts a specification from the developer's request, and a human reviews the requirements, constraints, and acceptance criteria.
After the specification is approved, the agent generates the implementation and produces a walkthrough connecting the resulting behavior to the original requirements. Reviewers can compare the approved specification against the implementation walkthrough and supporting evidence.
This introduces a clear contract between human intent and agent execution. It also provides an opportunity to identify misunderstandings before the agent spends time implementing them.
At this stage, code review remains a central part of the workflow, but reviewers have a clearer description of the intended behavior against which to evaluate the implementation.
Stage 3: Specification-First Development
The longer-term possibility is a workflow where the specification becomes the primary artifact humans review. Developers approve the intended behavior and relevant constraints, while agents generate implementations and provide verification evidence showing how those implementations satisfy the specification.
Code inspection would remain available, particularly for security-sensitive changes, architectural decisions, and areas where verification is incomplete. As confidence in the process improves, more routine review effort could move toward evaluating requirements and verification evidence.
This would change the role of a pull request. The central discussion could focus on the behavior we're trying to create, the constraints that matter, and the evidence that the implementation satisfies them.
The challenge is establishing enough confidence in the specification and verification process to support that shift.
Can Agents Verify Their Own Work?
An agent can misunderstand a requirement, implement that misunderstanding, write tests for it, and produce a walkthrough explaining why everything works correctly. The resulting artifacts might all agree with one another while the implementation still fails to satisfy the original intent.
This is particularly relevant during the first stage, when the walkthrough is generated from the code. A convincing explanation of an implementation can make it easier for reviewers to accept behavior that was never intended.
Several techniques could help establish confidence in the relationship between specifications and implementations.
| Technique | Purpose |
|---|---|
| Requirement traceability | Connect each acceptance criterion to implementation and verification evidence |
| Independent test execution | Run tests against the actual commit in a trusted environment such as CI |
| Independent implementation verification | Have a separate agent or process evaluate the implementation against the approved specification |
| Human review | Evaluate ambiguous requirements, architectural tradeoffs, and risks that automated checks might miss |
Each technique addresses a different part of the problem. Requirement traceability can reveal missing coverage, while independent test execution establishes that the reported results came from actual test runs.
A separate verification agent might identify mistakes missed during implementation, but independence depends on more than using a different agent. If both agents rely on the same interpretation of an ambiguous requirement, they can produce consistent but incorrect results. An approved specification and independently designed acceptance tests could provide a stronger basis for verification.
Even an approved specification can contain mistakes. Human review helps establish that the requirements reflect what we intend, but the quality of that review still matters.
For some changes, formal verification or property-based testing might provide stronger guarantees. For others, integration tests, static analysis, and targeted code review may be sufficient. The appropriate level of verification would depend on the risk associated with the change.
This is also where the compiler analogy has limitations. A compiler translates a well-defined source language according to relatively stable rules. AI agents interpret natural language, make design decisions, and can generate different implementations from the same instructions.
That flexibility is part of what makes agents useful, but it makes the relationship between intent and implementation harder to establish. Moving review toward specifications requires evidence that agents can implement those specifications reliably and that our verification processes can identify meaningful deviations.
How I'd Test This
The first experiment I'd like to try is relatively small. I'd take a collection of agent-generated pull requests and have an agent produce a structured implementation walkthrough for each one.
Every walkthrough would include the implemented behavior, architectural decisions, verification evidence, and unresolved questions. Behavioral claims would link to relevant source locations, and verification results would reference tests executed against the actual commit.
I'd then compare two review approaches. One group of reviewers would receive the normal code diff and test results, while another would receive those same materials along with the generated walkthrough.
I'd want to measure several things:
- How long does each review take?
- How accurately can reviewers explain the resulting behavior?
- How many defects or missing requirements do they identify?
- How often do reviewers need to inspect the implementation?
- How often does the walkthrough misrepresent the implementation?
I'd use comparable pull requests and rotate reviewers between both approaches to reduce the influence of reviewer experience and task complexity. Ideally, each reviewer would evaluate multiple changes using both methods, with the order varied to limit learning effects.
The outcome I'm most interested in is whether walkthroughs reduce review effort while preserving defect detection. Faster reviews would be valuable, but only if reviewers maintain their ability to identify problems in the actual implementation.
I'd also introduce known defects into a controlled set of pull requests. These defects would be selected independently of walkthrough generation and would include cases the walkthrough might overlook or describe incorrectly.
That would let me measure whether the walkthrough helps reviewers identify problems or creates false confidence. I'd want to distinguish between defects that reviewers find through the walkthrough and defects they discover only after inspecting the code.
Another comparison worth testing is who generates the walkthrough. The implementation agent could describe its own work, or a separate agent could receive the original request and completed diff and produce an independent explanation.
The second approach might reduce the influence of the implementation agent's original assumptions, although both agents could still share the same misunderstanding.
These experiments would help establish whether implementation walkthroughs improve review efficiency while maintaining or improving defect detection. If the results are promising, the next experiment could move the specification ahead of implementation and compare the resulting workflow against the code-first approach.
When the Specification Becomes the Source of Truth
Specification-first development also raises questions about how we maintain software over time.
Today, source code is usually the most authoritative description of how a system works. Documentation and tests help explain or verify that behavior, while the implementation ultimately determines what happens when the software runs.
A specification-first workflow introduces another possible source of authority: a versioned description of intended behavior that exists independently of any particular implementation.
If that specification is sufficiently complete, we could use it to generate different implementations, evaluate competing approaches, or update dependencies while preserving the original behavior. Verification would establish whether each implementation satisfies the same requirements, within the limits of the checks being performed.
The challenge is keeping the specification aligned with the system as it evolves. If an agent fixes a production issue, the specification may need to change. If the code and specification disagree, the workflow needs a clear process for resolving the difference.
I'd expect that process to depend on why the disagreement exists. An implementation defect should generally be corrected to satisfy the approved requirements. An intentional behavior change should update the specification through review. A specification that turns out to be incomplete or incorrect needs its own revision.
Architectural decisions also complicate this model. Two implementations might satisfy the same behavioral requirements while having very different security, performance, and maintainability characteristics. A useful specification would need to capture relevant constraints alongside expected behavior.
These questions become increasingly important as specifications take on more authority. They're also reasons to approach the transition incrementally, learning from actual implementations and review practices before treating specifications as the primary source of truth.
A Different Kind of Pull Request
I keep coming back to the original C++ analogy. We review C++ because it's the language developers use to express implementation decisions. The compiler handles translation, and our verification processes help establish confidence in the resulting program.
As agents take responsibility for more implementation decisions, I think it's worth reconsidering which artifacts deserve the most human attention.
The first question I want to answer is whether an agent-generated implementation walkthrough helps reviewers identify problems more efficiently. I'd start by comparing reviews with and without walkthroughs, paying particular attention to whether the explanations introduce false confidence.
If the results are promising, I'd move the specification earlier in the workflow and repeat the experiment. That would give us a way to evaluate specification-first development through actual review outcomes.
The interesting question is how much human review effort can move toward specifications and verification evidence while preserving our ability to catch mistakes. That's what I want to measure before changing the review boundary.