The short answer

Let the coding agent write the code and run the checks, but give the verdict to a verifier it can't edit. TestSprite is that verifier: a separate testing agent that tests your running app against an approved plan, and only changes what a test expects when a person approves. Your agent gets fast, precise feedback; your team gets a green check it can trust.

A separate author

Tests come from your words, PRDs and tickets, not from the code being checked.

Locked expectations

The coding agent can run tests and read failures; changing what a test expects takes a person's approval.

Honest verdicts

Every failure is a real bug, a test problem or an environment issue, with a replay and expected versus actual.

The pattern, and why it happens

An agent changes some code, a test goes red, and a few turns later it is green again. The diff shows one of several familiar moves:

  • The assertion was loosened: an exact total became "is a number", a specific error became "any error".

  • The test was skipped or marked as expected to fail.

  • The failing call was mocked, so the test no longer touches the code that broke.

  • New tests were written by reading the new code, so they confirm what it does rather than what it should do.

This is a coding agent doing what it does well: reaching its goal. If the task is "finish the feature and make the tests pass", editing the test is a short path to done. The fix is structural. When the author can also edit the checks, a green result mainly tells you the task finished; when the checks sit outside the author's reach, green tells you the product works.

Why the agent that wrote the code shouldn't grade it

Engineering and finance settled this long ago for people: the engineer who writes a change doesn't approve the pull request, and the person who books a payment doesn't sign it off. Authors share the blind spots of their own work. Applying the same separation to coding agents is what lets you merge their output at full speed.

Lauren Tan's talk "All Code Is Slop (Except Mine)" at the AI Engineer Code Summit shows where the control belongs. Her "trust pyramid" puts the most weight on constraints in the code itself (architecture, lint rules, types), then on runtime and formal verification, and only a little on written rules for agents. A line in an instructions file saying "never modify tests" is helpful guidance; the dependable control lives in the verification layer, where the agent being checked has no say.

Four properties of an independent verifier

  1. A separate author. The checks come from a different agent or system than the one that wrote the code, working from its own record of what the product should do.

  2. Expectations outside the coding agent's reach. The coding agent can run the checks and read the results; changes to expectations need a person's approval.

  3. The running app. Tests drive the real frontend in a browser and real backend requests, so they observe behaviour instead of restating the implementation.

  4. Verdicts that separate bugs from noise. A classified failure (real bug, test problem or environment) sends each problem to the right fix, so a product bug is never "fixed" by editing a test.

Who may change what: a permissions table

Separation becomes practical as a short table of permissions. This one works for most teams running coding agents:

RoleCanCan't
Coding agentWrite and change code; run the suite; read failures; propose a new test or an expectation changeChange what an existing test expects; approve its own proposals
VerifierRun the full suite on every change; update a test's steps when the interface changes; classify failuresChange what a test expects without approval
People (feature owner, QA, reviewer)Approve the test plan; approve or reject expectation changes; decide on overridesMerge past a failing gate without a recorded reason

The line that matters runs between steps and expectations. Updating a selector because a button was renamed is maintenance. Changing "the total includes tax" to "the total is shown" is a product decision, and product decisions belong to people.

What a well-handled expectation change records

Products evolve, so expectations must be able to change. The goal is to make every change visible and owned. A good expectation change records:

  • the old expectation and the new one, side by side;

  • the reason, ideally the ticket or decision behind it;

  • who approved it, and when;

  • the run that proves the new behaviour.

With that record, a reviewer sees every expectation change up front instead of piecing it together from a test that turned green. It belongs in the evidence every agent pull request brings, covered in the guide to reviewing agent pull requests.

Guardrails you can add today

  • Require a code owner on test directories, so test edits get a human reviewer.

  • Flag pull requests that change code and its assertions together, and read the assertion changes first.

  • Lint against skipped and focused tests on the main branch.

  • Keep end-to-end expectations where the coding agent can't write, and run them as a required check.

The first three rely on reviewers noticing. The fourth is the durable one, and it is what TestSprite provides out of the box.

How TestSprite puts this into practice

TestSprite is a separate testing agent with its own record of what your product should do. You describe it in your own words or point it at your PRDs and tickets (the feature map guide shows how to write them); it drafts a test plan in which every test states its expectation in plain language, and you approve it. That approved plan is what every later change is checked against.

It tests the running app, frontend and backend. When the interface changes it updates the steps, and what a test expects only changes when you approve it, so a failing check can't be quietly rewritten to pass. Every failure comes back classified, with a replay, the failing step, and expected versus actual, so anyone can confirm a verdict in a minute.

Your coding agent stays in charge of the code. It calls TestSprite from the terminal through the open-source CLI or through MCP, runs the suite, and uses the failures to fix its own code. On every pull request the same suite runs through the GitHub App or GitHub Actions, and the result can block the merge; the merge gate guide shows how.

On larger teams, QA stays the owner of what "correct" means. Access is scoped by workspace roles, every run is attributable to the account that triggered it, and enterprise plans add SSO with SAML, SCIM and audit logs.

Frequently asked questions

Isn't this just distrust of AI?

It is the same separation teams use for people: authors don't approve their own work. Coding agents are excellent authors, and an independent check is what lets you accept their output at the volume they produce.

Can't I just tell the agent not to edit tests?

Do, and it helps. Written rules are the lightest layer of control, though, and they compete with the task the agent was given. A verifier whose expectations the agent can't change removes the option altogether.

Does the verifier need to be a different AI model?

What matters most is a separate role: its own record of intent, expectations the coding agent can't write to, and tests that run against the app. A different model doesn't provide those by itself.

Won't approvals become a bottleneck on a large team?

Only expectation changes need a person. Step updates, such as a renamed button or a moved field, are handled as maintenance without approval. Expectation changes track product decisions your team already makes in tickets, so the feature's owner or the QA lead for that area approves them alongside the ticket.

What if the verifier itself is wrong?

Every verdict comes with a replay, the failing step and expected versus actual, so a wrong test shows up as a test problem you can see and correct, rather than a broken change slipping through.

Try TestSprite

Let the coding agent write. Let a separate agent check.

TestSprite gives your team a verifier with its own approved record of what the product should do, tests that run against the real app, and expectations that move only when a person approves. Your coding agent can run it and act on every failure; it can't rewrite it to pass. Start free with one flow you care about.