The short answer
Make your end-to-end regression suite a required status check. With TestSprite, that is one toggle in the GitHub App, one scaffolded workflow in GitHub Actions, or one CLI command in any other CI. Every pull request, whether a person or Claude Code, Codex or Cursor opened it, gets the full suite run against its own build, and the merge button stays locked until the suite passes.
Every pull request is tested
The full regression suite runs against the pull request's preview deployment, frontend and backend, in real browsers.
The result lands on the PR
Passed, failed and blocked counts, with each failure's replay, failing step, and expected versus actual.
Red means something
Every failure is classified as a real bug, a test problem or an environment issue, so the gate blocks bugs, not noise.
What a good merge gate does
A gate earns trust when developers stop looking for ways around it. These seven properties get it there:
It runs on every pull request, including the ones coding agents open, with no one having to remember.
It tests the pull request's own build: a preview deployment or an environment built from that branch, not yesterday's staging.
It runs the full regression suite. Agent changes tend to break flows far from the files they touched, so "affected tests only" misses exactly what you need to catch.
It puts evidence on the pull request, so reviewers and agents can act without opening another tool.
It classifies failures, so an environment hiccup doesn't read like a product bug.
It holds expectations steady. A pull request can't make the gate pass by editing what the tests check for; expectation changes need a person's approval.
It has a clear override policy, used rarely and recorded every time.
Option 1: the GitHub App, with no workflow file
If your platform deploys a preview for every pull request (Vercel, Netlify, AWS Amplify, or a pipeline that creates GitHub deployments), this is the quickest route, and it needs no changes to your repository:
In TestSprite, open Workspace Settings → Integrations and connect GitHub.
In your project's GitHub Action tab, connect the repository and choose the Pull request trigger.
Pick the event that fires once the preview is live, and enter the pattern for its URL, for example
https://pr-{pr}.example.com.Click Send Test Event, confirm the comment on the pull request points at the right environment, then create the trigger.
Turn on Block PR until tests pass. The TestSprite check becomes required, and failing tests block the merge.
The Vercel preview guide walks through each screen, including previews behind deployment protection.
Option 2: GitHub Actions
Prefer to drive the run from your own workflow? The open-source TestSprite CLI scaffolds one:
npm install -g @testsprite/testsprite-cli
testsprite setup
testsprite ci init github # writes a workflow for your pull requests
Then make it required. In the repository's settings, add a branch protection rule or ruleset for your main branch, enable Require status checks to pass before merging, and select the TestSprite job. From then on a failing run blocks the merge for everyone, agents included.
Option 3: any other CI
GitLab CI, Jenkins, CircleCI, Azure Pipelines and the rest only need the CLI and an API key from your secret store. The exit code is the verdict:
npm install -g @testsprite/testsprite-cli
# TESTSPRITE_API_KEY comes from your CI secret store
testsprite ci run --project <project-id> --env <name> \
--target-url "$PREVIEW_URL" --report junit --report-file results.xml
# the exit code is the verdict: a failing suite fails the job
The JUnit file (results.xml) plugs into the test report view most CI systems already have, and marking the job as required in your merge settings turns it into a gate.
Read a red check fast: three possible answers
A gate is only as useful as the speed at which people act on it. Every TestSprite failure arrives with its classification and evidence, so triage starts with the answer instead of an investigation:
| Verdict | What it means | Who acts | The pull request |
|---|---|---|---|
| Real bug | The app does something other than what the test expects | The author, or the coding agent with the failing step and expected versus actual as input | Stays blocked until the code is fixed |
| Test problem | The test's steps no longer match the app, while the expectation still holds | The test is maintained; the owner reviews | Re-run once the test is current |
| Environment issue | The preview didn't load, a login wall appeared, test data or a third-party sandbox was missing | Whoever owns the environment | Re-run after the environment is fixed |
| Intended change | The product is meant to behave differently now | The feature owner approves the new expectation | Re-run against the approved expectation |
The routine for each failure is the same four moves: read the classification, jump the replay to the failing step, compare expected with actual, and hand it to the right owner. Reviewers can then judge the pull request on that evidence, as the guide to reviewing agent pull requests describes. When a coding agent wrote the change, the pull request comment also carries a suggested fix prompt it can take straight into its next turn.
Policies that keep the gate trusted
One gate for everyone. Agent pull requests and human pull requests pass the same check.
Overrides are rare and recorded: who merged past a red check, and why.
"Probably flaky" isn't a verdict. Look at the classification and the replay, then decide.
Expectation changes go to the feature owner, with the ticket as the reason.
Fix environment failures at the source. Stable preview URLs, a dedicated test account and seeded data are the usual cures.
Review triage trends. More test problems means maintenance is lagging; more environment issues means the preview pipeline needs attention.
Rolling it out across a large engineering org
Start with one repository and run the check in report-only mode, so results post to pull requests without blocking. Use that period to cover the flows that matter, settle environment issues and agree on owners. Then make the check required on the main branch, and expand repository by repository.
TestSprite is built for this scale. QA stays the owner of what "correct" means: they review the tests TestSprite proposes and approve what becomes the regression suite, no code required. Access is scoped by workspace roles, every run is attributable to the account that triggered it, and TestSprite maintains SOC 2 compliance. Enterprise plans add SSO with SAML, SCIM, audit logs, single-tenant deployment, data residency and IP allowlisting.
Why teams use TestSprite as the gate
Tests come from intent. Your words, PRDs and tickets become an approved test plan, so the gate checks what the product should do, not what the code happens to do.
The suite keeps up. TestSprite updates steps as the interface changes, and what a test expects only changes when you approve it.
Frontend and backend in one gate. Real browsers for the UI, multi-request flows with authentication for the API.
It fits your stack. GitHub App, GitHub Actions, any CI through the open-source CLI, and it works beside an existing Playwright suite.
Frequently asked questions
Should the gate run the full suite or only affected tests?
The full suite. Coding agents often break flows far from the files they changed, and those are the regressions an affected-tests filter skips. TestSprite runs tests in parallel on its own machines, so the full suite stays practical per pull request.
How do I know the labels are right?
You don't have to take them on trust. Every failure comes with a replay, the failing step, and expected versus actual, so anyone can check a verdict in a minute. Run it on your own app on the free tier and compare its calls with your own before you make the check required.
What if our preview environments are unreliable?
Those failures come back classified as environment issues, not product bugs, so they point you at the pipeline to fix. Re-run once the preview is healthy; don't merge on an assumption.
Does this work outside GitHub?
Yes. The open-source CLI runs the suite from GitLab CI, Jenkins, CircleCI, Azure Pipelines or any system that can run a command, and its exit code is the verdict.
Can the coding agent act on a failed gate?
Yes. The pull request comment gives the failing step, expected versus actual, a replay and a suggested fix prompt, and the agent can re-run the suite itself through the CLI or MCP; the Claude Code testing guide shows that loop. It can't change what a test expects.
Keep main green, no matter who opened the pull request.
Turn your regression suite into a required check that every agent and every engineer passes. TestSprite runs the full suite on each pull request, posts the evidence where reviewers already look, and tells you whether a failure is a real bug, a test problem or the environment. Connect a repository on the free tier today.