Not Every Red Test Is a Real Bug.
A test that fails once and passes on rerun isn't proof of a working feature — but it isn't proof of a broken one either. testsprite test flaky <testId> gives that failure a stability score instead of a shrug.
Built Into the Same CLI You Already Run
Save What You Broke
testsprite test flaky <testId> runs a stability pass and tells you whether a failure is likely a real regression or test fragility — before anyone spends time chasing it.
Understand What You Want
Flaky scoring works on any test — generated or uploaded via test code put — so it applies to your existing suite too, not just new tests.
Validate What You Have
Stability scores come from actual repeated execution against your live environment, not a heuristic guess from static analysis.
Suggest What You Need
A test that comes back flaky still gets a failure bundle — so you can see whether it's timing, selector drift, or an environment issue behind the noise.
$ testsprite test flaky TC_checkout_promo Running stability pass... 4/5 runs passed — stability score: 0.80 → likely flaky, not a regression $ testsprite test flaky TC_orders_create Running stability pass... 1/5 runs passed — stability score: 0.20 → likely a real regression
Don't Spend a Fix Attempt on the Wrong Problem
An agent — or a person — that treats every red test as a real bug wastes cycles chasing noise. Stability scoring is the difference between "investigate this" and "ignore this."
Built for Suites That Have Grown Noisy
Works on Any Test
Generated, uploaded, doesn't matter — test flaky runs on any test ID in your project.
Feeds Back Into the Loop
An agent checking test results can call test flaky before deciding whether a failure needs a fix or just a rerun.
Free Community Version
Offers a free community version, making us accessible to everyone.
Pairs With Diffing
Use testsprite test diff alongside flaky scoring to see exactly what changed between the good runs and the bad one.
Trusted By Businesses Worldwide
"TestSprite offers rich test case generation, clear structure, and easy-to-read code. It also supports simple online debugging with the ability to quickly expand by generating new test cases."
"TestSprite's automation helps us reduce tons of manual work. The developers can easily catch and resolve bugs earlier in the development process."
FAQ
What does testsprite test flaky actually measure?
It runs the test multiple times against your live environment and reports a stability score — how consistently it passes — instead of relying on a single run's result.
How is this different from just rerunning the test?
Rerunning once and taking whichever result you get isn't a score, it's a coin flip. Stability scoring runs enough passes to give you an actual number to reason about.
Does this work on tests I wrote myself, not just generated ones?
Yes — flaky scoring runs on any test in your project, including ones uploaded with test code put.
What do I do with a low stability score?
Treat it as a real regression worth investigating — pull the failure bundle with test failure get to see what actually happened.
What about a high stability score on a run that still failed once?
That's more likely test fragility — timing, selector drift, environment — than a product bug. Worth fixing the test, not necessarily the app.