12 — RETRICS CRO
Finds the leaks.
Refuses the tests
you cannot win.
Resolved against a held-back control
Resolved, and the answer is no
1,900 sessions/wk — cannot reach power
DEMO STORE · A REFUSAL IS A VERDICT, NOT A FAILURE TO PRODUCE ONE
THREE THINGS IT DOES, AND ONE IT REFUSES
An underpowered test gives you a random answer.
Every other tool in this category is paid, directly or indirectly, by the number of experiments you run. Pricing here is flat and un-metered, so the tool never has an incentive to recommend more traffic-hungry tests.
Finds where revenue leaks
Reads the funnel against your own orders and ranks the drop-offs by the money behind them, not by how easy they are to test.
Runs the test properly
Sizes the audience before it starts, holds a control back, and reports the result once it has actually resolved.
Writes the verdict down
Proven, no effect, or refused — with the arithmetic attached, so a result survives the person who ran it leaving.
Refuses tests you cannot win
Below the traffic a test needs, it says so and does not run. An underpowered test does not give you a weak answer. It gives you a random one.
01 — THE LEAKBUILDING
Ranked by what it costs, not by what is easy to test.
DEMO STORE · SHARE OF LOST ORDERS, NOT SHARE OF SESSIONS
02 — THE POWER CHECKBUILDING
A refusal is a verdict, not a failure to produce one.
This test cannot resolve inside a quarter, so it does not run. The refusal is the verdict.
DEMO STORE · THE ARITHMETIC IS ABOUT YOUR TRAFFIC, NOT ABOUT THE IDEA
PROVEN DOLLARS AND MODELED DOLLARS ARE NEVER ADDED UP
Two ledgers. No total.
The most common dishonesty in conversion tooling is adding a projection to a measurement and reporting the sum. It produces a bigger number and it is not a number about anything.
Proven
Modeled
DEMO STORE · PROVEN IS RESOLVED AGAINST A HELD-BACK CONTROL · MODELED CHOOSES WHAT TO TRY NEXT
QUESTIONS
The four we get asked.
What if it refuses most of my tests?
Then most of your tests could not have resolved, and running them would have produced answers you could not rely on. A refusal tells you the honest thing early: this store needs a bigger effect or more traffic before that question is answerable.
Why is the pricing flat rather than per experiment?
Because a tool paid by the experiment has an incentive to recommend experiments. Flat and un-metered removes the conflict — refusing a test costs Retrics exactly the same as running one.
Why are the two ledgers never added together?
Proven results are resolved against a control your test held back. Modeled numbers are for choosing what to try next. Summing them produces a bigger figure that is not a figure about anything, so the layout has no total row at all.
Can I use it today?
Not yet — CRO is building and is not on the App Store, so there is no price to quote. If you are about to spend a quarter on experiments, tell us about the store and we will open it to you first.
Before you spend a quarter on experiments.
CRO is not on the App Store yet and there is no price to quote. If you are about to spend a quarter on experiments, tell us about the store and we will open it to you first.