The standard answer is two weeks. It is repeated everywhere, it is easy to remember, and it is wrong often enough to be worth replacing.
We have run one test that settled in six days and another that ran seventeen and still could not be read. Both were Shopify product pages. The calendar was never the variable.
What actually decides it
A test can be read when it has collected enough conversion events to distinguish a real difference from ordinary variance. Three things drive that:
- How many conversions you get — not visitors. Events are the currency.
- How big the difference is — a large effect announces itself quickly; a small one needs far more data.
- How certain you want to be — the 95% convention is a choice, not a law, but it is the one everyone else is using.
Duration is downstream of all three. Two weeks on a page with real traffic and a large effect is generous. Two weeks on a page with 300 visitors is a fortnight spent learning nothing.
Six days, and why it was enough
The InStrength Omega-3 page had three pack options presented as plain unpriced buttons. We priced every tier, showed the per-softgel unit cost, and moved the certifications into the buying block.
546 visitors. Six days. Add-to-cart 12.94% → 20.38%. p = 0.02, 99% win chance.
Six days sounds irresponsibly short. It was enough because the difference was big. When a change moves the metric by half again, it does not take much data to see it.
Short tests are not automatically bad and long tests are not automatically trustworthy. What matters is whether enough events accumulated, and a large effect gets there faster.
Seventeen days, and why it was not
The Pawpeye page ran nearly three times as long. Add-to-cart went 4.26% → 7.53%, which looks like a 77% lift.
It rested on 11 conversions against 6, on 287 visitors. Shopify put the win probability at 87.4% — below the bar — and flagged that significance was unlikely at that traffic level.
Seventeen days did not fix that, and thirty-four would not have either. The page simply does not get enough traffic for a difference that size to separate from noise in a reasonable window.
If a page cannot reach a conclusion in a sensible window, running the test for longer is not the answer. Test a page with more traffic, or accept you are measuring direction rather than proving anything — and label it that way.
Talk it through
Want to know whether your page can be tested at all?
Send the URL and rough monthly traffic. We will tell you whether a test on it could settle, and how long that would realistically take — before anyone spends money finding out.
Check your trafficA real read within one business day — not a sales call.
Three rules worth keeping
Never stop a test because it looks good. Peeking and stopping at the first happy moment is the single most effective way to manufacture a false positive. Decide the finish line before you start.
Always run whole weeks. Tuesday traffic does not behave like Saturday traffic. A test that runs six days spanning a full week is fine; one that runs six days across two partial weekends is not.
Do not test during anything unusual. Black Friday, a launch, a big paid push. You are measuring your page, not your promotion calendar.
A rough guide
| Page traffic | What to expect |
|---|---|
| Under ~500/month | Not testable. Fix the obvious structural problems and move on |
| ~2,000/month | Only large effects will show, and slowly. Test big changes, not button colours |
| ~10,000/month | Workable. Most meaningful changes settle in two to three weeks |
| 50,000+/month | Comfortable. Smaller effects become detectable and you can test continuously |
These are rough, and deliberately so — the honest version depends on your conversion rate and how big a change you are making. But they are closer to useful than "run it for two weeks."
The question to ask before testing anything
Not "how long should this run", but: can this page produce enough conversions to settle in a window I am prepared to wait?
If yes, plan the finish line and hold to it. If no, say so out loud, and either pick a different page or run it knowing you are getting a direction rather than a proof.
We would rather tell a client a page cannot be tested than run something neither of us can read. Send us a URL and your rough traffic, and we will tell you which one you have.
About the author
Manpreet Singh
Manpreet Singh is the founder of Proscube, an ecommerce growth studio. He leads the studio's Shopify and Shopify Plus engineering, headless builds, CRO, and its work on AI engine optimization, and writes its guidance on how to grow a DTC brand without wasting money. He works directly with founders — no account-manager layers between you and the people doing the work — and would rather tell a client not to build something than sell them work they don't need.
More about Manpreet
