A/B testing on your store: what you can test without a developer
What a store can test itself, what needs a developer, how long a test must run, and the tests that are worth the effort.
The short answer
A store can test more than owners assume without touching code: product page content and image order, how price and shipping are presented, collection sort defaults, bundle offers, email subject lines and flows, all through theme sections, native features and the email tool. What needs a developer is structural: layouts, new components, anything in the path to checkout, and many of those are better decided from evidence and standards than from a test. The harder truth is volume: a small store does not have enough orders for a statistically meaningful test of a small change. Test big differences on high-traffic templates for weeks, or decide without testing, and log every test with its dates, variants and result, because the log is the asset.
What you can test yourself, and what you cannot
| Yourself | With a developer | Better not tested at all |
|---|---|---|
| Product description structure and length | Product page layout | Accessibility fixes: just do them |
| First image and image order | New components such as sticky add-to-cart | Speed improvements: just do them |
| Shipping and returns message near the price | Collection page structure | Removing checkout surprises: just do them |
| Default collection sort | Navigation changes | Legal and consent elements |
| Bundle and threshold offers | Search behaviour | Button colours: too small to detect |
| Welcome offer amount | Anything in the checkout flow | Headline wording: usually too small |
| Email subjects, timing and content | Custom app behaviour |
Running a test that means something
- Pick a high-traffic template and a change with a plausibly large effect.
- Estimate the orders needed per variant for the effect you hope to see; if you cannot reach it in a few weeks, do not test.
- Define the metric before starting: conversion rate for that template and segment, or revenue per visitor.
- Run for whole weeks, at least two, without peeking and stopping early.
- Read the result honestly: no difference is a common and useful outcome.
- Log it: hypothesis, variants, dates, traffic, result, decision.
- Ship the winner and move to the next large question.
Deciding without a test
Most improvements a small store needs do not require a test to justify: faster pages, clearer product pages, shipping visible before checkout, express payment methods, accessible forms, honest reviews. The evidence for them is general and strong. Save testing for the genuinely uncertain and large: offer structures, merchandising, image strategy, email approaches. That split spends the store’s limited traffic on questions it can actually answer.
What this means for you
Test what you can test yourself on the pages that matter, with changes large enough to detect, for long enough to mean something, and log everything. Leave structural changes to a developer and standard improvements to evidence. A small store that tests three large questions a year properly learns more than one that tests thirty small ones badly, and it wastes far less of its traffic finding out.
Frequently asked questions
Do we have enough traffic to A/B test?
For small changes, probably not: detecting a small difference in conversion needs thousands of orders per variant. For big differences on high-traffic templates, perhaps, over several weeks. Before testing, calculate roughly how many orders each variant needs to show the effect you hope for. If the answer is more than you get in a month, decide from evidence and standards instead, and test only the large changes.
What is worth testing on a store?
Things with a plausible large effect on the pages with the most traffic: the first product image, the shipping and returns message near the price, the default collection sort, a bundle offer, the welcome offer amount. Button colours and headline wording rarely move enough to detect and are the tests that waste months.
Can we test the checkout?
The checkout itself is the platform's, and changing it is neither possible on most plans nor advisable. What can be tested is what feeds it: shipping thresholds, express payment placement, the cart's messaging. Checkout conversion is improved by removing surprises and friction, which does not need a test to justify.