Funnel experiments for a small business: one change at a time

Advice on testing is written for sites with thousands of visitors. You may get forty enquiries in a good stretch. If version A brings 21 of them to a quote and version B brings 19, nothing has been learned, because luck alone produces that gap. A small business can still test honestly, but only by changing one thing, splitting enquiries at the same time, judging against a gap you measured beforehand, and calling anything smaller a tie. This post shows how to measure that gap using your own records, with no formulas.

First, find out how much luck moves your numbers

Before testing anything, run a test where nothing differs. Take your last forty enquiries, sorted by date. Put the odd-numbered ones in one pile and the even-numbered ones in the other. Count how many in each pile reached a quote, or whichever step you care about.

Both piles were treated identically, yet the counts will differ. Say 9 and 14. That gap of five is pure luck, and it tells you something important: a real change must beat five, not just beat zero. Repeat with the previous forty if you have them. The largest gap you see across the splits is your noise floor.

Now you have a bar built from your own business, not copied from a textbook. Any test result that lands inside the noise floor is a tie, however good it feels.

Choose a change big enough to beat the noise

Because small counts hide small effects, test changes you expect to be large. Order them by how near they sit to the order. The reply can be the best first test, since every enquiry passes through it and no website change is needed.

A small change in a headline will rarely clear a noise floor of five in forty. Do not test it. Change it if you like it and move on.

The log, written before the first enquiry

1. The change, in one sentence.
2. The reason you think it will help.
3. The split: odd to A, even to B, unless the source differs.
4. The main count, one figure, and one guard figure such as complaints.
5. The bar: the noise floor from the first section, as a number of enquiries.
6. The stop point: forty per version, not the day your curiosity peaks.
7. The decision for B wins, A wins and a tie.

Write lines five to seven before you begin. After the fact, everyone picks numbers that suit what they saw.

Imagine: a fabric trader in Bhilwara and the swatch offer

Take an invented suiting-cloth trader in Bhilwara who sells to tailors and small garment makers across Rajasthan and beyond. Enquiries arrive on WhatsApp. The trader believes that offering five free swatches by courier wins more orders, but each parcel costs about ₹150 to courier, so proof is wanted.

The noise floor, from splitting old enquiries, was four in forty, and the bar was set at six, to be safe. Odd enquiries got "we will courier swatches free". Even enquiries were asked to pay the courier, refunded on the first order. The main count was an order within the next stretch of weeks; the guard count was complaints.

After forty each, the free-swatch group gave 13 orders and paid-courier buyers 9. The gap was four, inside the noise. The result was recorded as a tie. The cheaper offer, the paid courier, had not lost, so it stayed. Stopping at the first twenty, with 8 against 4, would have changed everything on a lucky run. All figures are illustrative.

The guard column was read too. A few tailors had said the refund felt fussy, and the message was reworded. That was a judgement, not a result, and the notes kept the two apart.

When not to test

Skip a test when you will not reach forty per version in the time you can wait. Skip it in a festival fortnight, a wedding season or a monsoon week, when the mix of buyers shifts under you. Skip it when the change is plainly right: if your form fails on a phone, mend the form.

Skip it too where you cannot hold the split. If one person takes every call and favours the script they like, the log records that person's mood, not the buyers.

What a result does and does not tell you

A clear win says the change worked for this kind of enquiry, from these buyers, at this time. It does not say why, and it does not carry over to another product. Run it again when something large changes, such as a new source of enquiries.

Keep a short list of every test with its bar and outcome, ties included. A tie tells you the thing you worried about does not matter much, which frees your attention for something that does.

Run the splitting exercise first

Sort your last forty enquiries into odd and even piles and look at the gap. It takes ten minutes and sets your bar. What you learn feeds your marketing plan, and funnel experimentation and A/B testing is the service for testing one change at a time. If you would like a second pair of eyes on which test goes first, tell us what you need.