Why A/B testing sometimes feels too slow
Let’s say you’re working on a product or marketing team and you need to choose the best option out of a few ideas.
Maybe it’s:
- 4 different email subject lines
- 3 versions of a landing page button
- 5 ad creatives for the same campaign
The usual approach is A/B testing (or A/B/C/D…).
You split users evenly, wait for enough data, then decide the “winner”.
That works, but it has one big problem:
you spend a lot of time showing weak options to a lot of users.
If one option is clearly better, you still keep wasting traffic on the others just because you want the test to be “fair”.
So the real question becomes:
Can we test multiple options AND still get good results during the testing period?
That’s exactly what Multi-Armed Bandits are built for.
What is a Multi-Armed Bandit?
The term sounds complicated, but the idea is actually very easy.
Imagine a casino with multiple slot machines.
Each machine is a little different, and each gives random rewards.
You don’t know which one pays the most.
You have two choices:
- Keep trying different machines to learn which is best
- Stick to the best machine you’ve seen so far and earn more money
This is the core problem.
In real life, the “slot machines” are your options:
- email subject lines
- ads
- page layouts
- product recommendations
- notification messages
And the “reward” is something measurable:
- click
- sign-up
- purchase
- time spent
- conversion
A Multi-Armed Bandit is a strategy that helps you choose between options in a smart way.
It learns over time and tries to maximize the total reward while testing.
The key idea: Exploration vs Exploitation
Bandits are built around one tradeoff:
Exploration
Try different options to learn what works.
Example: - “I haven’t tried Ad C much. Let me test it more.”
Exploitation
Use the best option so far to get results now.
Example: - “Ad A is performing best. Let me show it more often.”
The best bandit strategies balance both.
This is the main difference from normal A/B testing:
A/B testing tries to learn first and win later.
Bandits try to learn and win at the same time.
A realistic example
Let’s say your team is testing 4 push notification messages to increase app opens.
The A/B/C/D test way
- Split users 25% each
- Run for 2 weeks
- Pick winner at the end
Downside: Even if 1 message is clearly bad, you still waste 25% traffic on it for 2 weeks.
The bandit way
- Start fairly (like 25% each)
- After each day, update results
- Gradually shift more users to the better messages
So after a few days, the best message might be getting 50% or 70% of traffic.
You still test the others, but less.
That means, you learn, you improve performance during the test and you waste less traffic
A simple way to think of bandits
Most bandit systems repeat this loop:
- Pick an option (one “arm”)
- Show it to a user
- Observe reward (click, conversion, etc.)
- Update your belief about that option
- Repeat
Over time, the better options naturally get picked more.
Common bandit strategies (no heavy math)
You don’t need to know equations to understand the main types.
1) Epsilon-Greedy (most popular)
This one is very easy:
- Most of the time: pick the best option so far
- Sometimes: pick randomly to explore
Example: - 90% of the time → show the current best ad
- 10% of the time → test other ads
That small random part is important because the “best so far” might be wrong early on.
Good because: easy to implement
Not great because: exploration is random, not always smart
2) UCB (Upper Confidence Bound)
UCB is like saying:
“Pick the option that looks best, but also has uncertainty left.”
So it prefers options that: - have good results or - haven’t been tested enough yet
This means it explores in a more controlled way.
Good because: exploration is more intelligent than random
Not great because: can be sensitive if reward patterns change a lot
3) Thompson Sampling
Thompson Sampling works like this:
- It keeps a probability belief for each option
- It samples from those beliefs
- It picks the option that “wins” in the sample
The effect is very natural:
- if an option is strong → it gets picked a lot
- if an option is uncertain → it still gets chances
- if an option is bad → it slowly disappears
This strategy often performs really well in practice.
Good because: smart balance of explore + exploit
Not great because: harder to explain to non-technical people at first
When should you use bandits?
Bandits are great when:
– You have ongoing decisions
(example: ads run every day, not once)
– You want to improve results right now
(not only after the test ends)
– You have a clear reward signal
(click, conversion, open, etc.)
– You can track outcomes properly
(logging matters a lot)
They are especially useful in:
- marketing experiments
- product feature rollouts
- recommendation systems
- pricing experiments
- customer engagement messages
When bandits are NOT a good idea
Bandits are not perfect. In some cases, classic A/B testing is better.
1) When you need a clean final comparison
If your goal is to say:
“Variant B is statistically better than A by X%”
A normal controlled experiment might be more appropriate.
Bandits focus on performance, not clean reporting.
2) When you care about long-term impact, not short-term reward
Sometimes a click is not the real goal.
Example: If you optimize for clicks only, you might end up with annoying clickbait.
Bandits will do what you tell them to do, so the reward must be chosen carefully.
3) When the environment changes often
Users change. Seasons change. Trends change.
If your “best option” changes every week, then a bandit model might need to be adjusted for that (or re-trained often).
Common mistakes people make:
Here are a few bandit mistakes I’ve seen (and they happen easily):
Trusting early results too quickly
The first 50 clicks can be misleading. Bandits should learn gradually.
Not exploring enough
If you explore too little, you might miss a better option.
Messy tracking
If clicks are logged incorrectly or conversions don’t match the right variant, your learning becomes useless.
Optimizing the wrong metric
Clicks are easy to measure, but sometimes retention and satisfaction matter more.
The biggest takeaway
If I had to explain multi-armed bandits in one line, I’d say:
Bandits are a way to test multiple options while slowly shifting more traffic to what works best.
They are useful because they reduce wasted effort. Instead of waiting for the experiment to finish, you start benefiting early.
A/B testing is great when you want a clean conclusion.
Bandits are great when you want performance + learning at the same time.
If you’ve ever felt like A/B testing wastes too much traffic, bandits are a very practical next step to learn.