Shopify added A/B testing to the admin. It’s called Rollouts, it shipped as part of the Winter ’26 Edition, and it means you can split-test theme changes without installing anything.
Until now, running an A/B test on Shopify meant a third-party app. Shoplift, Intelligems, Convert — all charged separately, all injected external scripts into your storefront, and all added page weight. Rollouts removes that step entirely.
What Rollouts does
Rollouts lets you stage changes to your online store, control what percentage of visitors see them, and schedule when they go live.
You access it from Online Store > Themes. There’s a Rollouts button next to your published theme. Click it, create a new rollout, give it a name, and set a traffic percentage. The default is 100%, but you can start at 10% or 25% — whatever you’re comfortable with.
From there, you click Changes > Customise, and you’re in the theme editor. Anything you change only applies to that rollout. Your live store stays untouched.
You can run multiple rollouts at the same time. That’s where the A/B testing comes in — two rollouts with different changes, each getting a share of your traffic.
What you can test
Right now, Rollouts handles theme-level changes. Layout, sections, content blocks, design decisions. You can also compare your current published theme against a completely different theme in your library.
You can schedule rollouts to start and end at specific times, which is useful for promotions or seasonal changes where you want to test before committing.
What you can’t test yet
Shopify has been upfront about the current limits. You can’t change theme settings, app embeds, or Liquid templates through a rollout. Product pricing, checkout flows, and merchandising strategies are off the table for now.
There’s no audience segmentation — you can’t show version A to returning customers and version B to new visitors. There’s no custom goal tracking beyond what Shopify’s standard analytics provide.
Shopify has said they plan to expand Rollouts to more surfaces — products, discounts, and other areas — in the coming months.
How to use the traffic controls
The percentage slider is the most useful part. Start a rollout at 10% of traffic. Watch your conversion rate and session data. If numbers hold or improve, push to 25%, then 50%. If something breaks, stop the rollout. Your live store was never affected for 90% of visitors.
This is safer than what most merchants do now, which is publish theme changes and hope nothing goes wrong.
SimGym: the AI pre-test
Alongside Rollouts, Shopify released SimGym — an AI tool that creates simulated shoppers to browse your store. These aren’t bots running random clicks. Shopify built personas from aggregated data across billions of transactions on their platform.
SimGym shoppers navigate your store, add products to cart, and report back on what worked and what didn’t. You can compare your live theme against another theme in your library and see which one gets a higher add-to-cart rate from the simulated shoppers.
The idea is to catch problems before you run a live Rollout. SimGym flags friction in navigation, layout, and checkout flow so you can fix it before real customers see it.
Who gets it
Rollouts is in early access now. It’s available across Shopify plans, though the full feature set favours Shopify Plus. Standard plan merchants will get a reduced version — Shopify hasn’t confirmed exactly what’s included or excluded for each tier yet.
What this means for third-party testing apps
It depends on what you need. If you’re running basic theme tests — does this banner perform better than that one, does moving the add-to-cart button increase conversions — Rollouts covers it without the extra cost or script overhead.
If you need granular segmentation, custom conversion goals, heatmaps, or element-level testing within a page, you’ll still need a dedicated tool. Rollouts is a v1 product and Shopify has positioned it more as a safe deployment tool than a full CRO platform.
For most small and mid-sized Shopify stores that weren’t paying for testing tools at all, this is a significant addition. You can now make data-informed decisions about your storefront straight from the admin.
Author
