tacey
Install on Shopify
Shopify's Rollouts feature gives merchants a native tool for scheduling and testing storefront changes. Instead of publishing a new discount and hoping for the best, you can now show it to a small slice of your audience first. This lets you measure its impact on conversion rates and average order value before committing to a site-wide launch.
This guide explains how to use the shopify rollouts feature to test different discount offers. We will cover what it is, how to set one up, and what its current limitations are. The goal is to gather real data on what works for your customers without taking on unnecessary risk.
What is the Shopify Rollouts feature?
Shopify Rollouts is a feature for scheduling and testing changes to your online store before they go live permanently. Found in your Shopify admin under Markets, it allows you to create variations of your theme, checkout, or discounts and deploy them to a specific percentage of your traffic. This process is also known as A/B testing or split testing.
You can use Rollouts to test a new homepage layout, a different call-to-action button, or a new promotional banner. Since October 2026, this also includes the ability to test discounts directly. You can schedule these changes to go live and end at specific times, which is useful for seasonal sales and campaigns.
Because Rollouts runs on Shopify's servers, it doesn't slow down your site with extra code. This is a key advantage over many third-party apps that can sometimes cause page-load delays or flickering as the test version loads for a visitor. This server-side approach is a major benefit for maintaining site performance.
How does Rollouts differ from A/B testing apps?
The main difference is that Rollouts is a native feature built directly into your Shopify admin. There are no third-party apps to install or extra monthly costs associated with the feature itself. This integration means it has no impact on your store's page speed, a common trade-off with client-side testing tools.
This server-side versus client-side distinction is the most important trade-off to understand. Client-side apps run JavaScript in the visitor's browser to make changes after the original page loads. This can create a "flash of original content" (FOUC), where visitors momentarily see the old version before the new one appears, which can harm trust and conversion.
Rollouts avoids this flicker entirely by deciding which version to send at the server level, before it ever reaches the visitor's browser. The trade-off is that third-party apps often provide more advanced features, such as deeper analytics, audience segmentation, and the ability to test things Rollouts cannot. For example, many dedicated apps can test different product prices, which Rollouts does not support.
For most theme-level visual changes and straightforward discount tests, Rollouts provides the core functionality many merchants need. If your testing needs are more complex, like running multivariate tests or targeting specific visitor segments like new vs. returning customers, a dedicated A/B testing app might still be necessary.
Can you use Rollouts to test discount strategies?
Yes, as of an update in October 2026, you can test discounts using the Rollouts feature. This allows you to create an experiment that shows a specific discount to one group of visitors while another group sees your standard offers. This is a direct way to measure if a new promotion drives more sales or a higher average order value.
For example, you could test a "15% off" offer against a "Free Shipping" offer. By directing 50% of your traffic to each version, you can see which promotion is more effective at converting customers. The feature is designed to handle this comparison within Shopify's own analytics.
Good candidates for testing include percentage-off versus fixed-amount discounts, or different thresholds for free shipping. You could also test a "Buy One, Get One" (BOGO) deal against a simple 50% off one item to see which offer structure generates more overall revenue. This helps you move beyond guesswork in your promotional planning.
However, it's important to know that running experiments, including for discounts, requires your store to be on the Shopify Advanced plan or higher. While stores on lower plans can use Rollouts to schedule changes, the ability to split traffic and view experiment analytics is limited to higher-tier plans.
How do you set up a discount test with Rollouts?
Setting up a discount experiment involves a few clear steps inside your Shopify admin. Before you start, ensure the discount you want to test has been created under the main "Discounts" section of your admin. The rollout will activate or deactivate this existing discount for a portion of your audience.
- Navigate to Rollouts: In your Shopify admin, go to Markets and then click on Rollouts.
- Create a Rollout: Click the "Create rollout" button. Give your test a clear name, like "BFCM 20% Off Test," and ensure you select "Experiment" as the rollout type.
- Add Your Discount: In the "Changes" section, click "Add change" and select "Discounts" from the list. Choose the discount you previously created that you want to test.
- Set Traffic Allocation: Use the slider to decide what percentage of your online store visitors will see the new discount. For a simple A/B test, you might set this to 50%.
- Schedule the Test: Set a start and end date for your experiment. For reliable results, it's often recommended to run a test for at least one full week to capture different shopping patterns.
- Start the Rollout: Once you've reviewed the settings, you can launch the experiment. Shopify will then automatically divide your traffic and start collecting data.
Worked Example: Testing a Free Shipping Threshold
Imagine your store currently offers free shipping on orders over $100. You hypothesize that lowering this to $75 might increase your conversion rate. Your average order value is currently $90. You create a new free shipping discount for orders over $75 in your Discounts admin.
In Rollouts, you create an experiment named "Free Shipping $75 Threshold Test." You add the new discount as the change and set the traffic allocation to 50%. This means half your visitors will see the original $100 threshold (the control), and half will see the new $75 threshold (the treatment). You schedule the test to run for two weeks.
What metrics should you track for a discount test?
The primary goal of testing a discount is to see how it affects customer behavior and your bottom line. Shopify Rollouts tracks key metrics automatically, allowing you to compare the test group (treatment) against the control group (your standard store experience). Focus on a primary metric to make a clear decision.
Key metrics to watch include the online store conversion rate, average order value (AOV), and total sales. A discount might increase your conversion rate but lower your AOV. The ideal outcome is a positive lift in overall revenue without significantly harming your margins.
Also, keep an eye on your return rate. An aggressive discount might attract bargain hunters who are less committed to their purchase. If the test group shows a spike in returns, the initial lift in sales could be misleading. A successful promotion drives profitable, lasting sales, not a temporary revenue boost followed by costly reverse logistics.
H3: Calculating the Real Cost of a Discount
A higher conversion rate can be misleading if it comes at the expense of profit. Imagine a product that sells for $100 and costs you $60 to make, leaving a $40 gross profit. Your current conversion rate is 2%. For every 100 visitors, you make two sales, generating $80 in profit (2 x $40).
You test a 20% discount. The new price is $80. Your profit per unit is now only $20 ($80 price - $60 cost). The test shows your conversion rate increased to 2.8%. For every 100 visitors, you now make 2.8 sales. However, your total profit is now just $56 (2.8 x $20). The discount lost you money, even though conversion went up.
This is the most common trap of discount strategies: focusing on revenue while ignoring profit margins. A successful test must generate more total gross profit, not just more sales. Always calculate the impact on your per-unit profit before deciding on a winner.
Check the analytics for your rollout regularly. Shopify will display the performance of each variant and a confidence interval to help you determine if the results are statistically significant. Avoid ending a test too early based on a small number of visitors. A test needs enough data to be reliable.
What are the limitations of using Rollouts?
While powerful, the Shopify Rollouts feature has some limitations you should be aware of. It is primarily designed for testing changes within the theme editor, checkout configurations, and discounts. It cannot be used to A/B test different product prices or fundamental changes to your theme's underlying Liquid code.
The feature also lacks advanced audience segmentation. You cannot target experiments to specific groups, such as mobile users only or visitors from a certain country, though you can limit rollouts by market. Furthermore, you can only track standard Shopify analytics like conversion rate and AOV, not custom goals like "add to cart" rate.
Another edge case is non-human traffic. A sudden wave of bot activity can easily pollute your test results. If bots disproportionately hit one version of your site, they can skew metrics like conversion rate, making a losing variant look like a winner. If you see unusual traffic patterns during a test, it may be wise to pause and investigate before trusting the data.
One edge case to consider is visitor cookie clearing. Shopify uses cookies to ensure a visitor consistently sees the same version of your site. If a visitor in the test group clears their browser cookies mid-test and returns, they could be randomly assigned to the control group. This can slightly dilute test results over time, though it is a minor issue for most tests.
Finally, running a true A/B test with traffic splitting and analytics requires a Shopify Advanced plan or higher. Merchants on Basic or Shopify plans can schedule changes but cannot run experiments to compare performance between two versions.
How can you handle customer service issues from a test?
Running a public discount test can sometimes create customer confusion. A shopper might hear about a discount from a friend who was in the test group, or they might see a promotional code but forget to apply it before completing their purchase. This can lead to an increase in support tickets asking for the discount to be applied retroactively.
Before launching any test, prepare your customer service team. Give them a clear script and policy for handling inquiries about the test discount. Decide in advance: will you honor the discount for customers who ask? Having a firm, consistent answer prevents agent confusion and ensures all customers are treated fairly, protecting your brand's reputation during the experiment.
The cost of this friction is real. A support agent spending 15 minutes manually creating a draft order or issuing a partial refund for a $5 discount costs the business more than the discount itself. Worse, a frustrated customer who feels they missed out might initiate a chargeback or decide not to shop with you again.
Managing these requests manually can be time-consuming for your team. One way to address this is by giving customers a way to solve the problem themselves. After an order is paid for, a tool can give them the option on the order status page to apply a discount code they may have missed.
Your customer already wants to buy from you. The goal is to remove any friction that happens after they've already committed, not to create more work for your support team.
This approach helps save a sale you've already won and prevents the frustration that leads to order cancellations. For merchants looking to solve this specific issue, Tacey provides a way for customers to apply forgotten coupon codes on the post-purchase page. You can read more about how to handle this in our guide on applying discounts to paid orders.
What happens after the discount test is complete?
Once your rollout has reached its scheduled end date or you have gathered enough data to make a decision, you have two main options. You can either end the rollout or promote the winning version. The right choice depends entirely on the results of your experiment.
If your new discount significantly outperformed the control group, you can promote it to 100% of your traffic. This makes the change permanent for all visitors moving forward. You can do this from the Rollouts dashboard by selecting the winning variant and applying it to your live store.
Before promoting a winner, ensure the results are statistically significant. A test that is ahead by 5% after only one day is not reliable. Shopify's analytics include a confidence interval to help with this. It shows a range of likely outcomes, not just one number. A result is trustworthy only when the confidence ranges of the two versions no longer overlap.
If the new discount performed worse or showed no meaningful difference, you can simply end the rollout. When the experiment ends, all traffic automatically reverts to the original control version of your store. This ensures that an unsuccessful test doesn't have a lasting negative impact on your business.
The true value of using the shopify rollouts feature is not just in finding winning ideas, but also in avoiding the cost of rolling out losing ones. By testing your discount strategies in a controlled way, you can make data-driven decisions that lead to more profitable growth. Start with a small, simple test to get comfortable with the process, and build your experimentation habit from there.
Frequently asked questions
Does Shopify have built-in A/B testing?
Yes, Shopify has a native A/B testing feature called Rollouts. It allows merchants to test variations of their theme, checkout, and discounts by splitting traffic between a control version and a test version to see which performs better.
Is the Shopify Rollouts feature free?
The Rollouts feature itself is included in your Shopify plan at no extra monthly cost. However, to use its A/B testing capabilities (splitting traffic and viewing experiment analytics), your store must be on the Shopify Advanced plan or higher.
What can you test with Shopify Rollouts?
You can test changes made in the theme editor, such as homepage layouts, promotional banners, and navigation. You can also test different checkout configurations and, as of October 2026, you can A/B test different discount offers.
What can't you test with Shopify Rollouts?
Shopify Rollouts cannot be used to test different product prices, changes to underlying Liquid template files, or app embeds. It also lacks advanced audience segmentation and custom conversion goals found in some third-party testing apps.
Does using Shopify Rollouts affect my store's speed?
No, using Rollouts does not negatively impact your page speed. The feature runs server-side, meaning Shopify's infrastructure decides which version of the site to show a visitor before the page is sent to their browser, preventing any slowdown or flickering.
Where do I find Rollouts in my Shopify admin?
You can find the Rollouts feature by navigating to the Markets section in your Shopify admin. From there, you will see a "Rollouts" option where you can create, manage, and view the analytics for your tests.
Can I schedule a theme change for a future date?
Yes, one of the core functions of Rollouts is scheduling. You can prepare changes for a sales event or a new campaign in advance and schedule them to go live at a specific date and time, and also set a time for them to automatically end.
Install on Shopify
