In October 2017, Facebook added a flag called learning phase, visible for ad sets in the Ads Manager interface. As the name suggests, learning phase is the period during which Facebook is gathering the data needed to optimize your ad delivery. Your ads will be delivered more experimentally during this period, so that Facebook learns about the audience, the ad's performance, and how high it needs to bid in order to pace your budget optimally.
To see the learning phase status of your ad sets in Smartly.io, add the Learning Phase column to your Campaigns reporting view.
Note that some edits to ads and ad sets reset learnings. See Which edits reset learnings? below for more information.
How does the learning phase work?
You can ”bid for conversions” on Facebook, but in each ad auction, the item being auctioned is an impression – not a conversion. So, what should Facebook’s bidding robot bid for an impression on someone's timeline, if you bid €50 per purchase? Basically, the formula will be
€50 × the probability of that person clicking × the probability of that person converting after the click
How does Facebook learn those odds, then? By knowing the characteristics (age, gender, behavior on and off Facebook...) of Facebook users who purchased earlier, after seeing that ad. Even with machine learning magic, estimating those probabilities requires that you have information of roughly 50 successful cases of someone purchasing after seeing the ad.
Note: this whole thing hinges on the conversion being a purchase – if you change your bid conversion goal to an app install, the robot needs to re-learn how users who saw the ad and installed the app look like. Likewise, if you change the audience or creative, you change the definitions of “Facebook users”, and “that ad”, and all of this has to be learned again.
When you create a new ad or edit an existing one, some of the learnings gathered by the ad are reset, and Facebook needs to re-learn how the new ad works. Performance might be more volatile and more aggressive during this phase, as the ad will also be delivered to people who might not be the optimal target audience, for the sake of experimenting.
The above implies that before the system has learnt how people who convert look like, bidding for conversions can’t guarantee you better results than bidding for clicks – because those “probabilities” in the calculation can be way off. No advertiser who’s really after purchases wants to pay for clicks anymore, so keeping the delivery system happy is quite important.
Which edits reset learnings?
Some edits reset more learnings than others:
- For example, budget changes only affect the budget pacing algorithm, which needs to re-learn how to spend your budget optimally during the day (and lifetime). Click-through rate and conversion rate estimations, which are essential for delivery optimization, are not reset.
- Bid changes are a bit more severe, as even small bid changes might have big effects on who your ads can reach.
- Changes to tracking parameters don't affect ad delivery, so no learnings are reset there.
- Edits to targeting or the creative itself, such as the link URL, message, image or video, can make such big changes to the ad's performance that Facebook resets all learnings about how the ad performs, and what kind of people it works for.
In addition to learning what kind of users convert, there’s another layer of learning involved when your budget is limited. If you have €1000 to spend during a day, how high should the bidding robot bid in auctions to get the maximum amount of purchases? It needs to estimate how many users are online in the morning, afternoon and evening – and how likely they are to convert. If it bids too high and reaches too expensive users in the morning, there will be no budget left in the evening. Therefore, Facebook's bidding robot is adjusting the real "paced bid" up and down and seeing how that affects your spending rate. This mechanism is called budget pacing. This layer of learning is disturbed by bid or budget changes, but the good news is that Facebook can re-learn this part in hours.
There is no need to worry about this — the learning phase has always been part of Facebook's ad delivery system, although it has not been visible to end users. Smartly.io's optimization features have been designed with this in mind, and the Facebook Delivery team at Facebook has confirmed that the budget changes made by Predictive Budget Allocation are not harmful for performance. For example, Predictive Budget Allocation and Bid Optimization features change budgets and bids only once per day, shortly after midnight, to have a minimal effect on pacing.
For more information, read Jon Loomer's blog post on edits that trigger the learning phase.
Learning phase in Ads Manager
Ads Manager shows which ads are currently in the learning phase. This indicator is largely cosmetic – more of a rule of thumb than an exact measure. It is triggered when you make the following changes to your ads:
- Ad creative*
- Optimization Goal selection*
- Pausing your ad set (or the campaign it's in) for 7 days or longer (the learning phase will reset once you un-pause the ad set or campaign)
*Changes to targeting, ad creative and optimization goal are significant edits that requires Facebook to re-learn how the ad performs, and which kind of people it works for. Changes to budget, bid and scheduling do not reset Facebook's knowledge of click-through rates or conversion rates (CVR), so re-learning those is significantly easier. The Learning Phase indicator in Ads Manager is a cosmetic simplification, according to Facebook engineers in the Facebook Delivery team.
The learning phase will be shown in Ads Manager until there are 50 conversions that the ad set is optimized towards. If the ad set does not get 50 conversions in 7 days, it will display a Learning Unsuccessful status.
Changes in the ads can reset some ad level learnings. However, learnings are stored on multiple levels in the campaign structure: ad account, campaign and ad set. In other words, an edit to the ad does not cause Facebook to throw away all learnings. This has been taken into account in Predictive Budget Allocation.
Does Smartly.io's Predictive Budget Allocation harm performance by resetting the learning phase?
No, there is no evidence of this. Like mentioned above, although bid and budget edits trigger the Learning Phase in the Ads Manager interface, the important part of the ad's learnings aren't lost. Facebook will re-learn quickly how fast it needs to spend in order to spend your budget optimally.
This has been fully confirmed by Facebook Delivery, a team that is responsible for bidding and budget products at Facebook.
Here are more reasons why you should not be worried about Predictive Budget Allocation triggering the Learning Phase:
- Thousands of Smartly.io customers are spending millions of dollars with the feature, every day. They are doing this because tests have confirmed that PBA improves performance.
- Only big budget changes reset the learning phase. Predictive Budget Allocation only makes big changes if there is a strong reason to do so — if that is the case, there is such a big need that it's better for performance to make the change than not to make it.
- Facebook engineers have confirmed to us that the budget changes made by Predictive Budget Allocation aren't harmful to performance.
- The learning phase indication in Ads Manager is a simplified presentation of Facebook's complex optimization engine. Bid and budget pacing is only a small part of one of many levels of optimization. For simplicity, Learning Phase is just "on" or "off", but in reality, budget changes are minor compared to creative or optimization goal changes.
Read more: Facebook help article about Learning Phase.