Building the World’s Greatest Recommender System Part 20: Pacing Ourselves
Imagine a race with a finish line unknown to the runners. In their efforts to win such a race, competitive runners would sprint violently with as much energy as possible to beat the other runners - the finish line could always be a footstep away. This race is a real-time ads recommendation auction without the notion of budget pacing.?
Many platforms generally employ a Generalized Second Price (GSP) auction mechanism. In a GSP auction, ads are ranked based on bids, and the highest bidder wins but pays the price bid by the second-highest bidder. While this approach maximizes revenue per impression if the future number of impressions is unknown, it introduces issues for advertisers’ finite advertising budgets. Without knowing any better, an ads auction system would enable ads to voraciously compete against one another for an opportunity to be shown as impressions in the first few auctions of the day, not knowing that there would be countless other auctions throughout the rest of the day. Taken to the extreme, this competitive bidding would drive up prices for a thousand ads impressions (CPM) to $1,000 or more in the early auctions, using up the advertising budgets of most advertisers on these pricey advertisement impressions, and the few advertisers with some budget left would have the opportunity to show ads for pennies on the dollar (or on the CPM). This would result in an incredible disparity in advertising outcomes, and the many advertisers whose budgets were used up during the period of pricey advertisement impressions would not have many impressions after spending their entire budget. Receiving relatively few impressions from users would lead to fewer advertisement-driven (and advertisement-attributed) sales and a poor return on advertisement spending. As noted in a previous post, poor return on advertisement spending, over the long run, would cause profit-seeking advertisers to stop spending on a platform that gave them poor financial returns.
Consequently, budget pacing algorithms are essential to optimizing ad delivery for the benefit of both advertisers and the platform. A budget pacing algorithm is designed to distribute ad impressions more evenly throughout the day. The system would forecast traffic patterns and allocate budget proportionally, ensuring a steadier ad spend rate and prevent early budget exhaustion.
An example of a well-thought-out budget pacing system is that of LinkedIn ads. The system relies upon a combination of traffic forecasting, coupled with real-time monitoring and updating to adjust advertisement bids to effectively pace budget spending. If we break up the day into numerous discrete chunks, of say a minute, we can aim to keep spending consistent with traffic for each chunk of time, and in so doing, keep spending reasonably paced throughout the day.
Formally, we can phrase this as the allocated budget at a time t, known as a_it, should be proportional to (in line with) traffic up to time t, expressed as f_i,t, divided by total expected traffic throughout the day, expressed as f_i,T, multiplied by the budget. In other words, the fraction of budget spent should be in line with the proportion of total forecasted traffic that day that has been served.
Traffic forecasting is key, as by predicting the number of eligible impressions for each campaign throughout the day, the pacing system can create an allocation plan that aligns spending with expected traffic. One approach to generating these forecasts, used by LinkedIn (but also not unfamiliar to other platforms like Meta), is storing historic traffic data in a Hadoop Distributed File System (HDFS), which enables easy analytical querying through a data warehousing system like Hive. Thus, we can get traffic forecasts from Hive queries on historic traffic data.
Regarding real-time monitoring and updating, it is advised to start new advertising campaigns with a conservative auction participation rate, as LinkedIn’s pacing system does, as this gives the system time to adjust and learn optimal pacing for each campaign. The pacing system then should continuously monitor the spend rate of each campaign. An approach to updating the spend rate is to use a change data capture system like LinkedIn’s Databus, in which budget updates to a database are sent through a queue to a pacing module. This can use the publisher-subscriber paradigm, with the change data capture system “publishing” the updates to the “subscriber” pacing module. With the help of the impression (supply) forecasts, the pacing module can set the pass through rate (PTR), or participation rate in ads auctions. For example, if a campaign is spending faster than planned, it should be temporarily throttled, reducing its participation rate in auctions until it aligns with the forecasted spend rate. Other ads recommendation systems have been known to adjust the bids themselves with a multiplier based on pacing. However, it has been noted, as by LinkedIn’s engineers, that problems arise when the paced bid is too low, particularly, if a platform seeks to guarantee a minimum price for an ad impression in an auction, known as a reserve price. Furthermore, controlling auction participation is beneficial at scale, as it enables a reduction in the potential ads to rank via machine learning models, lowering computational cost.
The implementation of budget pacing leads to significant improvements in campaign performance and platform revenue. Campaigns remain active longer, increasing the chance to reach a broader audience throughout the day. Advertisers achieve more unique impressions per spend, indicating a wider reach within their target audience. By maintaining relatively even competition throughout the day, both advertisers and the platform see more even advertisement spending. This improves advertiser satisfaction due to less likelihood of their budget being spent on the most expensive advertisement auctions of the day, and it improves revenue stability for the platform, as advertisement spending is less likely to fluctuate drastically due to oversubscribed and undersubscribed auctions. Moreover, pacing and forecasts also decrease the instances of showing ads after an advertiser’s budget has already been spent, known as over-delivery. Given that it would seem sensible for platforms to maintain the trust of advertisers by not billing them more than their allocated budgets, these ad impressions from over-delivery lead to lost monetization opportunities, so reductions in over-delivery directly enable more billable ad impressions.
It should be noted that most changes in recommender systems can be measured by rigorous A/B testing measuring the differences for two different groups of users using different versions of the recommender systems. However, given that the budget pacing algorithm is key to the recommender system itself, such side-by-side A/B testing often cannot be done, as was the case for LinkedIn ads’ bidding experiments. Instead, in such cases, alternating the pacing algorithm on and off each day over several weeks, is needed to isolate the algorithm’s effects and quantify the significant benefits of adding a pacing system.?
领英推荐
Pacing systems show clear benefits to advertisers: longer-lasting campaigns, broader audience reach, and improved return on ads spend. For platforms, maintaining even competition to stabilize ads spending throughout the day is critical. Consequently, budget pacing is crucial to effective ads recommendation.
If you benefited from this post, please share so it can help others.
Sources (All Content Can Be Found In Publicly Available Information; No Secrets):