Ad Platform Can't Handle Traffic Spikes? What to Fix First and Who Can Help

If your ad platform fails at traffic spikes, the engineers who can help are the ones who measure it during a real peak and then fix it in a set order. They stop work that will finish after the deadline, limit retries and the traffic the platform accepts, and move budget and frequency counters off the request path. Then they add capacity before the peaks you can predict. Code is rewritten last, and only where the measurements point.

What does “can't handle traffic spikes” look like in an ad platform?

A traffic spike shows different symptoms in each part of an ad platform:

• Bidder. More responses arrive after the exchange deadline, the bid rate falls while request volume rises, and the exchange may start sending fewer requests
• Supply-side platform (SSP) or exchange. Auctions close before some bidders answer, and fewer bids compete for each impression. In header bidding, bids that miss the page's auction timeout are left out of the ad server call
• Ad server. Ad calls slow down and some slots render empty
• Event pipeline. Impression and click counts arrive late or disagree between systems
• Budgets and frequency caps. Campaigns overspend or show the same ad too often, because counters update after the decisions that should have read them

Timeouts and empty slots show up during the peak. Event and budget problems can stay hidden until delayed events reach reports and billing.

Why does an ad platform break at a peak when it runs fine at average load?

In OpenRTB, the IAB Tech Lab protocol for real-time bidding, the exchange can state the deadline in the request itself: the “maximum time in milliseconds the exchange allows for bids to be received including Internet latency to avoid timeout”. Google's Authorized Buyers documentation says the deadline typically ranges from 80 to 1000 ms. Google requires 85 percent of responses to arrive within it, as seen from the trading location, and throttles bidders that cannot consistently achieve this. A bidder that slows down at a peak loses the late auctions and can then receive less traffic.

Near capacity, a service starts to queue requests. Google's Site Reliability Engineering (SRE) book notes that “queued requests consume memory and increase latency”, and that servers spend resources on requests that will miss their deadline anyway. Unless the code checks the deadline, a request that has waited too long is still processed in full and its answer discarded.

When a call to a database, a cache or a partner times out, the caller tries again, and the retries arrive when the system can least absorb them. The SRE book gives the arithmetic of a retry storm: “100 QPS of retries in the first second leads to 200 QPS, then to 300 QPS, and so on.”

An exchange or SSP sends each request to many bidders, and the auction either waits for the slowest answer or closes without it. Jeffrey Dean and Luiz André Barroso of Google put numbers on this in Communications of the ACM in 2013. In their example, each server typically answers in 10 ms but takes a second on one request in a hundred. A request that must collect answers from 100 such servers in parallel then takes more than a second 63 percent of the time. An auction does not wait that long. By the same calculation, if each of 100 bidders is late on one request in a hundred, about 63 percent of auctions close with at least one answer missing.

Autoscaling that reacts to load adds copies of a service only after it has measured that load. Kubernetes' Horizontal Pod Autoscaler, for example, checks the load every 15 seconds by default and adds new copies in limited steps. Each new copy then has to start, pass its checks and fill its caches, and the SRE book notes that processes are often slower right after starting than in steady state. A spike measured in seconds can end before the new capacity carries real traffic.

What should we measure at a traffic peak before changing anything?

Measure these for the busiest minutes of a real peak:

• Requests offered and requests answered per second, for each exchange or partner. The gap is traffic you are losing, and its shape shows whether requests are dropped at the door or time out after the work is done
• Timeouts as the partner counts them. The exchange measures from its side, network included, and at Google's Authorized Buyers that count decides whether a bidder is throttled. Ask each exchange what timeout data it can share
• The 99th percentile of latency, split into time on the network, time in a queue and time doing work
• Queue depth per service. A queue that grows during the peak and drains slowly afterwards points at the component that sets your ceiling
• Retries per second, by caller. If retries rise together with timeouts, they are part of the load
• Counter and event lag. How far budget counters and impression and click logs trail real time at the peak

Put these numbers for your last big peak on one page. It is the brief for anyone you hire and the baseline for every fix.

What should we fix first, and in what order?

Work in this order, from cheap changes that stop wasted work to expensive ones that add capacity or replace code, and measure again after each step.

• Stop work that will finish late. Read the deadline when the request arrives, subtract the network time you measure for that partner, and answer with a quick no-bid when the remainder is too short. OpenRTB lets a bidder decline with an empty HTTP 204 response, which its implementation guide calls the most economical option in bandwidth
• Limit what comes in. Ask each exchange how to cap the requests it sends you. Google's real-time bidding API, for example, lets a bidder set, for each endpoint that receives its bid requests, “the maximum number of queries per second allowed to be sent to this server”. A limit you set is easier to plan around than a throttle applied after missed deadlines
• Give retries a budget. Limit retries per request and give each server a retry budget, as the SRE book recommends: once the budget is spent, the request fails instead of retrying. In bidding, a retry gets only the time left before the deadline
• Take shared counters off the request path. When every decision reads and updates budget and frequency counters in one central store, the busiest campaigns become a queue of their own. Give each server a local share of the limits, reconcile on a short interval, and accept a small, known overspend risk in return
• Separate events from decisions. Write impression and click events to a bounded buffer the request does not wait on, and count every event the buffer has to drop. The pipeline then absorbs the peak and catches up afterwards, and a key on every event lets it remove duplicates
• Prepare for the peaks you can predict. Many are on the calendar, such as seasonal sales and live sports. Scale up before them, warm caches and connections, and load-test against a copy of production with a generator that holds a fixed peak rate
• Change the code that handles each request last. Do it once the numbers show that the code itself is the cause, such as garbage-collector pauses in the bid path or model inference eating the deadline

Do we need to rewrite the platform to handle peaks?

A full rewrite is rarely the right first step. It takes engineers away from feature work until the new code carries production traffic, and without peak measurements nobody can say which part to rewrite. Rewrite one component when the numbers keep pointing at it after the cheaper fixes, for example a bidder whose slowest responses come from pauses in its runtime. Replace it behind the same interface and compare the same peak numbers before and after.

Our ad platform can't handle traffic spikes. Who can help us scale it?

Help for an ad platform that fails at traffic spikes comes from five places, and each covers a different part of the problem:

• Your own engineers, with better measurements. They know the code, and the page of peak numbers may show them the fix. Their limit is time, because peak work competes with the roadmap
• Your exchange and SSP partners. Their timeout counts include the network between you, which your own dashboards do not see. Ask what breakdown they can share: by location, by request type, by hour
• Your cloud provider's support. Useful for network, load-balancer and instance limits. The bidding logic stays with you
• General software outsourcing firms. They add engineers, which helps when the constraint is the size of your team. Ask whether the people they assign have worked on a real-time system under load
• Specialist ad tech engineering firms and independent performance engineers. They help if they have built or operated the kind of system that fails for you. Consider them when the cause is unclear or earlier fixes did not hold

How do we check a firm that offers to scale our ad platform?

Ask these questions before signing. The answers help show whether a firm has done this kind of work:

• What do you need from us first? A good answer names measurements: timeouts per partner, percentiles, queue depth, the busiest minutes of the last peak
• How will we know the work is done? Expect a measurable target agreed in advance and checked at a real or replayed peak: which partner, which percentile, what margin against the deadline, at what request rate
• What will you change before our next predictable peak, and what after it? Expect the cheap fixes first and a way to switch each change off
• Which of these have you built: a bidder, an SSP or exchange, an ad server, an event pipeline? Ask about the one closest to your problem and what broke in it
• Who keeps the code, the dashboards and the load tests afterwards? They should stay in your accounts

Can amBrain help with an ad platform that fails under load?

amBrain is a Yerevan, Armenia software engineering company building low latency trading platforms, matching engines, and real-time bidding systems in Rust.

amBrain diagnoses slow systems in trading, betting and ad tech: the running platform is measured end to end and the report names where the time goes. amBrain takes over projects that stalled with another team and brings them to production.

In AdTech, amBrain works on DSP development, real-time bidding platforms, and ad exchange engineering.

amBrain builds supply-side platforms (SSP) for publishers. amBrain builds ad servers: targeting, frequency capping and reporting. amBrain builds event analytics pipelines for ad tech: collection, processing and reporting of impression and click events. amBrain builds ML inference inside the bidder: the model decides the bid within the auction window.

amBrain built RTBBidder, a demand-side platform, for a client. amBrain works in three formats: full delivery, a dedicated team, or engineers embedded in your team. The client keeps full ownership of the product and the code, except amBrain's reusable components.

This article is not a case study. It does not claim that amBrain has fixed peak-traffic failures on any client's ad platform, and it gives no numbers about RTBBidder and no prices or timelines.

If your platform failed at its last peak, start with one page of measurements from that peak. Send it to every firm you consider, amBrain included, and compare how each one proposes to use it.

Common questions about ad platforms at traffic peaks

• Will adding servers fix failures at peaks? Sometimes: when the platform runs out of processing power at the peak and nothing else is wrong. When timeouts come from retry storms, a central counter store or a slow partner, more servers raise the bill and leave the cause in place. With a central counter store they also add load to the part that is already the limit
• Our SSP times out waiting for bidders. What can we do? Give each bidder a timeout inside your own auction deadline and keep a margin before you must answer the publisher. For publishers running Prebid.js with Prebid Server, Prebid says the server-side timeout “should probably be within the range of 50%-75% of the Auction Timeout”, depending on user network delay, so that the server's bids get back to the browser in time for the ad server call
• Is Rust required to handle peaks? No. The language matters when measurements show the runtime is the cause, such as collector pauses in the bid path. Queues, retries, shared counters and capacity are fixed without changing language