We’ve all experienced it before: sitting down in front of your computer to watch the latest news clip, TV episode, or cat video and being frustrated by how long it takes for the player to start playing an ad. It’s bad enough that there’s an ad taking up precious time, but it often feels like an eternity before the ad even starts playing. This delay has a large impact on how we feel watching videos online, and, at its worst, can cause us to spend our time elsewhere.
I’m not advocating for removing ads completely; we live in a world where ads are necessary, and arguably even beneficial to end users (how else can we expect to watch content for free?). However, the experience of watching ads is fraught with technical challenges, a lot of which result in viewer frustration. Until this point, it’s been difficult to measure an ad’s impact on viewer experience, and a focus of ours here at Mux is to help our customers provide their viewers the best experience possible.
Before I dig into the metrics Mux provides to diagnose ad issues, a little bit of technical detail on the world of video advertising is in order. Feel free to skip ahead if you just want to read about how Mux is helping diagnose issues with ad playback.
Video advertising is an immensely complicated technology, with many different systems and components working together to decide which ad to deliver to which user, and then play back that ad.
This is going to be a relatively high-level overview of these systems, components and their interactions, in an attempt to make this comprehensible to most people. As such, some components may not be depicted here; only the most critical are included.
The above diagram shows the simplest client-side ad insertion architecture. The ad-adept of you will note the lack of redirects and waterfall requests, but I’ll get to those shortly. At its core, this workflow for client-side ad insertion includes the following steps:
preroll), at a specific timestamp during playback (
midroll), or at the end (
ad tag URL, and includes parameters passed from the client to help the ad server decide which ad to deliver back to the client.
While those steps may seem simple, let me assure you that they are not. There are many details that are not mentioned here. For example, inserting a different video seamlessly into the middle of another video is difficult anywhere, but some devices make this much harder than others.
In addition, exactly how the ad server decides which ad to return to the player could be the sole subject of a longer post than this one. Even taking those details for granted, you can start to see how inserting an ad into the playback for a video can start to introduce those frustrating delays.
However, it gets worse.
Remember above, where we were dealing in a world where the ad server just “knows” what ad to return to the client? Unfortunately, due to various business and technical reasons, this is not always the case. This leaves us in a world where both the client and ad server may make multiple requests to one or more ad servers, networks, or exchanges. The reasons for this are many, and may include (but are not limited to):
The end result is that the single request/single response model is not applicable for a large portion of video ad playback on the internet. Instead, the following scenarios may happen:
With all these possible ad requests, even if every one of them completes in milliseconds, you can see how the time can add up. And this is only to inform the player which ad asset to play. Once it knows which ad to play, it may take some time to retrieve enough of the ad to begin playback, further adding to the amount of time you sit waiting for anything to start playing.
I’m not going to go deep on this topic here, but I’d be remiss if I didn’t point out that what I laid out above does not cover everything that makes video ad delivery what it is today. For instance, there are many details about what an ad server does, as well as the difference between ad servers, networks, and exchanges.
One more important thing to call out: the above implicitly assumes that the ad in question is a simple video ad (like an MP4 or WebM file). For interactive ads, this is not the case. The IAB has defined a format called VPAID which, from a layman’s perspective, allows the ad to control the user’s experience on the video player. These VPAID ads may make their own requests for the video to play back, so it’s possible for you to get penalized twice in startup time.
Previously, Mux’s Quality of Experience metrics were what we called “ad aware.” This means that all of the scores and metrics were calculated in a way that ads would be taken into account, but that there were no ad-specific metrics. For instance, Video Startup Time (a.k.a. time-to-first-frame) was calculated as the time it takes from the first attempt of playback to the point when moving video was shown in the player, be that the actual content or the first preroll ad.
This methodology allowed us to provide our customers a holistic view of the overall experience, as we reported the amount of time the viewer waited with a loading spinner at the beginning of playback. This information is useful, for sure, but it fell short on actionability, which is another focus we have here at Mux. Imagine you ran preroll ads on some videos, but not on others; how could you tell exactly the impact your ads had on your viewer experience? There were ways to accomplish this (for instance, using different experiment names for ad vs no ad), but it was not straightforward.
Keeping the goal of actionability in mind, we chose to make our initial focus on preroll ads and, more specifically, how they affect startup times. In order to do this, we introduced three new metrics under the Startup Time category:
As a reminder, Video Startup Time is defined as the wall-clock time from when playback is initiated (either by the user or via auto-play) until the first moving frame of either content or ad. So, for a situation where a preroll ad is played:
Video Startup Preroll Request Time + Video Startup Preroll Load Time = Video Startup Time
In order to provide more context for these ad metrics (and all other metrics), we have also included four new dimensions:
These first ad metrics will enable people to determine the impact ads are having on their overall viewer experience, and guide them towards actions to improve their experience. However, there are many more places that we want to take our ad metrics, including (but not limited to):
Ads are a necessary part of video on the internet, and up until now, there has been a major lack of information about how ads affect the overall viewer experience. Our goal at Mux, as with all of our other metrics, is to provide actionable information on the various aspects of your video architecture, and we believe these metrics are a great start to empowering publishers to make data-driven decisions on how they approach ads.