Low-latency HLS (LL-HLS) is a notable improvement for live video on the internet. You can now use standard, highly scalable technologies to deliver low-latency live video. If your use case could benefit from latency closer to 5 seconds than 30 seconds, LL-HLS can make HLS an option where it wasn't previously. Latency is an important element of user experience, and there’s always been one question developers haven’t been able to accurately answer: What is the live stream latency in the real world, at scale?
To make this question easier to answer, for Mux Video AND for any developer streaming live HLS or LL-HLS, we’re launching the beta of a new Live Stream Latency metric in Mux Data, making it the first Quality of Experience solution to offer this metric for standard HLS protocols with support for a range of video players. This feature is available to all Mux Data users.
If you’re new to Mux, get started for free. If you’re using Mux Video, but not yet using Mux Data, our documentation will help you get set up.
The old way: “Download a clapperboard application . . .”
The old way of measuring live stream latency is cumbersome and provides a limited view of latency data. The “clapperboard” method can help you measure latency for one stream, or maybe a handful of streams, but you can't get a broad, accurate measure of the stream latency behavior at scale, in production. Additionally, you can’t drill down into the data to understand where there might be opportunities for improvement.
For example, AWS describes the following process to measure live latency:
What is live stream latency, and what can you learn by measuring it?
Latency refers to the time from when a camera captures an action in real life to when viewers of a live stream see it happen on their screen. It’s an important component of the viewer experience for events like live sports, which rely on shared viewing experiences across multiple mediums, like internet video and cable broadcast; and for live experiences like weddings, where audience interaction is paramount.
This metric helps answer some important questions about your streaming operations and the live user experience:
- How much latency are live stream viewers experiencing?
- If I change something on my platform, how is the stream latency affected?
- How does latency correlate with user engagement (watch time, etc.)?
- Are there global locations or networks where the stream has high latency?
- Are all the players and devices I'm using able to play the stream with the latency I expect?
How do we monitor latency?
When you broadcast a live stream, you can specify the time as the event is happening. In HLS, you can use the EXT-X-PROGRAM-DATE-TIME tag in the rendition manifest to denote the time of the stream from when it broadcast. For example, you would see tags like this in the manifest:
...
#EXT-X-PROGRAM-DATE-TIME:2021-06-28T17:53:25.533+00:00
#EXTINF:2,
https://chunk-gce-us-east1-production.cfcdn.mux.com/v1/chunk/3aJUOua6jsMHYybcqXRBpcXH82aCYXTu02TPTKHzIokndAPmz300ZThlCZbeNAy1t73003iytFZNJdjcvjTsOrCVTaGZgQ9J00uU/0.m4s?skid=default&signature=NjBmMjFkODBfOWJkMzMyMTc5YzgwY2VmMTdlYzIwODgzZGI2NWFiMThiM2U1NDM0NzM0NDZhMmQwOThhZmI0NDQ5OWY5N2VmMA==
...This specifies that the start of the segment was captured or encoded (depending on where in the video streaming pipeline the PDT tags are inserted) on June 28, 2021, at 5:53:25.533 PM GMT.
It’s important to note that the times reflected in the PDT tags can be derived in a number of ways. The times can be actual wall clock times from the initial camera capture, or they could instead reflect the time when the frames were transcoded for streaming. Mux Video inserts PDT tags when the video is ingested for streaming. Because of this behavior, you should expect the latency measured for Mux Video streams to be around 1 second lower than the actual glass-to-glass latency. Some other platforms behave similarly and would also omit the time before ingest time from latency. For each streaming platform you use, you should assess at what point during the streaming pipeline the PDT tags get inserted so you know what is being measured and can take that into consideration.
Using the time from the PDT tags, the player can tell us the current time within the stream. The metric compares that time with the wall clock time to see how far behind the video is streaming, with some corrections for clock skew and round-trip time to improve accuracy. Because we’re trying to understand the latency of the stream itself, rather than capturing the latency of viewers who want to, for example, pause and watch the event later, we’ve designed the metric to exclude high latencies from viewers who are not trying to play at the live edge.
Show me the latency!
You will find live stream latency under the Video Quality section in the Metrics page, and it can be reported across all of the dimensions we normally support in Mux Data’s historical metrics. Latency can be aggregated and filtered by country, player, video title/id, device, browser, and many other variables. For example, you can filter by CDN to see which network provides the best performance for your viewers. You can also focus on viewers that are experiencing the highest latency by reporting on the median or the 95th percentile of performance.
In this next example, we can see a view of the live stream latency broken down by geography. Looking at this data, it becomes more clear where the latency is as expected and where you should focus on improving your experience.
What do I need in order to collect this data?
To collect this data, you'll need to use Mux Data with your HLS live streams, insert EXT-X-PROGAM-DATE-TIME tags in your manifests, and have an up-to-date SDK release. The following Mux Data SDKs are able to collect stream latency metrics:
- HLS.js (Mux SDK 4.4.3+)
- Video.js
- iOS AVPlayer (Mux SDK 2.7.0+)
- Android ExoPlayer (Mux SDK 2.6.0+)
- THEOPlayer Web
- JWPlayer Web
- Kaltura Web
The video ecosystem is constantly evolving and we want to get your feedback on what you need to be successful. We’ll continue to build out support for more players as they allow the retrieval of PDT data. Let us know if we’re missing your favorite player. Additionally, the Live Stream Latency metric currently supports both HLS and LL-HLS, but let us know if you would like to collect this data for DASH streams as well.
