Case study

Improving lives through learning

Since 2010, Udemy has been on a mission to improve lives through learning. Today, the online course provider is the world’s largest learning marketplace, enabling over 40 million learners access to 155,000 video courses taught by 70,000 expert instructors on numerous topics. These videos help individual learners, businesses, and governments prepare for whatever comes next.

While Udemy has been democratizing education for over a decade, the platform saw significant growth in the wake of COVID-19. Course enrollments surged within the first three months of shelter-in-place alone, as learners across the globe took to the platform to hone or learn new skills that would give them a competitive advantage in an uncertain job market.

As Udemy’s learner base scaled, the company committed to providing deeper, more immersive experiences to learners, and a key focus was the quality of the video delivery experience they offered.

Doubling down on video experience

Udemy is constantly striving to improve the learning experience for its customers. Even before the pandemic, the team was acutely aware that as an online learning platform, video was core to their business.

When they recently added a new encoding format and frontend media player, they wanted to ensure the adaptive algorithms were working by delivering the best quality video experience for each learner’s own bandwidth. This meant performing robust QA with a third-party vendor, and leveraging Mux to monitor everything including startup and load times, upscales and downscales, rebuffering percentages, and manage all of this across various geographies and browsers.

In one testing instance, Mux identified an almost 100% failure on Internet Explorer Windows 10. After seeing the error data, Udemy had a fix released within a day.

The Udemy team turned to Mux Data to better understand the customer experience with the video. The deep analytics capabilities enabled Udemy to monitor key metrics like performance and latency, but also to filter data to identify differences across geographies, video formats, time zones, and devices.

Continuous improvement

Serving a consumer market means that user experience issues are often surfaced through app reviews, which Udemy is constantly monitoring.

When the Udemy team recognized their rating in the Android app store had taken a dip, they wanted to uncover the problem and quickly resolve it. Through Mux Data, they identified that video delivery scores for Android devices were significantly lower than other devices. Believing there was a meaningful correlation between the Mux scores and the app reviews, the Udemy engineering team was tasked with improving the Mux score for Android video delivery. Following the development work, Udemy’s Mux scores increased 33%--and as expected, Android reviews saw a positive increase as well.

With digital natives increasingly turning to the Udemy platform to stay current in their skills, it’s more important than ever to ensure that the virtual learning experience is seamless across delivery mechanisms and devices. With quality of experience video analytics, Udemy is able to stay ahead of issues and keep learners engaged.