It’s Tricky: Storyboards and Trick Play

Today, storyboards are available to all of our customers currently developing with Mux Video!

TL;DR

• A storyboard is a collection of thumbnails from regular time intervals throughout a video that are combined into a single image in a grid layout.
• Trick Play is a player feature that allows you to see these thumbnails as you hover over the scrub bar of your video (psst - if you’re on Roku and your content is longer than 15 minutes, you’ll probably need this).
• Storyboards can enable UGC creators to set a poster image with a screen capture from their content’s thumbnails, and give video editors access to specific moments in their content without having to view the full video.

Storyboards work just like our thumbnails and animated gifs, but for more implementation details check out the full documentation. Read on for some more context on these features and how we made storyboards fast.

What are storyboards?

In a traditional video pre-production process, a storyboard is a sketch of the important elements of a scene, tiled across a page. With the Mux API, you can now obtain a collage of your video - a storyboard - with thumbnails spaced over regular intervals. If you’re familiar with sprites in other contexts, such as CSS sprites or video games, the concept is the same!

Traditional Story Board

Mux Storyboard

How to get a storyboard using Mux

Getting a storyboard is as easy as getting any other thumbnail via the Mux image API:

https://image.mux.com/{playback-id}/storyboard.jpg

https://image.mux.com/{playback-id}/storyboard.png

Very cool, very easy. But what are some of the use cases for storyboards?

Use Cases for Storyboards

Content review

One of the most challenging aspects of publishing uploads from a community or “user generated content” is to quickly screen the content at scale. Storyboards can help a moderation team quickly review videos at a glance without having to open a video to play it through completely. We’re already using this feature for content moderation on stream.new:

Trick play

Also known as “trick mode” or “seek previews” . Trick play is a player feature that shows a thumbnail preview of the video around the seek bar.

What do storyboards have to do with trick play? As we previously touched on, web developers might be familiar with image sprites, and many trick play implementations build on that concept: the player can make a single request for a larger image, and display thumbnails of smaller slices within that image as necessary. This is much faster and more efficient than downloading dozens of individual thumbnails.

Trick Play Formats

Different players support different methods for trick play, so Mux has a few trick play formats available today.

VTT

Some players cleverly implement trick play using VTT “subtitles”. Our VTT implementation works with the following:

• JW Player
• THEO Player
• Video.js
• Bitmovin
• Flowplayer

Access the storyboard VTT with the following URL:

https://image.mux.com/{playback-id}/storyboard.vtt

JSON

We wanted to give developers something more accessible in the absence of a VTT parser, so a JSON format for storyboard coordinates is also available. See the reference docs along with a demo of how storyboards+JSON could be incorporated into an application.

Access the storyboard JSON with the following URL:

https://image.mux.com/{playback-id}/storyboard.json

Image media playlists

Roku has its own trick play format using a special type of image playlist which contains a storyboard. To use Roku trick play with Mux, append a roku_trick_play=true query string parameter to the HLS manifest URL eg:

https://stream.mux.com/{playback-id}.m3u8?roku_trick_play=true

Try it out

Storyboards with Mux Video are quick, easy to incorporate into your projects, and best of all - free for our Mux Video developer community. We are excited to see what you can do with it. Try it out and let us know how you like it.