If you’ve been following our Launch Week Series, you’ve seen firsthand how busy things have been at Mux. I lead Engineering and I’m proud to say that this is no mirage — our changelog HAS been on fire lately. Shipping this fast is something we’ve worked to make great strides on in the last year. Let’s talk about what was slowing us down before and what we did to solve it, including a peek into how our Product and Engineering teams plan, prioritize, and execute. Video is hard, especially at scale, but as a startup it’s critical to maintain an engineering velocity that allows us to ship things that turn customers into fans. Let’s get at it.
Velocity is key
Our mission is to democratize video on the internet so our customers can build amazing experiences around it without worrying about the hard things that go into engineering video at scale. Believe me when I say that there are a lot of really hard things in video. However, being an abstraction over complexity isn’t enough — our customers need to innovate and we have to enable that. The first reason why velocity is so important: Mux is a product-led company and our business only moves as fast as Product and Engineering. We need to move fast to help our customers succeed.
The second reason is that we’re a startup. Speed is the lifeblood of startups. It’s the one strategic advantage we have over larger incumbents. Larger companies have access to more resources: engineers, money, sales organizations, customer bases, and time. A startup’s advantage is speed and adaptability.
The Iron Triangle of software engineering is a model of constraints/tradeoffs between speed, scope, and cost.
Managing cost is important because we have a finite amount of moolah (bummer!). Scope (features/functionality) is important because our customers need us to innovate, and that means an evolving product. Speed is the one strategic advantage we have.
This leaves us in pursuit of defying the constraints of software engineering in order to find ourselves at the center of the triangle. The software companies that can best strike the balance between speed, scope, and cost are the ones that will inevitably win out over the long run since they’ll be able to quickly innovate in a cost-sustainable way. It’s that easy, right?
Now, no matter the domain, every software engineering team faces these constraints, but, these constraints tend to be uniquely difficult when engineering video at scale.
- Video is complex and complexity is the bane of engineering velocity.
- Video is costly, it fundamentally takes a lot of resources to encode and deliver video
- Video is vast and perpetual — a scoping tar pit
A critical look at our process
When the pandemic hit and the world turned to video, Mux scaled to meet the demand. Our team and customer base was growing rapidly and we had more software to build than ever. Yay! But that rapid growth meant we were spending more time on-boarding new team members and shoring foundational product work to support scale, which impacted how consistently and quickly we were shipping. Not great. The urgency was there, but it started to feel chaotic. Urgency is great in a startup. Chaotic urgency is not.
Overall, we lacked consistency and focus which was paired with low accountability and transparency. The end result: we were moving too slow, which meant Product and Engineering slowed down the entire company. If we were moving slowly as a company, we were giving up our strategic advantage — speed and adaptability — which, if not corrected, would almost certainly impact growth.
The 6-month cycles
In attempts to fix things, we’d find ourselves implementing new structures and processes. And with every planning cycle, we talked about how this was going to be the one where we actually maintained our focus. Yet, at the end of every half, we still came up disappointed with a team that felt like they weren’t reaching their potential. Velocity continued to slow. Despite everyone in the company agreeing we wanted better focus, from the CEO down, we still struggled — how can that be?
We took a step back and began to dissect what was going on. We realized that lacking focus was a symptom, not a cause, so simply wanting better focus was not a fix.
At that time, our planning cycles were on a 6-month cadence. We’d spend 3-4 weeks planning, finalize the plans, then send the teams off to execute against those plans. Now, we didn’t go heads down writing code for 6-months — teams were operating within a 2-week agile sprint cadence and planning locally every two weeks. While Product and Engineering were talking regularly we didn’t set regular times to zoom out of the details and give feedback on what’s working and what’s not in the overall plan.
We were “doing agile”, so that meant we should be able to make changes quickly, right? It’s right there in the name! Well, sort of, but not in the ways we’d want.
Strategic change is adaptive, and non-strategic change is reactive. Unfortunately, the majority of the time that we’d make changes, they were the latter. That’s because our execution was too far disconnected from our 6-month strategy, leading to individual teams making decisions in isolation: Is this a distraction, or is this a strategic change? Should we disrupt plans to do this now? What is the priority compared to our 6-month plans? When enough of this flows in, you end up in a game of whack-a-mole: whatever topic is the most “pressing” at the moment gets the focus, leaving you in a constant state of reactivity. When reactivity is your default, you’ll be lucky to hit your strategic goals with any sort of certainty and it’s not exactly fun as you try to stay afloat.
We were out of rhythm with the pace of the business and it meant we were too often falling on the side of reactive, not adaptive. Our previous attempts at focus tried to get the business to slow down the pace of change and this simply wasn’t feasible. Revisiting the diagram, we unlocked the root of our velocity problems — Our Cadence != Rate of Change in the business.
Finding the right cadence
Beginning in June of 2023, Product and Engineering made a major overhaul to how we would plan and execute. At the core of this change, we moved to 6-week planning and execution cycles (still maintaining our 2-week sprints inside). If you’re wondering how we landed on six weeks, we looked back at our previous execution cycles and it felt like it was at about the 6-week point that our focus started to waver — this was indicative of the natural cadence of the business. The key to this change was not getting the exact right number of weeks, but that we closely aligned planning and execution on a cadence that was much shorter than the previous 6-month interval but long enough for us to ship meaty features within a cycle.
How does this work in practice?
Let’s step through some of the structure and process to see what it looks like in and around our 6-week cycles. We’ll start outward and zoom in.
At a company level, we still plan at the start and midpoint of the year. These are our halves — we creatively call them H1 and H2. Establishing company priorities gives us directional alignment with the biggest priorities we need to solve as a business in the coming six months. These priorities begin at the top, with full representation from our Marketing, Revenue, BizOps, and Product and Engineering functions.
Once we establish company priorities, Product and Engineering begin the process of outcome planning.
Outcome Planning
Outcoming planning involves establishing product outcomes necessary to hit the 6-month company priorities. The goal is to validate, at a high level, that the company priorities are achievable and establish the target outcomes necessary for accomplishing them. In our outcome plans, we specifically ask that we do not overspecify. The goal is not to lock into a list of projects — it’s okay and expected to evolve these. The outcome plans inform our 6-week cycles, but they are malleable. Here’s an example from our H1’24 outcome planning.
From there, we move into a series of 6-week cycles until we complete the half.
Cycles — the heart of it all
Within a 6-week cycle, we plan, prioritize, execute, and demo. Each cycle is followed by a buffer week, which provides a week of rest, bug fixing, and planning for the next cycle. Product and Engineering identify the top two priorities in the cycle. We fully staff the top two priorities and fill in additional work around those, as capacity allows. We aim to complete 100% of the cycle scope for the top two priorities. Each initiative is scoped with the goal of completing a demo-able milestone by the end of the cycle, if not complete the entire initiative. At the beginning of the cycle, we communicate the cycle goals to the entire company. At the mid-point and end-of-cycle we have an All-Hands we call “Build & Tell”, where the teams review the cycle progress, cycle stats, and most importantly demo and celebrate their work.
We track and report on a few cycle stats — our Say/Do Ratio, which measures how much of our plan we achieved, and an unplanned ratio, which identifies the number of unplanned epics brought into the cycle. We strive for a 100% Say/Do Ratio and 0% Unplanned Ratio. These get reported cycle-over-cycle as a lagging indicator of how well we are executing against our plans.
To track initiatives across all workstreams, we have a Program Overview project board with all initiatives in the cycle. Each initiative gets a weekly status update and a red/yellow/green trending indicator. Anyone in the company can track progress, and the weekly status updates give us an opportunity to celebrate progress, align dependent work streams, and course-correct if/when things fall off track.
Once we’ve completed a cycle, we rinse and repeat. As we plan the next cycle, we coordinate with the 6-month outcome plans, evaluate new priorities or information from the business, and adapt as necessary. A big advantage of the shorter cycle time is that at any point we are six weeks or less from prioritizing a new ask, greatly reducing the amount of unplanned work we’ve had to absorb – significantly increasing focus.
Since implementing these changes, our feature velocity has increased by over 50%, from two years prior, and the complexity of the features has also increased — launching some major product upgrades like 4K, auto-generated captions, SRT, Metrics Filtering, and Cold Storage.
Keep moving to the center
Our Product and Engineering team will always be looking to conquer that Iron Triangle. As we look ahead, we’re excited by how the momentum is carrying us. We’re building (and maintaining!) a lot of amazing, innovative things for our customers, and we’re doing it with high velocity. If you use Mux, you can be certain of one thing — you can get back to working on your product (and sleeping) while we handle the hard things that go into engineering video at scale.
