Published on February 2, 2024 (3 months ago)

Building Efficient Video Workflows: A Chat with Joel Hooks (egghead.io Founder)

Dave Kiss
By Dave Kiss14 min readCustomers

Joel Hooks: The year is 1982 and I'm in third grade and they had a lab with the TRS-80 computer and it was a big clunky thing. There was this programming language called Logo, which actually still exists today. It's a derivative of Lisp if you really get into it. But at the time I was like you had a little, what they call, the turtle and the turtle would draw for you.

And you'd be like, forward 60 right 90 forward 60 right 90. And you'd have a square. And then we graph like our name and then when you draw these pictures and it was fascinating. I was young and, it was... Logo is actually a wonderful language.

It's kind of slept on these days. But as a like a learning language, it's really fantastic and the tactile feedback of telling this little turtle and then watching it do what you tell it to do was awesome. Like it was one of the like first technical victories over the computer that I ever had and it really stuck with me. I was like, oh, I like this stuff.

Dave Kiss: This is Joel. Joel is a full stack developer who founded egghead.io a successful video course site for web developers and he also works as a consultant over at badass.dev.

In this conversation, we picked Joel's brain on building efficient video workflows his preferred tech stack and what he likes about working at an early stage start-up after years of building his own successful businesses. All right, here we go.

Joel Hooks: I've built my own video infrastructure. That's something I've, I've, done in my life that I'm also done with. Several times, I've built my own video player for the browser and that's a miserable slog. I'm in the video business though. So it's like, oh, I want to do this. Which brings me, like, why we're here in this conversation because now I don't do that anymore because there's somebody out there that can just do that for me. So, thank you for that, Mux.

Dave Kiss: Can you share more about, like, maybe that workload that that you've realized within Mux and how that kind of takes it off your shoulders and some of the nightmare maybe that you ran into before with that infrastructure approach?

Joel Hooks: Delivering the actual videos themselves to people across the globe. Because there's a lot of underserved markets and when you're talking about that much bandwidth and you're trying to do this. So, you know, it used to be that we do progressive video which means you have three video files, you know, like mp4 format or whatever and they'd be in certain sizes and based on that they'd switch videos. And then at some point we wanted to get into adaptive streaming. Which means that you have a bunch of tiny pieces of videos that can be switched to different bit rates. That really helped out that helped with our global distribution but it's still really, really difficult.

And there's this wonderful human Christian Pillsbury who now actually works at Mux. And I was like, Christian, I just, like, I'm like, people every single day I have people in my inbox that aren't in San Francisco or the West Coast of the US that they're having a miserable experience. And how we do this? And he's like well, it's, you know, it's like some sort of caching. Its Edge. We're using AWS. We have S3 buckets with videos in them and you know, you're gonna have to learn all this stuff.

So I have to spend a week of my life learning about caching and configuration and configuring AWS and IAMs and how to like set up my cache. And instead of working on my product, I have to, like, work on core infrastructure. So when we started building our new platform last year I was like I'm not doing that anymore if I can kind of avoid AWS for our platform I'm gonna do that.

The other big thing was was the the player is actually a big deal. Do you all use Real Player? Have you used that? Are you old enough to have?

Dave Kiss: I'm fortunate or unfortunate. I always swore my music sounded better with Real Player for some reason. But yeah, I don't wish that upon very many people.

Joel Hooks: Like yeah, it's like we went through Flash and we went through Real Player and we went through, you know, Winamp was cool. But like whatever. Like, like the way we have watched video on the internet has evolved a lot over the last couple of decades.

But the player experience is, is pretty difficult. And it was for a long time, over the last 10 years anyway, where you'd use a proprietary player, right? Like, building a robust media player that's both like accessible works how expected and then it's something that you can you can actually like build and grow, it's it's a, it's a real challenge. And everybody benchmarks you against what Google has done with YouTube recently and Mux's player, in particular, has seen like leaps and bounds of improvement with Media Chrome.

And Media Chrome is the underlying framework that's built on web components and it's an open-source project. Then Mux Player is basically the commercial version that works if you're a Mux customer. But like Media Chrome itself is like the first time I looked at web components and went like, oh I can see why you would actually want to use a web component every now and then.

And then it's like also one of the like three GitHub repositories that I actually subscribe to because I've spent so much time building a player on what I've watched the evolution and watched the team and you know like watch Christian and the other folks in there just like really like get into the weeds. I don't even have to think about the player anymore. And having gone through, like, the last four years having built three iterations of of a player. Yeah, it's a marked life improvement for me. Quality of life, plus, plus.

Dave Kiss: All right, so I want to switch gears towards a specific project that you've been working on. As far as I understand, this was for a specific course that another developer, creator, was working on and your solution was to try to like basically simplify again. Like, there's a lot of different moving parts here. You have the tools like Inngest at your disposal and Mux at your disposal. Maybe you could just walk through it a little bit. What the scenario was like. Why this stack ended up being the right fit for you for this project.

Joel Hooks: So the Inngest stuff is pretty new this year. We, so, we've had egghead.io and it was a Rails app so it's a serverful app. And over the last year and a half or so I decided for new projects I didn't want to use the Rails API's it was kind of a mistake. We use Next.js. I wanted to embrace serverless and everything lives in a mono repo. It's actually skillrecordings forward slash products on GitHub.

So we use visible source. I don't call it open source because there is no support implied or given. So, if you want to check it out you can check it out. But like we're not trying to like build an open source product that people can actually use. More, more, just keeping it open because it makes things easier.

Part of that was like, alright, every product is gonna get its own database. We use Planet Scale. Love Planet Scale. Every product gets its own Mux instance. And like, all of those, like, every one gets its Notion account. Every one, you know, like, gets this. We consolidate some things, like the Vercel account is all one stack, and there's a couple other aspects where we use shared infrastructure. But every one of these products looks the exact same, but gets its own slice of the infrastructure. So we are servicing clients, at the end of the day, that are the instructors of these courses and, you know, just to make it not as bad. So it's not, we're not quite to nice yet, but like, how do we how do we make it not as bad where they can actually self- service and don't need like API tokens and logins to various services and all that fun stuff to make it happen.

Media processing workflows are really a pain in the butt. Like, it is not super easy to, you know, like, Mux makes it pretty easy. There's a URL you can go to the web interface and you can actually upload stuff. But really it's like the best thing you can do is pitch a URL so it's like you need to get that the, the raw media file someplace and then I also want that in a backup. So if I ever have to rebuild, like some happens to Mux and I have to rebuild all our video infrastructure and use AWS. That would make me sad, but I need to be able to do that right.

Like so, you have to upload files, access those files, and then wait for them to process. So send it to a service provider like Mux where it gets transcoded for, for, streaming. We want transcripts. We want subtitles. We want to like send it through Open AI to get weird LLM suggestions that that might produce for us. Sometimes they're good. It's good at like tagging and stuff. So like we get all that done. All through that process, if you do not say, like, here is where this is at, this is doing this right now. It's 70% complete. 80%. Send notification. Do you like to be emailed? Do you want a Slack message? Do you want to be hit up on Discord? Do you want an SMS message.

Managing that asynchronous workflow is where, like, I have been saying lately, like, distributed systems are easy. As an asterisk on it. Because they're super difficult, but like it's actually now it's now it is actually easy. I can just make that happen. I can have an idea. Oh, man, wouldn't that be cool if that also notified us over here? [imitating typing] Done, right? Like, it's not like I had to go in and change it or re-figure it out. I know exactly where to go in the app, what to change, how to add that step. Error handling is kind of built into the whole thing. And then it just looks like TypeScript at the end of the day.

Dave Kiss: Speaking of Inngest, you actually have a new role over there. Was that a part of just being super stoked about the product or what was what was really the motivation behind that?

Joel Hooks: Right now is a very early company. But even at this early stage they have like a really good product that fills a gap in my my tech stack that was really difficult and something I've always wanted I love event-driven architectures and distributed systems. It's not a hat I was able to wear I would have to like staff up and have expertise in Kafka and AWS and Celery or whatever like alternative solutions you might choose. And I wasn't able to do it. I was never able to get it to a point where it would be actually reliable and I could run it and Inngest let me do that. And I was like I love this so much. It was a kind of a joke Tweet, I love this product so much that I'm considering their open DevRel position. And Dan Farrelly hit me up in the Twitter DMs with the with the old eyeball emojis. And it just kind of escalated from there and you know, it's like I wasn't looking and I wasn't thinking about it. I get to sit down and think about education from a different side. From, you know, like an early stage startup position, which is that's building something that's like truly useful and interesting.

Well, it's cool to me and probably the biggest difference is the code and configuration. It's all serverless, but it's in our app. So we use Next.js and all of this configuration is in our GitHub. It's in our Git repository and it's committed. And we we get to own that I get to see how the sausage is made now and what they're doing is incredibly complex. And, you know, like as I described it's like this infrastructure where you need a team of incredibly smart people to manage that. You know like building a product and early is really quite fascinating and, and it's good, it's an accelerator for me because I can, you know, like being an experience that kind of deep technical product growth that I just, you know, I wasn't, I wasn't getting in, in the businesses that I operate for.

Dave Kiss: One last question for you. Do you remember like a moment where you not only realized you were interested in the stuff that you were like this might be like harder than it looks?

Joel Hooks: No, actually I don't. I, I've never found the technology to be, I mean, it's it's challenging, daunting, I don't know what I don't know and you have to dig in and you don't know the answer. But to me that's always been the most fascinating aspect of it and the, maybe, even the most joyful aspect of it is the, the just, the, the puzzle it's like a big Rubik's Cube.

And having to deal with the human aspect of technology has always been like way more of a struggle for me. Like, I've, I've, I love to sink into impossible technical tasks and rise superior to the machine and curse it with my dominance when I'm finished with whatever it is I was trying to do. It really just boils down to like that's the joy of it. It is hard. I'm not saying that this technical stuff isn't hard.

But like that's, you know, like it's like looking up at Everest and, and, get into the top. Like that's like if you lose a couple fingers along the way, that's just part of the cost.

Dave Kiss: Joel, where can we find you on the internet?

Joel Hooks: So I spent a lot of time on X. And I actually play around with TikTok a little bit. I haven't posted a lot of you think there's a synthesizer that sits over my shoulder over here. So I make screeching noise and bad generative art and put that on TikTok a little bit. And then I, I, blog on joelhooks.com infrequently and badass.dev is kind of my my current personal project that I love quite a bit.

Dave Kiss: Sweet. Well, thanks for taking some time to chat with us.

Joel Hooks: Yeah, my pleasure.

Written By

Dave Kiss

Dave Kiss – Senior Community Engineering Lead

Fake programmer. Likes bikes, trees, and cookie dough custard. Collects wood scraps. Lots of spiders live in his basement.

Leave your wallet where it is

No credit card required to get started.