Yeah. All right. So that's how Streamplace VOD works. I hope you all enjoyed the presentation. My name's Eli Mallon. I'm the CEO and founder of Streamplace. I don't care about my Caribbean or update version. So the the crux of this talk was uh if you are going to make a system that has both VOD and live. Can that be muted at all? The uh you should probably start with live. Live video is more difficult than VOD. Uh it is much more difficult, it's much easier to generalize live video to VOD than it is to go the opposite direction.
Um so we started out with live. We're pretty happy with that. Um and uh, you know, knock on wood. Of course, as soon as I walked out of the other room, people started reporting playback issues over there, but overall everything's been going really well uh uh at all of this. Um I'll start by sort of uh explaining how Streamplace uh a quick explanation of how Streamplace Live works because I've talked about this a lot before, and then we'll talk about how we're adapting that for uh a decentralized VOD primitive. What does it mean to have uh something like uh, you know, what would a decentralized YouTube look like?
What things do you need in order to make that happen? So uh yeah, this is just about Streamplace. There's been three Streamplace third party clients created since the beginning of this conference. Uh so I I don't know if that reflects poorly on our front end or if it just represents the you know the entrepreneurial spirit of the of the atproto community. But um this one's from Jack.xyz, and it shows all of the all of the streams at once, really uh given the Streamplace servers the maximum workout to to give everybody uh what they're looking for there.
Um before we do anything else, hey, I found this uh I found this MP4 file on the ground. Let's see what it is. You can all imagine the great noises Trabio is making here. What's gonna what's gonna happen? What's gonna happen? Uh uh uh. And wait for it. Yeah, we got um, yeah. Uh so that MP4 file I found on the ground turned out to be my glorious victory over Trabio. So how do I know that? How do I know it's it's Eli's live stream that happened there? Um and that has to do with uh Streamplaces uh signing format.
So uh we needed to. So so decentralized VOD is something that's kind of existed before, right? You could argue, this will be a popular opinion. You could argue video NFTs are kind of like a decentralized VOD, right? Where you've got an IPFS reference to something that's like out there and not controlled by any one party. Um lots of other like IPFS-based, um like there's like dat-based live streaming that's that's that uh maybe maybe falls into that. But for the most part, we needed to invent what a decentralized live stream is. So what we did was we took uh the first thing that happens is you generate a signing key, same as the one with kind that's used to uh secure your atproto repository.
Um we then take that signing key. You then uh configure your your OBS instance just like we're doing here to to stream in to Streamplace. Um you actually provide it with that signing key so it has the capability of signing on your behalf if you really care about your key custody, you should run Streamplace locally uh and then syndicate it out to a server. Uh then uh this is basically every live stream you've ever watched if you haven't messed with video infrastructure before is just uh a bunch of short video files that are thrown into your face in sequence and it provides the illusion of a live stream.
So that's how we do it. Uh each uh MP4 file comes in, we give each one of them an embedded signature using technology from the C2PA, the coalition for content provenance and authenticity. Um cool format. Uh it provides a mechanism for giving you an MP4 file that contains an embedded signature over a hash of all of that mp4 file except for that part of the signature. That's really tricky to get right. And that's the one of the things I really like about the C2PA's tooling is you've got this um I a lot of uh decentralized media projects have uh external signatures.
So you might have uh this actually including like Bluesky videos, right? How do you know it's my Bluesky video? Uh it's because there's a blob out there that is referenced by a record in my repository and my repository has a signature on the root commit. So that's how you know it's mine, right? What I like about this format is I can have this is the uh the mp4 file on the ground example I just gave, right? You can have arbitrary mp4 file on your computer and all of that signing and provenance data is intact and embedded in the MP4 file.
So I like that a lot. It's something we really wanted to preserve as we move from live to VOD. And lots of very cool metadata in here. We're not doing too much with this mechanism yet. I spent a long time talking to Trezi yesterday about embedding this part of the embedded metadata here should be that I'm playing Silksong, right? And then if you care about Silksong, you can watch that live stream and get that data uh from like pretty much any direction, right? Like you can download it from sketchiest website in the world.ru and still be confident that you're looking at the right thing, right?
And then I'm playing Silksong. Those, yeah, the the that that's this example. Uh I have uh yeah, we've made Streamplace as easy to run as possible. So you can run your own Streamplace nodes, syndicate our uh syndicate my streams, they get copied over in this sort of trustless, unauthenticated way, um, and then you see it over there. So uh the the thinking here is we probably won't beat Twitch by making a bigger data center or buying more bandwidth than Amazon, but we might be able to beat them by following the example of BitTorrent, right? That you can have lots of decentralized infrastructure that's replicating all of this, but through the magic of digital signatures, you know that the video is authentic.
So what's the problem? Uh this isn't very convenient. Uh so when I say slice the video up into one second MP4 files, I'm being very literal about that. If you run Streamplace on your computer, you can navigate to this and see lots and lots and lots of one second MP4 files. And when the computer is processing them all and feeding to use a live stream, that's okay. Afterward, when you want to work with this as a VOD, it's a pain in the ass. So uh we wanted this was one of the design goals of VOD for us is how do we um how do we handle that?
And unfortunately, because we used regular MP4 files, uh they're non-concatenatable. Uh you can't take two MP4 files, squish all the bytes together, and get valid output, right? That's just not now MP4 works. It's not it's not valid data there. Uh so you'd have to mux it and the and the signature that you get there wouldn't match the wouldn't match the input, and all of a sudden you've broken this entire cool BitTorrenty provenance mechanism that I was just describing. So uh this is the other, this is the other reason we were already looking at forking um C2PA.
Uh so here we have a list of signing algorithms. Uh as you can see here, we have uh the good and virtuous uh ECDSA signature over uh uh a SHA 256 hash over the uh over the P256 curve, but not over the uh SCCP256 K1 curve, the evil shady curve that's used inside BitTorrent and Ethereum and atproto, right? Uh so I think my theory here is that somebody in Adobe had to like sell their their uh the some exec that they weren't making blockchain software here, so we didn't do that kind of signing. We only did this kind of signing.
So technically for as long as Streamplace has existed, it hasn't been C2PA compatible. Um and I had been looking for a while to formalize that. So uh oh yeah, this is the other. So um this is the other uh so Adobe's out there right now lobbying, especially European governments uh and uh the state of California to require um provenance data on everything that you post on social media. This is the reason for the CTPA's existence, getting us ready for a world filled with deepfakes. This is not inherently a bad idea. Uh you know, once anybody can deep fake anything, how do you ever trust what you're looking at?
No one technical mechanism can solve that problem. But one way you could be more confident is if there was provenance data embedded in the in the live stream, right? Where it says, oh, this video is taken from Eli's camera and then edited using this software and and makes it way it makes its way everywhere, right? But the reason Adobe is doing this lobbying is because they want to try and do some regulatory capture, right? And so they want to sell governments these expensive uh creative cloud subscriptions and be like the only software that you could possibly use in order to do all of that stuff.
Um also the part I think is really funny is they want you to go pay like a a CA, like 107 dollars a month for a signing certificate. Uh like let's encrypt and everything never happened. And of course, like in the atproto and decentralized web communities, we have you know DIDs and totally other stacks of uh of how we would handle some of this stuff, right? Um how we would handle key custody, how we would handle authenticity, that kind of thing. PLC directory's got its own, got its own sort of take on everything. So um yeah, some good ideas needs to be modernized and not try and pay these vendors uh a bunch of money for for certificates.
So um I said all of these things out loud a lot, and uh uh the folks, the good folks at the IPFS Foundation and the Dazzle project were very, very receptive to it. So um they've given us a grant. Everything you see so far, um, everything you see past this point in the presentation is a result of the the grant that we got from the IPFS Foundation. Um so we're working on two specs with them stupa and Muxil. Stupa is the simple standard for provenance and authenticity. This is the simpler of the two standards. It's very much thank you, Ted.
Look at that. Ted got me a glass of water. Everybody thank Ted.
Stupa is our uh I don't know what makes a fork a hostile fork, but it's definitely a fork of the CTPA that takes out all the things that it doesn't really it actually doesn't take out anything. Like if you want to pay digits 200 bucks a month, you totally can. Uh but we have some different opinions about that. We have DIDs, right? So instead you uh instead of signing it with some cert um that you're given by a CA, you sign it with a signing key that has an authentic uh has a has a provenance chain back to your atproto identity, right?
Um and then we also like make it consistent with with Dazzle, uh uh uh the C2PA signatures, C2PA manifests rather already use CBOR, um, so we're just upgrading it to use um Drizzle, Drizzle uh uh Dazzle's uh very strict subset of CBOR used in atproto, right? Uh that's that's stupa. That's all it's stupo. The rest of the presentation is about Muxil because it's way more complicated. This is just uh a little change that we're making here. Would love if they upstreamed it. Would love if there were DIDs and uh K256 signing in in uh in S C2PA.
So let's talk about the design goals for Muxil. Um this is basically what changes do we need to make to those MP4 files that I described before in order to uh make them appropriate for VOD. So you don't have thousands and thousands of MP4 files on your computer after a log after a long live stream. So we want to keep the property that they're self-certifying. It's awesome that you can just find one of these files and then go verify its provenance chain. Shouldn't be you shouldn't require external data, right? Uh as a design goal, we want to support live stream to VOD so that uh what what video engineers call DVR, basically if you're watching a live stream that you can seek back and see what happened one minute ago, two minutes ago, three minutes ago, that kind of thing.
Uh it would be nice if it's a useful format. I don't want to give you some opaque CBOR blob or something. Uh a design goal of this is that at the end of it you have at the end of a 24 hour live stream, you should have a 24 hour MP4 file on your computer. Um easiest to work with, maximally compatible, preserves all of the provenance information I was describing. Um it should be streamable with minimal overhead. Um I'm not gonna get too deep into media over quick in this presentation, though I would love to. Um but uh basic, you know, we don't want to do a bunch of extra effort to uh go from like the canonical uh uh format that we're gonna use for all of this uh to actually streaming it to users, right?
We shouldn't need a re-encode or anything like that. Umfortunately, uh that is an impossible series of constraints. Uh uh fundamentally the reason being that uh if it's good if the output is gonna be just an MP4 file on your computer, that is completely different than the format the Media Overquick uses that's gonna be useful for sending things to users, um, in particular with things like multiple tracks, right? We really want to do multi-track video, multi-track audio, 720p uh uh uh bunch of other uh 1080p um different audio formats, AAC and Opus for different things. All of that complexity we would like to be able to package in this one file, and that like skyrockets the bandwidth that you would use to uh to send it out to people, right?
You don't want to send them every rendition, you want to send them just one. So how do we get around that? Uh and the answer is we mux. This is why it's called Muxil. Um so the key insight here is that the archival format. So this is to say the MP4 file on your computer, the streaming format, which is the format the video is in when it goes over media over quick and uh and streams to a user. And the self-certifying format that has this intact signature that you use to validate that the media is who it says it is and get all the all the nice metadata.
Uh we they don't all have to be the same format. As long as we can deterministically uh mux between them. As long as if you have one of these formats, there is a deterministic algorithm you can use to turn it turn it into any one of these other formats. Um so we've accomplished this with WASM uh primarily. WASM, especially the latest version of WASM has a fully deterministic profile. Even short of that, you know, uh Rust compiled to WASM and then you pipe some bytes through it, you're very, very likely to get the same same bytes out through the output, right?
So uh this is uh yeah. So let's get into the different formats that we're using inside of here. I'm gonna get deep into mp4 file specifics. I don't I wouldn't wish this upon anyone having to dive this deep into the mp4 specs. I had to pay $300 for a PDF file. Uh so yes, this is the the the whole point of streamplaces. These are the these are the hard, the hard work we do, the hard problems we solve, so none of you have to, right. So we uh uh the base primitives here in that are that are part of Muxil.
Um first we have video metadata. Um I I borrow a lot of this terminology from uh a spec called hang that's from uh Luke Curly who created the media over quick spec. Um uh so this is uh cat catalog is his term for this, but this is all of the uh metadata that you need to understand the video. I can't just uh video decoders won't just accept some random video bytes. Uh you have to say, no, this these these video bytes are for example 1920 by 1080 at 60 FPS. Um you've got the the time scale and then a canonical track ID for for everything that goes into this, right?
Um how many channels is the audio. I don't have it listed in here, but this would have stuff like what language is the audio. If we've got the audio translated into several different languages that would be included in this metadata. This is nice to separate out because uh from a low latency transmission um uh perspective, you don't want to repeat this stuff, right? You don't want to say, hey, 1080p video, 1080p video, like you know it's 1080p video, because the last 500 segments were 1080p video, right? So we send the we can send this once at the start um and not have to repeat it.
Um I'm not quite getting into the atproto stuff here, but just at the bottom there, a little a little teaser of of what this might look like if you wanted to put it in atproto repository. That this could be a place.stream.muxl.catalog record. We'll get to how it actually works in a minute. Uh this lets us make init segments. Now we're talking about mp4 atoms. This is fun. So uh this data represented as like JSON or CBOR or whatever and stored in your in your atproto repository is enough to create, actually deterministically create the start of the MP4 file, uh which is called the init segment.
The very beginning of the MP4 file that contains the file type atom telling you this is an MP4 file. And the move atom that describes, for example, um the duration of the video, what all the tracks are. Uh there's a bunch of the this is like the the MP4 encoding of all that metadata that I just described. This is the beginning of the Muxil file init segment. We then split up the video into Muxil segments. Um this again is borrowed, this is pretty much verbatim hang CMAF, which is part of the BD over quick spec, which is awesome because that means it's gonna stream very efficiently to users eventually.
Um so this is just uh repeated every frame gets its own uh what's called a MOOC and an MDAT, so little segments, little little slivers of the MP4 file uh that represent each frame, uh each video or audio frame, essentially. Um easy enough, but we're gonna do a lot with these in a minute. And then uh we uh go back to that stupa specification I was talking about a second ago, the C2PA stuff. Uh so this is we can take um this is the canonical signed format. This is the thing that you uh when you get this format, you can validate the signature, validate the provenance claim.
No, this is actually definitely Eli's uh uh CTPA signed uh video file. This is actually maybe in Trabio or whatever. Um this is uh this is great. This is not necessarily the best format for uh for archival and playback, right? So one key insight and like the the like Eureka moment I had when making this is like okay, so this is like the canonical thing, but you wouldn't ever write this to your hard drive. Or if you did, because you're repeating the the beginning of the file, the the file type stuff, this is how this would get you right back to one second mp4 files on your computer over and over and over, right?
So while this is like you consider this like the base unit of Streamplace streaming or of or a VOD or what have you. Um this uh yeah, you'd never actually write one of these to the hard drive unless you're like debugging or something like that, right? What would you write to the hard drive? U ask. You would write a Muxl archive. Uh Blake 3. I know hangout with atproto people. I know a lot of protocol engineers. Every one of them has had a moment where they take me aside and they're like, Eli should really be using Blake 3 for this stuff.
And I agree with them eventually. So this is what you would write to your hard drive, right? So this is the init segment. This is the part where you say it's 1080p video. Then you have all of these.
And uh the cool thing about this is uh we've totally messed up the signature now, right? Because we took a the signature is over this file with uh uh init segment and a video gop and an audio gap and then a manifest, right? This is this is the thing that we signed. Uh this doesn't look like that at all. I've even just kind of put the manifest at the at the end there. But because we have deterministic muxing tools, we can recover this. We can or we can recover the signed canonical thing, right? Um this format, however, is really really useful for VOD.
Because if you are uh I'll show you why. See? It's like this. So this is a uh HLS playlist. This is how you uh you don't know it, but most of the time you've played video on the internet. You've had one of these happening in the background. Um what this does is takes all of those, this is just the video playlist. This just tells you how to do the video. The audio is referenced in a in a different um in a different manifest. And uh basically what this gives you is a uh uh byte range in an index into that file.
So you can say this this here says this is one second of video, you can find it at this index of this file, this is one second of video, you can find it at this index of this file. Um and this is well, this is all you need for efficient VOD playback. If you're doing it live, there's disadvantages to doing it this way. You probably want to use media over quick and some of the other formats that we talked about, but for VOD, this is really, really good. Um we've achieved one of our design goals because all of these, it's just referencing different byte ranges in archive.muxel.mp4, right?
So we've uh the thing that is I've been saying on your computer, but the thing for video, the thing that's in your atproto repository is um uh is just one big mp4 file. That's really easy to understand. The other um the other thing we get from this is uh is the Blake 3 SID. The magic of Blake 3, why why was it that all those uh video engineers were telling me that I needed to use Blake 3? It's because um this Blake 3 hash is a uh Merkle tree over this uh file that you see here, which means you can request just that part that says track one GOP1 there, and and uh through cryptographic verification, you can be 100% confident that that is the the correct data that you've pulled out of there, right?
You can verify yes, this is in fact part of this Blake 3 SID, which is really really useful when you're doing something like a CDN because then uh you can make all these, you can have be be backing it with just one big file, you can make these byte range requests, you still get the cryptographic integrity that like, oh yes, that that that hash matches what I saw in Eli's atproto repository. So this is a really really good format for this. Um yes, three different formats that we uh uh that we have here. So um the archival format, which is the big MP4 file that goes in your app proto repository, the self-certifying format, which is the part that has the uh C2PA signature attached, and the streaming format, which is just when you're actually sending it out, you just send uh you just send this part, right?
You you stream this stuff uh uh each each frame of video and audio directly to users. So for all the atproto people that didn't care about all the different structures of the MP4 file, how do you use all of this? We're not quite ready for it, but the very first VODs available on Streamplace are going to follow this format, and it's going to be all of the talks from this conference. Uh they will be published to atproto repository. Um you'll be able to work with them. This is a little preview of what this looks like.
Uh so we have a uh, yeah. So the man uh as you might expect, the the lexicon type for this is a place.stream.video. It's got uh things like creator, um some of the some of this metadata in here. This is like I'm pretending this is the one for this talk. I don't have a VOD for this talk because I'm still doing this talk, so you don't know what the hash is yet. Um it's got it's just as I said, it's just a blob, it's a video mp4 file, um, but it is very, very easily cacheable. Um you've got a nanosecond duration in there.
Uh we the design constraints for this, I've been targeting that the maximum length of one of these is going to be 24 hours. Um because uh at a certain point it gets sort of ridiculous. Yeah, yeah, yeah. Uh and you I don't want to like overflow the nanosecond duration. But if you've got a 24-7 live stream, that seems like a reasonable primitive that like each day of streaming that you would stream into would become its own VOD uh with a 24 hour duration, right? Um and then it's got a live stream reference. So uh yeah, so uh all of this will be available.
Yeah, I was hoping to have all of this available and I was like, oh, I'm gonna announce it in this talk and it's gonna be big and we're like really really close, but I wasn't I didn't want to push a bunch of changes to Streamplace right before we did all the live streaming and screw everything else up, right? So I I did try to do that. Uh uh we did that. Uh uh I did at about 710 this morning. I'm like, all right, let's push the Muxle changes, let's see. And then immediately that laptop couldn't stream into Streamplace anymore for whatever reason.
So I was like, nope, nope, back revert. We're gonna we'll do it, we'll do it after. Um so that's all by way of saying this is all very, very close. Um and we're hoping to give it to the community very, very soon. Um yeah, and then build it with everybody, right? So we don't have a front end for it. Um I'm gonna challenge the we're gonna do a little VOD jam and say, hey atproto community, here's the videos, build me a front end for it. Given that without me asking, we got three front ends for Streamplace in the last three days.
I feel like in the vibe coding era, we're gonna we're gonna get some submissions. If you submit one of these, uh you're gonna earn a badge next to your chat name in Streamplace from now until the end of time. So uh yeah, that's it. That's how Streamplace works, VOD.
We have a few online questions. Great. Most of which I don't fully understand. Awesome. Um so uh there was a question about format Y MP4. Yeah. Um not something like A C A D C1 or something. Uh yeah, yeah. So the the big alternative, um Y MP4, uh there's a couple reasons. Um because uh it's the most ubiquitous media format and the that's the most highly compatible across everything. Um the second is um it gives us the uh it uh the the media over quick stuff um uh tends to support it a lot already. Um also the uh FM uh so this is like CMAf back to the this thing.
This is like CMath MP4 playback. Um and the advantage here is it lets you um separate out the it gives you that lack of redundancy that I was describing where the init segment isn't there are other formats that embed all of that init data in every segment, but we don't actually want that because it's redundant for for a long video. So um yeah, those reasons choice of codec, why uh vp9 instead of each sure, sure, sure. Uh question was cho wh why vp9 instead of or wh why h264 instead of uh like vp9 or av1. No reason other than maximally comp maximum compatibility right now.
We really really want uh AV1 in in Streamplace. Um we fully intend to support AV1 as much as we do H264, especially when we do stuff like uh uh I want Streamplace to do a lot of things that no other streaming platforms do, like for example, uh like 4K 120 FPS AV1 video would be so sick. Uh uh can't even do it. I've tried to do it with H264, it just like doesn't. No encoder can even come close to handling that. So um yeah, yeah. Uh in terms of codec support, we want to support like every codec that I make sense to.
We just started with this because we're not quite in a world where I could probably get away with Av1 only, and we would have needed an H264 backup anyway. In that context, that's why we started with H264. But yeah, definitely want to support lots of codecs.
Thank you. Thanks.