So, hi, I'm Matt. I am my main thing I do, I'm the I'm on the core Astro team for the Astro Web Framework. I also work at Cloudflare. But I'm not here to talk about any of that today. I'm here to talk about Cirrus, which is my PDS. So I have built a PDS that I use to help self-host, and I hopefully might convince some of you people here to self-host your PDS as well. But first what is a PDS? PDS is a personal data server. It is where all of your stuff is on the App Proto network.
Now a lot of people here probably know this, but your PDS holds pretty much everything on the App Proto network for you. It's like it's also it's all of your posts, of course, but it's also all of your um your link your likes and your um your friends and every your whole social graph, but it's also everything from any of your other third-party apps that you're working with, all of the different network, all of the things on the approach network um live on your PDS. They also your all of the blobs live on there as well, which are you know the files, the images and the the videos and things like that.
So um the whole point of a PDS is to make it so that you are um able to have the network as decentralized so that you own all of your data. Um and what I didn't realize is as well as being where all your data is, it's also where you talk to. So it's where your API calls go when you're using the using the network. So when you've got um when you've got the like Bluesky app, for example, on your phone or on the web, it's not talking to Bluesky when it's when it's connected.
If you look at the network panel, and honestly, I mean the first time I knew this was when I opened up the network panel and saw that it was hitting my PDS. You talk directly to your PDS. So all of the all of your um connections and everything go directly to your PDS. So most of the stuff a PDS does is talking to your talking to you. Um this obviously your PDS doesn't have all of the data on it. Most of the data on the network is somewhere else. Um so for anything that it doesn't have, like all of your timeline and everything else, it proxies it to the um the app view, which you know in most cases is is Bluesky, um, which holds everything.
But how does the relay how does um Bluesky or the um the rest of the network get your data? It's done through a relay. So there are two relays that Bluesky has, US East and US West. Um they recently changed them around. And basically your PDS, when you publish something, it sends it out to the relay. So the way that works is it's a a web socket, like a permanent web socket. The relay connects to your web socket and it's whenever you post something, it sends it out into your um into the relay. And as you can probably see from this at the moment, it's not actually doing a huge amount of work, the PDS per person.
It's only doing something when you post on it or when you make a query on it. Because the app view isn't um so when somebody goes and looks at your timeline, they aren't hitting your PDS. What they're doing is it's being relayed to the app view from the relays, and the relays hold every single every single post on the network. Every single record, every single event on the network goes through the relays and they send it off to the app view. So all of the work really, the heavy work is done by the relays in the app view.
So it's a PDS, doesn't actually do a huge amount of work, but and you can anyone can host them, but that's not what people are doing. So you know, 99 point this, I mean these these are numbers as of last week. 99.4% of all um users are using the Bluesky uh PDSs. They're the the mushroom ones that are named after mushrooms. Um there are there are some obviously fast growing um other PDS hosts. Uh the the new the new kid on the block is Eurosky. I'm not including the bridgie ones on here because they don't really count, But the bridgie one is is like 10 times more than that, but I'm not including it because they're not real, you know, not real accounts.
So Eurosky is the new kid. It's grown very fast and has managed to make it all the way up to almost 0.02% of all users. And obviously BlackSky is also very, you know, is a very solid percentage of users in terms of use the ones who are using a third-party PDS or an independent PDS. And then there's another, you know, less than 3,000 PDSs out there which collectively make up 0.03% of users. And that includes all of mine. Now, why is that? Why is it that people aren't hosting these things? It's because it's really heavy and not designed for hosting your um hosting an individual thing.
It's to this is the um the Bluesky reference PDS is a like a Docker image, there's a hell of a lot of stuff going on in there. It's you know, it's well documented and you can host it, and it's great for hosting thousands of users, but it is complete overkill for hosting a single um a single account. Um so I wanted to build something that was easier and that was simpler and was a bit more like how you would host a single website. You know, if you're deploying a little single user website, you're not going to spin up most people are not going to spin up a Docker image.
They're not going to hold, you know, ran out a whole um a whole thing that's got you know all the databases and all that kind of stuff. Most people will just deploy it to a serverless platform or something and host it for free somewhere. And I, you know, I thought let's, you know, why can't we do um PDSs that are as simple as that? So I'm going to show doing exactly that. So this is my my risky demo. This is where I'm going to attempt to actually deploy this. So I've um. Yes, I can got that up there.
So I somehow managed to get the create PDS name on um NPM. So I'm making a big thing of that and have made stickers with it and everything because I'm so pleased that I got hold of the PDS name. So what I'm going to do is npm create PDS. Actually, I'm going to do P NPM because I don't want to wait around all day. Um I can get there in I'm watching this on the um the screen mirroring on here, so it's not the not the easiest things of doing. So um this is just like a CLI that what I wanted to do was just to make it as easy as possible.
Because moving over to a different PDS is actually there's quite a lot involved. You know, there's a lot of tools out there for helping you like PDS Mover and so on. But what I wanted to do was to make something that worked end to end. Now, if I ran through this without talking, it would take less than a minute. It wouldn't be much of a talk. So I'm gonna spin it out a bit. You bear with me. Um hope you don't get bored. Um so folder name. Okay, I'm gonna call this Atmo PDS. Package manner, PMPM, initialized Git repository.
Now the point for this is it it creates like a little folder, like you are deploying a website on your on your computer. Um and it's just pulling it down, setting up a PDS. I am not going to migrate my account on this demo here. Um I'm gonna YOLO this a bit, but I am not gonna try migrating my primary account or any of I mean also it it can take a while because it's got to move all of your blobs from one to the other. So that is the bit that takes a long time. Um when I move my main account, it took about 10 minutes to to move between them.
Um I'm sure if I'm sure that there are people who've complained to me that once they started to get to 50 gig, it was hitting limits. So um some people post considerably more than I do. Um so I'm not. And then this is where the domain is. Now I said before that I work at Cloudflare. This is not a sponsored talk. One of the key things that's gonna come out of this is that you can do this without paying Cloudflare anything. Um the idea is that this is you know basically free tier. Maybe you'll pay a penny or so for um for the blob hosting.
Um, but you need a domain that is on Cloudflare. So I'm going to do atMOS.mk. This is really difficult to see. Umk.gg. Um. Now I'm going to choose the same account handle as the domain. And that unlocks something that makes everything a lot easier. Because if I've got the same name and the same as the as the PDS, because this is a single user, this is something we can do. The PDS is also able to serve the documents, the identity documents. So you don't need to do any verification. So while I go through this, I'm not going to be going off and doing any DNS records or anything like that.
I'm going to just say, yeah, I'm going to choose the same thing. And I'm generating a DID web document. So if you know what a DID web is, then great, well done. I'm in the only place in the world where there's going to be more than about three people who do know what that is. But for those who don't, DID PLC is the centralized record. So the DID is your kind of unique identifier that you can take around even if you change your change your handle. DID PLC is the centralized one where it's sort of like managed for you on the PLC directory, which you'll find at PSE dot directory, which is hosted by Bluesky.
DID web is a kind of more simple, but it's got some downsides, but basically all that is involved for hosting a DID web is that you have a file on a web server somewhere that's like dot well known slash did.json. I am going to use the same domain for my DID, my handle and my PDS, which means that I can host them all, which makes everything a hell of a lot easier. That's doesn't mean anything, that just uh has to be unique per my account. So I'm going to take this opportunity to have a sip of water.
So this one, I think some of you people here might find this interesting. You can choose where your data goes. Now this is going on Cloudflare, as I said. It's storing it in a durable object, which is one of these weird things that we have at Cloudflare that I didn't really understand until I started working there a few months ago. And as soon as I started working that I was like, I want to use these for everything because they're amazingly cool. That's my sales pitch for the for the day. But one of the things you can do with a durable object is they they come with their own little embedded SQLite database, and you can say where you want it to go.
So most people would say automatic, which will basically put it near to where you are when you first create it. But we also for um for if you want to have data localization, you can have a guaranteed localization in the EU. So this is you know, this one is where it's guaranteed and is GDPR compliant and you know e-privacy and whatever. Um so you can choose that. Um or you can just say if you want it to be located near, you know, if you're creating it at you know a conference, which is not where you are, you want and say actually I want it to be created somewhere else.
These are not these are not ones that are going to give you a guaranteed location, but I'm gonna just choose auto. So I said that there was downsides to using a DID web. Um if you're using DID PLC, you you need a sign-in key to sign all of your um records. Um if you're using uh PLC, then your uh you can rotate your sign in key. So it's not quite so much of a disaster if you lose your sign-in key. If you are self-hosting your DID web and you lose your sign in key, you are screwed.
Um you basically can't sign any more records anymore. Um so save them. Um I am not going to save this, I'm gonna save it to a file because this is disposable. Um have you saved your signing key securely? Um there's a password. Now I wanted to make this all as non-password as possible. So this is most stuff you can do on here, I use do using pass keys, but obviously, as many people here know, um, Bluesky doesn't support pass keys. So I need a password, and it is password is just in order to sign into Bluesky itself.
Um I'm going to generate one automatically. So you've noticed that this is saying that a lot of these things have got saved to one password. That's just because it's detected that I've got one password CLI installed, so it will just use that. Um I'm gonna copy it to the clipboard. But it is if you're doing it with one password, that's the like easiest because it saves it as a Bluesky login, so it just works straight away. Um so I'm gonna copy it to clipboard. So now I just need to remember to not copy anything else before I finish this demo.
Um the email address, um, I don't use the email address in this at all. I in fact I don't have it even wired up to email sending because it's a single user. So I don't need to verify anything, I don't need to notify, I don't need to, you know, if I'm moving away, I don't need to like send in it. But so the reason I support email addresses is that some services want one. So the first one that I ran into that with was deck.blue, which gave a perverse error message that I eventually tracked down to meaning that I needed a um needed an email address.
So you can create an email address and we will use it for nothing at all. So now we've saved all those secrets locally. I'm going to push those to Cloudflare. None of those are secrets, you can see up there. Those um, but now I'm just going to push the secrets.
And now I'm going to deploy it to Cloudflare. And this is the risky bit. So I'm compiling this with VET. It's basically this whole thing is just a it's all written in TypeScript. The actual thing it's generated, if I've got time at the end of this, I can show what it is generated. Is basically a Cloudflare worker that is two lines. It is literally just importing and exporting the handler for it. But I've created a whole project in there so that I've got somewhere to put all of your secrets. So it does mean that I for every single account that I've created on here, I've got a separate folder in my repos directory.
That's the downside of single user PDSs. I'm sure there's a smarter way I could do it, but I wanted to make sure this was single user. So that should actually be live now. So I am going to attempt to oh, I almost did something very stupid there. Nobody stopped me. I'm going to uh that's not what I yeah, that's where I'm at. So the very first thing I'm going to do is log in. So this bit here is where you choose which PDS. So it says a hosting provider, it means PDS. So my PDS that I it was at MOS, wasn't it?
Um you can see Atmo2. That was that was this morning's run through. Um so if this has gone wrong, that would be the video you would be watching. Um at mos.mk.gg. So that's the the PDS. And because we use the same, I'm going to do atmos.mk.gg here and place that looks about the right length. I think we've lucked out. Um add your birth date. Yeah, that'll do. Um the um if I'm in I don't know why it started doing this because I did set it up that it um verifies me because I'm in the UK. So in the UK you have to verify your age on on this.
So I've got it hard coded that everyone is verified when you create it because you know if you're responsible enough to create your own PDS, then you're responsible enough uh to chat with people. Um I've um I have now got this account on here. Um, if you were to go and um look for me and I was to post anything, nothing was actually gonna happen. So what I'm gonna do here is actually connect to the network. Um so this is uh okay. So this is the dashboard. So this is how obviously we showed before that we were happening to be part of the network.
Um the uh the Bluesky network has to know that we exist. Um this I managed to log in here without the Bluesky, the without the whole Atmosphere atproto network knowing I exist because I talk directly to my PDS here. So as far as you know the network knows, I don't exist, but as far as my browser here knows, I have just gone gone through a you know a jot signing, hit the APIs, all that blah de blah. So at the moment nobody knows, but I've got one subscriber here. That one subscriber is me. That is the um the dashboard connecting to it.
So um nothing is going on there. Oh my god, I was hoping this might happen. So some people know about me. Um what I'm going to do here is I'm going to request a crawl. And what that is doing is telling the relays I exist. And look, US East and US West now know I exist. Basically, what this DID is just pinged them with a with a crawl request. And now we can see that the fire hose is connected. So we've got two things connected on the fire hose. Um they're at cursor zero because we have nothing in this repo.
So let's change that one. Let's write a post. Um I'm going to say hello at MOSFEA. Um and let's post that. There we go. So this is the event here that has happened on the um on the the fire hose. So this is my this is my most trickly little fire hose you're gonna get. I've dropped a single, um, a single event has come off that that fire hose. And I can see it's worked because it's shown up here. It wouldn't have shown up otherwise if if the thing's not working, it wouldn't show up. So what some people here might find is that if you search for me on Bluesky, um I may not be visible because it might not have been indexed.
So what I was trying, and I might be able to do this, is it is visible? It is visible. So um I've got 10 followers now. Well, I can see all this stuff going on here. I'm so I've still only got one post. I've not got done anything else. So I think the first thing I need to do is give myself a name. Um I'm gonna just say um I'm gonna say that I am uh at Mo PDS account. I know, very interesting. Um so what that has done is that's created another record here. Um so this is the profile is a record itself.
So but I know I've got all of these followers. So if I follow all of these people back, you can see that this is creating a follow record. So each one of you people here say say hi if you're if you're one of the people I'm following. Each one of these here is creating a new record. Now if I was to then go on to any one of these other um Atmosphere non-blue sky things and created another record on here, it would show up. So you can see I've got three types of thing here. I've got a lot of a lot of follows in there, but I think what I'm gonna do is I'm gonna go and see Willow's stuff because I know that she writes some good things, and I'm gonna say um, oh yes, those are amazing stickers.
That's a that's oh hi James. I'm gonna like both of these. And there we can see those are coming up through as likes on there as well. Um these notifications are not on the network. What I've got here is I'm directly logging in to the app view to grab the notifications because the notifications are authenticated, which means that they can't be on the um on the app proto network itself. Um so basically it's the same as the the stuff that's that's coming through on that. So um I what I what I've got is that I'm able to um show all of these things that I've um that we've created.
And this is you know this if I'd done this without waffling, that would have taken less than a minute. And some of you people here might have tried it out while I've been sitting here. Um you would I mean the difficult part is you would need an account with a a domain on Cloudflare. You also unfortunately do need a credit card or a card or some payment method, not because we're gonna charge you, but because we need to store blobs on there and blobs are massively abused on Cloudflare. People you know put all kinds of crap up there, which means that you can't put blob, you can't use R2, which is the S3 um compatible thing, um without a card on there.
So even though you know it will you if you get charged, it will be pennies. Um but that's that's the way that that goes. So let's um that was uh I'm not done yet, guys! Thanks for the applause. Thanks for the applause. But I haven't told you how it works yet. Um I'm gonna put my glasses on so I can see you lovely people again, and I'm gonna actually talk about how this thing works. Um I've got still got a few minutes, six minutes. Okay, I've got time to talk about how it works. So it's inside Cloudflare.
Um it's uh that's mainly just because it's an easy way for me to put everything all in one place. And the key part of that is um the uh the fire hose. And I'll get onto why that why that's that special. But basically, so as I said, the client um is you know the browser, the web browser or the um uh or your mobile app and that talks to the worker, which is basically is running a Honno um little web server on there that just does the XRPC um API endpoints. And there's like a load of API endpoints that it serves on there.
Um and it's you know doing the sending messages to it. And the durable object, which I mentioned there, is where the um the kind of the magic happens in there. So it's got a SQLite database in there which stores your whole repo in there, all of the MST stuff in there, it's all one big database or little database really. Um then there's the fire hose. Now the fire hose is the thing that makes it really difficult to host a PDS on a serverless platform in most cases. So you can't put it on like an AWS Lambda or something like that because it needs a per uh continuously connected web socket.
The actual thing that made me decide to build this was when I heard about found out that you could do hibernating WebSockets on durable objects. What that means is that you can have a WebSocket on this PDS that is open all the time and connected to the relay all the time, but you're not paying unless you are sending an event. If there is nothing going from it, it just goes to sleep. It hibernates it, but as far as the relay is concerned, it's still connected. It thinks that it's connected, but it's actually just being held by you know some part of our Cloudflare info or something.
But when you then want to add send an event when you're posting something, it wakes it up and pings it off onto the onto the relay and you get charged for a few seconds. Well, you don't get charged because it will be within the free tier unless I think even the craziest of posters won't be able to get outside the free tier on this one. And then it uses R2 for storing storing all the blobs. That's all still inside Cloudflare. And then it proxies it to the app view, which is you know wherever it is. And I didn't show you a few things.
So there's account migration, which is built in. So that was one of the things I didn't want you to have to use these uh any thirds party service, however good they are. I wanted it to all be just done through this create PDS um workflow. So if you are using, if you are migrating something, you will move over to where the you know it'll take you through a different flow, you'll put in your password, and it will migrate everything. It'll download it all and then upload it, and then it will help you do the PLC um the DID PLC move migration and all that kind of stuff.
It uses pass keys for most stuff. I was going to demo pass keys, but it sticks a bloody great QR code up on the screen. Um so I was like, no, this is probably not the best way of doing that. But um that's something I'm particularly pleased with because to add a P uh uh add a pass key on there, it does a P it does an uh QR code in the in the terminal that you can use to scan it. And that means that works for all of the you know all the third party services and everything.
Um this is a personal data server. This is the thing that's the this is you can build one for yourself. And um I think there are I think that there's a you know most people do want to have their um their their PDS hosted by Bluesky. You know, that's where it makes sense to do it. But I think that if we want the app proto network to be properly decentralized, we need to have lots of different ways of hosting. And I think that you know we it needs to be easy for a service to have their own PDS, you know, one of the any any of these um other services can provide it for their users.
There they want, and but I think it also needs to be an easy way for people to deploy it themselves. And I think that one of the things that I really love about this group of all of us, you know, weirdos most affectionately here is that it reminds me a lot of early web stuff, and one of the whole things about the early web is that you could you know deploy your own stuff and do all kinds of and I think that the fact that all of the reference architecture of the Atmosphere is Bluesky stuff means that it's you know very much angled not just towards the social um use case but also to the massive scale and it's great that people are building all these other different implementations, um, but I think that we need more ways for people to just make this as easy as you know sticking something on Netlify or Vercel or Cloudflare or whatever, whatever you think of them, they are very, very easy way of putting a website up.
And I wanted this to be a way to make it that easy to put a PDS up. So hopefully some people will try this out. Um follow me, follow my 15 um other uh variations of this that I used over the past week while I've been practicing. Um I'm sure they're all gonna have really useful things to say. Um and thank you everybody. One minute.