Add Protocol Mark with airSlate SignNow

Add protocol mark

all right we are live hey uh everyone uh welcome to this first of many um where i'm kind of doing a live stream and having a guest and we're gonna be talking about stuff tech stuff usually and for the first episode of this that i'm going to be doing i have mark nadal so uh mark welcome to the show is it is that how you pronounce your name hey thanks for having me on um there's there's a debate between two sides of the family i i pronounce it in a doubt but there we go well um well cool so do you want to give a quick intro and uh tell everyone who you are yeah uh so i'm the lead maintainer behind gun is it okay if i start screen sharing please yeah so let me pop up gun is an open source firebase effectively and we're going to come back to this little index here to talk about what that means i think it'd be more fun to start with some hardcore demos and just show off uh what's going on so um first thing is a hello world application 17 lines of code you don't need jquery you can use react or vue or whatever you like but the gun parts of it is right here you connect to a particular peer it could be many peers or one peer we're then going to listen to any change on a text area and we're going to save that into a key value store inside of done with a put and then down here on this line we subscribe that data and we put it in so let me um spin up one of these self-hosted peers again you don't have to pay me um you can run this all on your own or you can use the volunteer network for it so as a demo of what's going on we have hello world woohoo nice all right so that's just the the starting piece and i'm going to jump into a couple other uh demos that are a little bit more um exciting and useful and then i'll come back and talk about the different feature sets so the next one is actually um a unit test that we run before publishing a new version of gun and it combines mocha with a thing we built called panic that does distributed load testing and correctness so rather than going through this 184 um line test i'm just going to run it for you so mocha tess panic chat and it's going to basically stream 10 000 chat records between these two um browsers and we're going to make sure that um each we've gotten each one and let me pull up uh oh and it already started so this is us making sure gun can behave performantly and we're done 2 000 chat records um synced in about uh 10 seconds um now this verify is right that the performance is working and there's a whole set of conversations here like i had to build a custom cpu scheduler in javascript to actually make this perform well and then let's jump into um user systems and authorization here is a multi-user to-do app in about 50 lines of code let me just give you a quick um example of it i need to actually switch over to uh here because there's the new version i'm working on that's the high performance stuff and then it's not released yet um and then there's the old version that's a little bit more stable so i'm going to pull up basic user right here so i'm going to create let me close that a user and these users are actually cryptographically secure so they're end-to-end uh they're using like end-to-end encryption for stuff and again we'll talk about some of these things later a b c so it's really easy to add um multiplayer to your ass so where is uh i mean i know we have a lot to talk about so but but quickly like where is this user being saved is that being saved in the in the uh the peer-to-peer network or is it saved in a database somewhere um all of the bob so it's stored and saved in the browser in the case of the panic test that we just did um the browser actually relayed and that the data um we'll start answering the questions just one last thing we have a lot to talk about and then i'll let you do a choose through an adventure of which of these pieces you want to explore the most let's do a tick tock clone and 34 lines of code here it is um so let me uh switch over to that it's kind of ugly but whatever uh 34 lines of code if it wasn't ugly out you know i don't know what i would say all right so i'm going to drag and drop um from over here a profile photo of me syncs over there let's drop in um a gif of a video conferencing tool i'll and then let's drop in some retro wave music oh wow so that seems pretty fast and then of course the classic big buck bunny video so i'll drop that in and let's see oh wow yeah and we're synced and this is this is on the old version which does not have the performance improvements okay i don't know if you can all right i don't know if my audio was playing for the russia wave in the video but those are the quick set of of demos we're going to explore some of the example code as well technology and architecture um behind it so yeah i mean i think this is amazing i mean there's so much to talk about as far as the implementation but also the the technology behind the implementation and how it's different i think than a lot of the stuff that you typically see in this environment because a lot of the stuff that you see uh for building real-time applications are built are based on services that you kind of have to pay for right and they're also typically uh you know these services are kind of managed services some company like aws or google firebase or one of these newer startups is kind of you know holding that data somewhere and you're kind of locked in uh in a way right but uh what is different about uh gun and and just kind of let's start with like a very broad overview because we're gonna dig into the details but like how does this work yeah so the easiest way to summarize it like i said before is thinking of it as being open source firebase for jam stack apps um i think we've kind of gotten used to porting over our you know our blogs and just putting in a git repo and then like publishing that on netlify and versal and so kind of this utopian dream i'm going for is that well what if we also have dynamic data uh tweets real-time gps coordinates um all sorts of information that is also in a jam stack like setting and the best way to do that is by having the browsers themselves be the host for the data and then having some backup peers so yes like the data will get back up to amazon s3 i'm sure you could plug it into amplify or any of the backend services as well but those are more acting as kind of backup um fallbacks they're not the the default structure of the system so kind of the three main features of gun are highlighted right here multiplayer by default with real-time peer-to-peer state synchronization um the biggest difference from firebase other than it being decentralized um is that it's a graph database so you can store key value um table relational data documents hyper graphs files jsons all sorts of stuff in it um while with firebase you're kind of stuck with a document store on its own um and a lot of people will just choose to use postgres instead and then the last piece is this kind of movement recently around having like local first offline data that is end-to-end encrypted so even if your data is being stored in amazon s3 or on some backend peer for reliability purposes none of those servers can actually decrypt the data if it's encrypted of course there's plenty of public data um but if it's your user data it's encrypted into end and that that isn't mineable by others so how does the real-time communication happen between the different connected clients and how do i know which data to listen for for instance um if we have a chat app that has a million chat rooms like how do you kind of connect to just the data that you you want to be subscribed to through the api have gone so down here on that um paste example um when we say gun git test we're saying we want to subscribe to this table or this node or this document um the cool thing is you can chain a bunch of gits if you have a deeply nested document you can then say well i actually only want to listen to the the paste property on the the test object and that means even if there's a million other records on the test table or document you're only going to be subscribed to that um that one property and then that message gets synced out over the network and that is actually what gun is inside of the stack um so gun is the kind of the ecosystem name for a bunch of modules that can be used with or without the database itself um ham to answer your earlier question about um how do you make sure that the data is consistent ham is a crdt that stands for conflict-free replicated data type and it's kind of a peer-to-peer version of google docs algorithms operation transformation um there's things called woot at a time crdt's have kind of taken over as being the better way to approach this whether you're a centralized service um or whether you're doing kind of peer-to-peer operations it's just more robust more reliable you can kind of think of git as a type of cdt itself um it's not entirely true but it's a it's a limited subset of a crd team um and git if you're not using github is is decentralized but still manages data consistency but gun itself is the one that's handling kind of the the protocol for what data is sent where what you're subscribing to and what the data looks like itself and there's a ton of these other pieces dealing with encryption networking routing storage and other components so um so if i if i'm using gun and i kind of uh create an app and then i'm um you know fetching some initial data from um i guess so well okay i guess first of all where would that data come from if i create if i create an app and i distribute it in a bunch i have a bunch of users i have a brand new user and they and they um kind of sign in or they do whatever they need to do to kind of fetch that initial data that's going to come from other browsers correct yes so when you initialize gun you can point it to several peers so one of those like i was diminishing here is just my local host here right which is usually on 765 but there's a bunch of um publicly hosted volunteer on peers so in this case like uh gun js.hiroku app.com you can like one click deploy to hiroku you can run this on amazon directly or a digital droplet or um there's docker containers as well this is is not because gun um requires using some sort of server to connect this is because webrtc which lets you do peer-to-peer browser connections itself um as a browser feature requires using a signaling server um so we've had to work around that and that's what these relay pairs do and they have built-in webrtc signaling servers inside of them so when you first connect to the network you are probably bootstrapping through some public facing um relay pier but regardless of whether you're using webrtc or not those relay peers then connect you with other browsers and those browsers sync back so if i show you the panic the designers panic test here that we stream the 10 000 records the topology that's set up is you have alice which was the browser on the left that's only connected to a relay peer called bob and you can think of that as as being some self-hosted service you run or um one of the many that's inside of our community they can just use for free because they're volunteer and then that connects to carl which is another browser so it's not i specifically coded this test that bob the server the relay does not actually store the data oftentimes relay computers will also store and cache data to performance in the network so in this case alice makes a request for the chat table and there's a cache miss on bob and bob relays that to then carl and um depending upon what modules you have in gun most of the default ones it'll do optimized routing so it's not going to send that request to browsers that are not subscribed to that data set um then carl winds up having that data in memory or in local storage or in indexeddb and carl will literally then host and serve the data in this case um i'm specifically not testing webrtc um carl could reply with directly to alice over webrtc but i'm trying to test the performance of the network as a whole because webrtc will fail a lot so that means carl then um puts the data on chat and starts streaming it through um carl up to bob the relay and then alice receives that data and then after it receives all 10 000 records the test confirms and verifies and then closes out so that kind of gives you an example of one of the many different topologies that the network could form um this is the closest to the traditional server model but again the data has been hosted and served from the browsers got it so um i guess when when you think about some of the technologies that are starting to become popular like if you think about the way that um i guess a lot of the newer hosting services like amplify console and uh verse cell and like you mentioned uh netlify they have the this idea of like edge computing and then of course edge computing in and of itself is like a huge huge thing now is this kind of like the ultimate form of edge computing yes there's all sorts of crazy things we can do right um let's assume that well right so if you are in california um versus you are sorry what what what's your mississippi actually okay um you want the data to not be pulling from you know amazon east in virginia it'd be really nice if you could actually pull the data from um a user nearby now there may not always be a user nearby but we can then combine this with versal and netlify it's actually something um we're supposed to be working on the next couple months but i've been busy with the performance testing is you can then actually have netlify and versal take the the on disk files so let me let me step back to this index um so rad is a storage engine you can use rad without gun okay um and what rad does is it uses a radix tree and it splits that radix tree across many different files on disk whether that be in the browser with indexeddb or on a relay pier we are theoretically able to then take those files that are split across on disk and cache them um on netlify or verso such that even if you have a browser that's never loaded any data before and it's not already cached in gun in local storage a completely brand new browser um tab can open up and then pull that file that chunk from the cdn and then swap it into indexeddb and local storage in the browser and then hydrate the application straight out of um the cdn delivered shard and then connect to the network so that way you can get even faster upstart time um render to paint even though we've been testing uh if i come back to actually i think i have to rerun the test um even though gun itself is very very very fast to can you move your uh your terminal up a little bit there you go yes uh mocha test panic let me rerun this one because it showed um first paint um time so chat um so first okay so the first reply and again remember this is being streamed from the browser was was and this is all localhost so of course when you run this on um a larger network it's going to be larger but it it was um 86 uh okay what is that in milliseconds yeah so that's yeah 86 milliseconds so that's sub 100 millisecond response for a record that a table that's streaming 10 000 items now that that's for the first chunk of that 10 000 items so i i just to clarify my comments around using the netlify and versal on these cdns for improving performance it's not that gun itself is slow for doing um first responses but there's still that um startup time where websockets and webrtc has to do the handshake so we can supercharge this yes by combining it with jam stack um uh cdns and it doesn't really matter where this data is anywhere in the world it could be a browser it could be um on the cdn or it could be it could be anywhere so how does uh how does gunjs know who the closest peer is um usually that that's kind of an in-works thing so there's this thing called axe in our system which is um it it does the routing and sharding so gun by itself is you can think of as being a mesh network it works very very well um inside of local area networks bluetooth we don't have any bluetooth adapters but you can think of it as like you could use this for self-driving cars emitting data like physically over over radio um but you don't want in these settings that that structure of mesh data to then um operate when you're running a service that has users in london um san francisco and mississippi uh new york like you want to optimize the routing for this so axe adds that as the layer without violating any of the peer-to-peer parameters because the optimized routing doesn't doesn't work it then falls back to the mesh network structure um for people who aren't familiar with mesh networks i want to just explain this a little bit more mesh networks are really good for settings like burning man or music concerts or local area networks where you have your phone and your laptop and a couple other laptops on the same wi-fi router um they're also like i said really good for doing like physical low-latency um networks so is it good for communicating when you don't have an actual like signal and you're not connected to like the internet yes exactly very offline settings um much much more concise explanation however those um formats don't scale as well when you're then talking about the internet you don't want to be sending every single message over udp multicast um to all other laptops in in the wi-fi router you don't want to be sending those messages to then millions and billions of other computers um so what acts does it says hey we're detecting that there is a up and up in there's a hierarchy to the the peer-to-peer network and it's then going to rearrange that network based off the hierarchy and then only route um packets in gun um based off of the subscriptions so it's kind of like ip in that sense um ip addressing where it's only going to send packets to um you know tier one tier two tier three versus uh sub networks based off of its routing um and then the culmination of acts will end in a dht a distributed hash table which will automatically reorganize the network based off of data subscriptions and off of latency so that piece is not done um but even right now what winds up happening for your original question is that acts um when a request goes out acts will first look if there's any peers within side of the sub network with inside of this kind of area network or or this relay pier that are subscribed to that data and then it'll immediately forward um some requests to a random sampling it'll like load balance the request to those browsers with inside of the local area network or inside of this server cluster and then directly respond back and while it's doing that it's also simultaneously then saying hey is there any external peer in the dht or network um that'll then forward the messages to that will then um perhaps hit some more persistent long-standing node that has the data uh more permanently versus uh peers that just have it cached so that i could explain the whole argument to that i'm going to be i'm trying to be short um i'm not trying to yeah there's a lot to it yeah it's a super interesting i mean it's obviously a very complex system like underneath and i'm i've looked at the api though and like even though it sounds complicated it's actually pretty simple to use but like um no it really is but but but like what what most developers are interested in are like a lot of a lot of developers especially these days are kind of like so bought into uh their framework that they're looking for framework specific implementations um and and for instance i don't know like if uh like in your demo you're basically manipulating the dom and i don't you know most most developers are these days are not writing jquery and and htm i'm sorry and um and document.createelement all that stuff so like if they were to kind of start using it now like what packages would they be using also i know there is like a graphql gun i want to talk about that but not right the second i don't know how much you know about that i know it's like a library from the the community or at least i think it is so there's view gun there's graphql gun there's react gun there's react native guns there's even some native guns mobilizations there's several gun there's um angular gun might be really really old both of these are community contributed but we're trying to um make sure they're they're as uh up-to-date as possible it was a little hard when reacted view and a lot of these systems are upgrading so fast so right so yes the big query there was just a like common um ground kind of dom manipulation right right just like a basic use case yeah um so then my encouragement is for people to take those really basic um example apps and then build the react version of it build the view version of it so gun works really really really well with any ui framework so um quickly before jumping into the ui framework the last thing i wanted to comment about like the scale the network is i do know that the routing algorithms work very very very well because um we've been testing against uh and this month is actually pretty small about six million monthly active users we've seen that as high as 30 million monthly active users so the system there's some bugs i'm still working through there's a lot of battle testing i'm meaning to do but um just because gun is peer-to-peer doesn't mean that it doesn't scale extremely well we've seen you know up to like 17 000 concurrent users per second that work now if you are running any system with that production you're certainly gonna have to come to me um because there's all sorts of bugs and problems i'm still fixing at that scale but but uh it's getting there hopefully by next year we'll be um uh pushing about 100 million monthly active users but i still gotta get this this new high performance release out and and test it against the current network well how does it feel to be like working on something that that many people are using that has to be a really cool cool feeling it's extremely gratifying as an open source developer that like yes like these tools not only can work but can scale but also have um adoption the the crazy thing though is that as an open source developer you're still always comparing yourself against other packages oh right right oh you know react and view even though they do different things right but like i know how many hits firebase has i know how many hits you know back in the day parse had um so it i it is it's extremely gratifying um but there's always the next stage i don't want to assume that the technology works i always want to assume that my code has the bugs in it um and so if you if you saw in our in our in our example apps if you go to actually i guess it's still running um the if you go to any app it kind of logs out this message at the very beginning which is like look if you spend more than five minutes trying to figure something out and gun i want to assume it's my fault that we can make this easier and better so come to our community and complain we'll figure out a workaround in the meanwhile um but that that definitely is an example of an area we can polish and improve gun so i'm i'm trying to make sure i stay as humble as possible and assume the worst case because it usually does wind up being my code well um so i guess the question i have is someone that hasn't actually built something in production with gun even though i have kind of played around with it um coming to this right now someone that's never used it before if they were to kind of want to build you know um an app that stored some you know some information that is kind of like a to-do app or something like that and then maybe have some user authentication in front of this app would you say that they would use gun for both the users authentication system and to store the data or can you mix and match or do you recommend doing you know one of the other and also can you walk through the steps like from very very scratch like i create a new react app using create react app what do i do next yeah so to quickly answer your first question it is highly recommended that you use both the the gun database and the gun authorization system because they both kind of assume peer-to-peer structures um we have had companies and startups that try and mix like firebase login with gun but then you're doing this like this weird match up where the public keys the cryptographic verification is being like saved back to firebase so this either um either the startup wants to manage the account of the users in which case like there's no benefit of having the private key because now the private key is exposed to the server which by default and gun it's not or um you're then trying to do this like matchmaking so it's a little bit complicated to integrate it with other um user off systems but it's been done and if that's important do it but you know we are trying to encourage uh companies and startups to switch over to a more ethical system of having pretty pair user apps where the identity and the login is owned by the user not by the company then the next question okay if you want to get started or run through the ropes um i try and have everything go out of the browser and then for developers know how to do command line they can follow up there so just go to gun.eco eco hit get started and this is going to take you to our classic to do app and this will walk you through the to do app i showed you earlier it's an interactive coding tutorial in about five minutes you'll learn about the basics of gun and that's the best way to get started i do want to note that because we've been shifting from this kind of old version of gun to this new version of gun where i've rewritten it for um that performance stuff i do think that this to do tutorial is maybe slightly broken towards the end um so please uh have patience and hold on but but code wise yes this is as simple it is um to get started in about five minutes to build a multi-user to-do app on top of guns so like uh sorry go ahead okay i was gonna say so say say that i wanted to connect uh my my gun.js app to some database for persistence um like what what would be the best way to kind of to get started understanding how that all works well gun does persist your data by default um like i said in the browser with local storage and indexeddb if you include rad or storage engine in the browser or um any most of the the relay peers that you connect through that bootstrap your connection to other browsers um a lot of them will also store data and they will dump that to disk and they will typically back it up to amazon okay so by default i mean so you don't even have to worry about that at all you were saying like okay got it okay that makes sense that's interesting i do want to be clear if you want to use gun purely for its in-memory cache you don't have to use rad you can like disable the storage and there's plenty of people who who do that um and there are other people who have then created both gun adapters as well as rad plug-in adapters that dump the data to not just s3 or indexeddb or disk and also dump it into [ __ ] or level or postgres now i want to be clear that there's a distinction between the interoperability between postgres versus gun it's not saying that you're importing your postgres data um it's just saying that we're using postgres or mongodb like a file system so don't think that that means like there's good interoperability between your existing [ __ ] database set and gun database set gun is is if you want to get all the cool tricks it offers at the real-time synchronization state resolution it has some metadata and a structure it has to save the data as so you mentioned crdt's state resolution i'm guessing that the whole conflict detection and conflict resolution is already kind of all built in how does that work for as far as complex objects are concerned and offline like use cases and things like that is anything like that taken into consideration ah so i have a wonderful cartoon that i've made for you um we won't go to the cartoon in this video of course i'll just reference it for people to check out gun.edu matters.html it explains the difference between um the archaic um uh bad systems that you know use this terminology of master slave that we just need to destroy you know databases that are not egalitarian that still have paxos and raft built into them so it kind of gives an explanation of what those systems look like and then does a comparison against what crdt based systems are so the gist um if you if you don't want to read through the entire uh tutorial there is in the docs here um if you browse through on the side you can go to learn architecture or crdt and it'll pop up actually a a 60 second video a one minute video that just very quickly um kind of explains how the crdt works it's a deterministic cdt so every single pier runs it there's no single um master or um source of truth so we're deploying that notion and making systems more equitable and more egalitarian that's awesome yeah i know one of the things that we've been working on and that we've kind of been trying to build out um some type of solution for is kind of this this answer to the um offline real time you know uh use case where you do have conflict detection conflict resolution um but across multiple different types of clients and i think that um you know that's always been a really tough problem to solve in the industry and i feel like there's a lot of different approaches to it but it's not an easy problem to solve it's just like you know there's so many things to take into consideration and and depending on your approach there's just going to be a you know typically some type of trade-off you know it's true um we're not a strongly consistent database we're not globally consistent so that means you probably don't want to run monetary or financial transactions through gun um some people have tried to do that and like you you can build guns such that it makes those trade-offs underneath but then you lose the performance right gun is what's called in the cap theorem and highly available um you know offline first uh partition uh partition resilient system so i would argue that those are more interesting because 99 of the applications we use on the web are you know streaming cat videos looking at cat photos um a twitter gps coordinate like you don't need a sense of global consensus so don't use gun for you know uh financial purposes um or you can experiment with that but don't complain i want to do in advance so but that you know banks are not the most popular apps out there people don't use their banks that often compared to facebook or twitter um the other thing here is because i'm come from more of like a mathematical background honestly the crdt stuff the math the the protocol and synchronization stuff was much much easier for me than implementing that gun api the chaining api that's as you described kind of very simple and elegant because there's all sorts of like nasty weird javascript edge cases that i had to go through and you know even six years later i'm still rewriting it for performance purposes so the hardest thing for me to build and gun was actually an api that's extraordinarily simple and elegant and deceptively hides all of the complexity underneath but the complexity there is javascript not the actual cdt or the database management um that the protocol stuff for the most part is easy you just sit down you talk with your friend and you figure out how humans would build such a system and then you implement it in code and then you suffer the misery of the code you implemented and realizing you have to rewrite it five times because it's not performant or you did a for loop in a place you shouldn't have done or json parse is ridiculously slow um yeah so well let go i was gonna use that as a kind of a moment to step back and look at the rest of the stack yeah yeah um i in order to to switch over to the high performance mode i implemented a custom json parser and stringifier which sounds like the most ludicrous thing to do and i've avoided it for five six years but um unfortunately when people are trying to live stream video through gun um and drop in like these assets it's still work in progress so there's still some performance issues i needed to make sure that um large objects inside of gun were not going to get serialized into json um because json um with v8 chrome's v8 will just stop and then process the data and that that could take like 250 milliseconds a quarter of a second um to process and you can't have that when you have a relay peer or a browser that's a video game right like you can't just have the video freeze for a quarter of a second or the relay pier freeze for a quarter of a second when there's 10 000 other connections so ysun is a yielding json parser that i implemented from scratch in in javascript and the fascinating thing that i kind of want to encourage a lot of application developers is hey if you're just building an app um chase that like don't don't get into all this crazy low-level performance just use the frameworks but if you're curious about this stuff you're always going to be able to one-up your career by rebuilding kind of the lowest level pieces of technology from scratch and you'll find out that um it's fun it's like a puzzle it's a game and and oftentimes it's not that hard if you if you sit down and think about it from first principles um if you wind up googling your answer for everything and you try to mash those all together it's going to be difficult and complex and and weird but if you just like sit down have a whiteboard or a piece of paper and you would like think through the logic a lot of these systems aren't that hard to rebuild from scratch i'm not saying you should rebuild them from scratch but they're great for educational purposes and then go back and use the frameworks as long as they're still delivering on the performance um that you want so i found out that while my custom json parser is on average three to five times slower um in in than um v8s json parser it's only slower for small json messages but if you happen to have an extremely large um value inside of there the javascript implementation of the json parser is actually 10 to 20 times faster wow wow that's great that's wild and the reason why i think this is the case is because v8 in the c code or whatever it is um for those large records we'll still do um byte by byte iteration and i don't know why in my implementation i use index of to seek to the next quotation mark which is the json delimiter and so i'm pretty sure for the like these large records that are getting saved um my code is just skipping a thousand bytes and not doing per byte um uh checks on it so as a result for large records it happens to be you know 10 to 20 times faster while not slowing down the the ui or the the relay peers because it's the same code that's shared by both um but yes for for smaller messages it is a little bit slower but i'd much rather have um the responsiveness and a non-freezing ui that doesn't lock up right right that makes sense i mean yeah that makes a lot of sense i have to say one thing i really think is pretty cool are these acronyms they all sound pretty cool like ham gun rad panic party yeah ptsd i can't can i come back here because it's fun a lot of them have like recursive acronyms to them so uh rad is radix at disk and then of course you could take a rad there uh radix and then like recursively [Music] um gun stands for graph universe node um the argument here is that you represent any data structure as a graph you've probably heard me mention this in other podcasts is that um a document database in mathematics is just called a tree um and then a table database in mathematics is just called a matrix both matrices and trees um can be represented as graphs so a graph database whether it's gun or neo4j or the other ones out there can store any other type of data but a document database or a table database cannot necessarily store any other data unless you do like special formatting because a matrix um while it's isomorphic to a graph a graph loses information when you convert it back into a table or back into a document um and there's some talks i've given that show visuals on this so um what are the most popular types of apps that you see people building with gun yes um there's a ton of people doing video stuff lately and i want to be clear like what was the last thing i imagined gun would would work doing so i'm still video is so hot right now like i see stuff a lot of the work i do at aws now is is helping customers build uh live streaming stuff or or uh just video stuff in general yeah so i'm i'm that's why i've been doing this performance stuff to up the game and make sure the video streaming and video playback is uh performing better i still need some time on that um so video's been very popular and then what gun was actually originally intended for was like metadata small pieces of information like uh tweets or gps coordinates updating in real time so here's just kind of a quick sample of the crazy diversity of the different types of applications built on gun so people are doing 3d video games inside of gun they're doing distributed machine learning and ai models on guns so this is like showing um a food or energy network where two browsers find different um sources of food and then you synchronize it with gun and then the other browser realizes that the that um there's a larger uh score there's a larger um uh treasure to be found from the learning um that happened in this browser then there's like the uber analogy which is the gps coordinate here updating in real time on my phone that i can track where the car is going and then jump out in front of the car and then there's um of course like fraud detection data visualization and graph analysis where you're actually using graph systems for the full power where you're traversing them and looking at different interactions and finding um common patterns this was supposed to be a social network but it just kind of looks like a christmas um the rock concert but um social networks are being built on top of gun um the most popular one we have right now is iris um and that's a whole library on its own as well and then there's iot systems that are synchronizing um temperature readings across different systems then there's ar vr apps inside of 3d worlds and then uh augmented reality this is the cad system where it's synchronizing your cad software um across different devices ipads um from browser tabs and then finally the most recent one that i started with is mozilla sponsored us to do live video conferencing so yeah i'd like to talk about that um so what what part of the video conferencing app is is using gun is it the actual is it the chunks of of video data that's that's kind of uh being used to stream between the peers that to create the video in real time or is it like buffering the video or how does that work so it's three-tiered um the developers originally were trying to use like web torrent and direct web or tc connections for streaming the video where they'd have um and then that should work but because webrtc is a high failure rate um they kind of then switched to actually using gun which shocked me because they're like gun was actually a little bit more reliable than the other systems i i don't quote me on that because i would say that's not possible um but they did actually get streaming audio and video chunks through gun working reasonably well not for live streaming um like it would it kind of work but you would get like the the classic stutter or it would have to catch up on old it like wouldn't fast forward the latest one so then they uh revised that again um where they used gun to coordinate the webrtc connections and then use gun as like a fallback for some of the data and then streaming so meat thing that was um sponsored by mozilla is using gun for coordination however um in order to scale to larger room sizes webrtc just starts to stop working after about six peers especially if you're streaming video because that means you're streaming video out to each six peers and those six periods are streaming the video to you so they combined it with media soup which is what's called an sfu that they baked in the same way the webrtc signaling server is baked into um the gun relay they also added a media soup as an sfu into the relays here so that way the browsers could dynamically switch between you know if you're just doing a one-to-one video call you probably actually want to keep it to the webrtc video stream mode coordinated by gun but then if the room gets too big it would switch to sending the video streams to the sfu relay which would downsize and decode and do extra sampling on the relay pier and then broadcast a single stream out to the browsers however um that then starts meaning that the systems becomes more centralized or federated and you can't there's only so much you can scale that unless your zoom um so the next tier that they're still working on adding is combining that second version which was live streaming the video through gun itself with this webrtc so you can imagine that the combination you have this tiered structure that if you are doing um a presentation if there's a hundred people in a room and there's only two people presenting well probably all hundred people will form a direct webrtc connection to that um single presenter and live stream that then all of the photo icons of the 100 people in the room repo will probably be down sampled and saved as thumbnail like video thumbnails into gun and then that will be broadcast out however depending upon um whether you're subscribed to that chunk of the video room because like on zoom you can like scroll through um gun would only be subscribed like these five thumbnails and then if you scroll you know it would it would subscribe to the next chunk and so then you would be live streaming like yes a video through gun but the down sampled size such that it's indexed and cached um versus the larger sample size and then simultaneously mixing with sfu for different things is there an example of an app that's that's open source to show how to do video streaming with the gun or is there any examples on the website so there's the difference between video playback which i gave you um over here video playback i mean that's just referencing like an mp3 or something there right right yeah and then there is video streaming there's probably three or four but again the one that's gone in the most consensus and community developed around it is meeting um so you can try out anything by just going to meet thing dot space and then start up your own room and that will do um uh person to person video conferencing and then if you want to dive into the code oh cool check out so it's open source that's cool can you spin up your own version of this in your in your browser oh really this is well um in this case you'd be spinning up your the the s the the relay webrtc signaler and um and uh sfu as on like digitalocean um so that way you still get the performance benefits of that particular combination got it um but yeah you can go here to to look at how that's done if you want a peer streaming video through gun example you're gonna have to jump into the community and we'll reference you to some of the code from about like a year and a half ago that was doing that okay cool man uh i i think that we covered a lot and this is actually so like i think that um you know i meet a lot of people in the uh in the in the javascript ecosystem and the in the programming ecosystem i think that some of the stuff that you've talked about is is atypical from a lot of the stuff i see in the front end community and it's really interesting to kind of deep dive into this and i i feel like we could go on for like 10 hours into each like we went into each one of these little things that we've kind of gone over we could we could actually talk for a long time so i think that this is a really interesting conversation i think a lot of people are also going to find it interesting um so i guess one thing real quick before we jump off and finish so how can people find you on social media and how can they uh keep up with what's going on with with gun js and can you kind of show also the react and graphql and maybe view uh quickly where that stuff is just to kind of take a quick look at that so i'm mark nadal um on twitter follow me there i do a lot of updates on gun um gun dot eco is an entry if you want to start with that to-do app i mentioned earlier um you can watch some videos what's going to be most important though is to jump into the community so that's at chat.gun.eko and that's our discord um server we're trying to actually switch over to our own chat app and dog food but there's a lot of people who've been on gitter and discord will get you into the community where you can talk and play around gun eco will get you into the to do app and then um from like once you enter into the docs here um you'll see that over here maybe we need to improve this a little bit better where is the react stuff let me just what um it is listed somewhere in the readme but if you go to gun dot eco doc react unfortunately you have to do a capital r because github um wikis are sensitive um this will get you started with um react as well as then react if um and then sevelt is where you would expect right there as well and i can tweak these for you guys so that's how you can get started with the other ui frameworks that will play nicely with done and again my twitter is mark nadal n-a-d-a-l awesome awesome so um thank you for coming on and for talking about all this stuff it was really cool is there anything that you wanted to kind of uh shout out before we wrap this up it's a huge honor thanks for having me um on the show and the things i like to shout out is is sure there's all the technology there's all the algorithms but the most important thing i've found is creating a community that has shared values around freedom um fusion and and oh my goodness i'm blinking on my own value systems uh i talked about this at the very beginning of the the github repo um which is like we we want to build tools that are not just like free but they're also fair right so that way like i have a bunch of other podcasts i'll just link yeah yeah there's the whole values philosophy behind gun but i don't and it's very opinionated but um none of the the tools themselves have those opinion baked into them and none of the algorithms have that baked into them you can use all the pieces in the stack independently from all the other pieces in the stack however um for me the mission that i'm driving towards is trying to create a more equitable fair egalitarian free and um interconnected fusion interconnected um society and internet and and that's the purpose that drives me so take all the technology of the grain of salt if you don't apply it for actual meaningful social impact then um it doesn't at the end of the day it doesn't matter it's just a to-do app uh so let's do something a lot more important than that love it yeah that's awesome um yeah so that this is all really really great stuff so thanks again for coming on and um we will uh be talking again soon i'm sure thanks for having me on