Select text to annotate, Click play in YouTube to begin
00:00:04
after that round of intros i wish that i'd made this like more philosophical but uh this uh uh slot or at least the description in the schedule is mainly about um uh you know some of the
00:00:16
people are building platforms to do apps on ipfs and seeing how we can you know do cross-pollination and uh work and help each other so this is essentially a mainly a technical review at uh roughly breakneck pace uh to see
00:00:30
where there might be uh points of overlap but again please do ask questions along the way and if we go totally off from these slides it's fine by me so uh the web native sdk um which is uh the name for or the
00:00:43
umbrella term for a bunch of stuff that we're building over at fission uh which is a portable edge stack or uh just an encrypted at rest file system location independent user controlled data self modifying apps and services
00:00:54
off protocol in a trench coat uh i'm brooklyn zelenka i'm the cto at vision and uh where we started with all of this stuff was uh looking at a lot of people in the web through space
00:01:08
were struggling to build um sort of classic web 2 apps and we were looking at you know a lot of the tech and said well this should actually make things easier right as much as we care about decentralization and security
00:01:20
that's a little bit like um uh you know eating your vegetables right like it's something you should do but like you know people say they like security but they don't they don't behave like they do right like lots of people use facebook
00:01:34
literally billions of people use facebook on a daily basis so how can we make things um not just better from a security and decentralization point of view but like what's the
00:01:46
um how can we make people's lives better on like literally a day-to-day basis right so the original idea was this is you know sorry your typical web 2 stack today has all these parts in the middle
00:01:58
and that's a lot for people to have to train hire for um maintain all the stuff in the stack you know suddenly now you have to do all this devops stuff these days in addition to just trying to get an app to your
00:02:12
users right um and by using some of these newer techniques and newer technologies i think we can shrink this down and make it so that it's building an app
00:02:22
on the web or on a desktop is essentially the same right we can make everything feel like like an ios or android app everywhere
00:02:35
um and then the the actual benefit this would then bring to people is that they have much faster iteration you don't have to train them in all of these you know you have to become a full stack dev plus devops plus all these other things um you focus on the users you can
00:02:48
respond to their feedback much faster and you have a lower barrier to entry right so if we think software's going to eat the world we need to train a lot more people a lot faster right high level dependencies we have always wanted to get to compute um
00:03:04
but to do compute you need to do data and to do data in production you need some sort of auth system so that people can't uh look at all of your all of your files so we ended up
00:03:16
building it in roughly this order off then data then compute um this is the stack that ended up resulting from that uh we're not all the way through this yet we're about roughly here
00:03:27
working from the bottom up um so and we'll we'll touch on a bunch of these throughout this presentation so uh using dids um for identity uh tree um modified cryptotree you
00:03:40
can networking this durable file store which i talked a bunch about yesterday so i'm gonna skip uh like mostly over that but that's the web data file system uh we have offline and async sharing and we're now starting on a
00:03:52
uh scalable fully distributed uh database uh local first on top of ipfs as well um and we will eventually get to that portable compute section uh near the top there
00:04:05
and then there's this nice line and then abstractions to get that into developer's hand so that they don't need to know that there's this towering stack of things underneath in terms of actual experience um
00:04:17
[Music] the end goal right is to make it so that people don't have to know that there's something uh special happening so sorry i didn't create a new demo this is from a previous presentation
00:04:29
where it just looks like a regular app but the data underneath is owned by the user and portable between apps so here's that file from the previous app and we can now load it up in this sort of file
00:04:40
explorer view uh or have um data you know the user gets to see essentially almost like an oauth login right um and uh loads of music into a
00:04:54
music app right so this this kind of thing so starting with that bottom of that that three-layer stack um authorization and authentication uh because we decided to start with the
00:05:11
web uh that meant that we're working in browsers and browsers are extremely hostile environments um and doing key management here is hard right um
00:05:23
and in terms of threat model uh the biggest problem that you can have is uh somebody runs off with your keys right so it doesn't matter how secure your key is if somebody has it they you know they've installed a malicious extension into your browser or their app is going
00:05:36
to go around you know collecting your keys you're essentially hooked so um if we have this uh malicious user uh they're trying to get to that key but
00:05:48
with the web crypto api uh we have non-extractable keys so that sits behind this barrier and you can then pass the things uh to sign but they never actually get direct access to this key so we put a line in
00:06:00
the sand right at the beginning for anything that needs signatures it has to not have exportable keys for uh encrypted data we don't really have a choice right you need to hand a key around to somebody else so that they can
00:06:11
decrypt it but for uh signatures uh everything's done this way now when you're not moving keys around uh but you still need identity there's two ways of doing this um
00:06:23
one is with dids and then you're going to register all of these keys to a single identity and then the other is with capabilities so um dids broadly it's a standard from the
00:06:35
w3c um and uh actually has tons of adoption uh so microsoft bunch of governments are using it um and it's also based on public key crypto and you end up with the document that
00:06:47
essentially says here's the identifier and all of the keys that are associated with it currently um we technically use these but only the id key is just a wrapper around just a
00:06:59
single key and using ucan which we'll talk about in a second we transfer not the identity around in fact we don't really care so much about who the user is we transfer the authority around only
00:07:11
now the other advantage of that is uh you can register the same user into multiple apps uh and uh you know let's say
00:07:23
boris has an app and he wants to share some capabilities with me we don't have to create some sort of shared user to do that we can just pass around the little bits of things to each other so it ends up being very very flexible and you can do things like progressive accounts so somebody can
00:07:35
show up not sign up at all start using the app immediately and if they later want to register a username that's the thing that they can do but they don't have to they don't have to register anything with anybody by default there's often reasons that you want to for
00:07:47
discovery but you don't have to we're also increasingly finding that so we have some data and placing some of this information about who the original creator was or the key that created it directly in that data
00:08:01
structure in a well-known uh as a pattern is uh extremely useful so that we can then have some data structure that is owned by or at least has some you know it's like signing your name on the bottom of your shoe sort of
00:08:13
thing right it's like this is owned by so-and-so right by this key at the at the root um [Music] so ucam this is how we move uh capabilities around and we like to say
00:08:25
that if dids say who you are who you know fundamentally is about asymmetric crypto so you know they can sign with this thing therefore they're and nobody else has that key therefore they're who they they claim to be but
00:08:38
you can show what you can do okay so this is divided really neatly into uh authorization and authentication uh they look pretty familiar in fact we designed them to to be familiar to web two devs so
00:08:53
they're jwts uh with some special fields in them and they have these two with which is a resource it's just literally a uri any uri can go in there and uh can what they can do with this
00:09:05
thing and that's some some action and then uh potentially some extensible fields as well so if you want to put say a regex in there you can do that and then it works on this chained model
00:09:20
so we have these different users or they don't have to be different people these could just be different devices uh or even different processes on the same machine it doesn't really matter
00:09:31
so you have the original creator in this case the little artist in the the cloud has some ambient authority because they've signed their name into the data structure right um they'll delegate by saying well only these things only these capabilities
00:09:44
only these actions i'm going to delegate to the astronaut here and then that astronaut can go further and delegates more and more as long as these are getting narrower as we go along they can only do the same or less
00:09:57
uh as as we move through uh and so it forms this chain that's uh cryptographically verifiable as we walk back so if you get this last invoked one where you know the person actually goes to use this um then we just check um all of the
00:10:14
proofs uh walking back in this chain and we can prove that yep the the owner the person who signed into this data structure um was the original creator of it um those proofs
00:10:26
are just embedded in the regular jwt like this they're serialized and then signed so we need that serialization um ahead of time so that the signature is verifiable because in jwt you serialize it first and then you sign it
00:10:38
so these have to be serialized to base64 url and embedded inside and in the latest version of the spec that we are hopefully going to merge this week these have turned into cids
00:10:51
so they've been extracted out now and you can put it in a car file for example and then the signature has to match whoever issued the credential and then as you're unwrapping these and you're walking back through and checking
00:11:04
that these are all valid you do the same process to each proof which may recursively have more proofs inside of it the upshot of this um is we're using the
00:11:16
same essentially the same tech as uh as oauth underneath uh where this is the oauth sequence or fairly typical one it's about i haven't counted this in a while but you know 12 or 13 steps
00:11:28
to just access some resources on a resource server and requires you know four actors in the in here um and uh we can actually so this is uh actually being generous to oauth this
00:11:41
is a you can flow so this could actually just be a single line but let's say that we're doing it in like a classic application style uh it would look more like this um we and it also um because it's uses cryptographic proofs instead of looking
00:11:54
up things in a database or some shared secrets this all works offline as well so you're not always going out to some server and incurring that latency cost so file system uh if you were around
00:12:10
yesterday uh you saw a bit more of this uh user controlled data means that the data is controlled by the user not by the app uh and so you know we have alice bob and carol
00:12:24
bob has a photo gallery but we can also share data across multiple users that are using the same app so uh or or potentially different apps right so carol's video game is going to be spread across bob and carol's stuff
00:12:38
and we can load them both in at the same time um alice's music player maybe it even has some overlap with the video game and everybody's stuff in this in this picture so it makes data portability
00:12:50
really easy there's no more export function and because of how um you know as i was saying before we don't really care about who is doing the action like literally which human just the signing keys
00:13:02
these could also be the same person just different devices and so we've made it very easy for these things to then also merge together later so if you create you know two different in giant scarecrow users on two different apps or
00:13:14
even on the same app you can later come back and merge those back together we'll talk about that um in a little bit uh by default uh everything is immutable uh so we never delete anything
00:13:28
um you can go through and delete stuff um but it's more manual so uh just like how if anybody's on a mac or or it has been on mac for a while rather time machine where you can if you delete
00:13:40
a file you can always get back to it or see a previous version of the file we've implemented almost exactly time machine they do that with unix hard links we do this with cids essentially hard links
00:13:52
um and uh using all the nice uh structural sharing properties that we get from uh from ipld to do that so this is the same directory adding one file we only need to add
00:14:04
was this four links three files and we get full versioning uh underneath uh and then we've also made it evented so that from the root you can see
00:14:16
you know this kind of event happened below and then there's this insert new and then which specific file was inserted so it's actually it's more than a graph it's a technically a hypergraph
00:14:28
because we have these uh events on there as well um because we're building things for apps uh we need to have multiple writers at once and uh to do concurrent rights
00:14:42
so uh these um merkel crdts uh were originally developed at protocol labs and has been uh extended by uh martin klepman and and a bunch of
00:14:53
others um and essentially we can take um a file system over time and use a reconciliation mechanism to essentially do like a
00:15:06
git merge automatically without having to uh ask the user to do this for you um by picking one of them um automatically and then uh if we picked the wrong one uh based
00:15:20
on heuristic you can always go back and say well actually i wanted the other one and bring that forward um big advantage of this is if you have these two completely separate file systems uh it's the same you can just
00:15:32
merge any two arbitrary data structures now so if i have two users i can switch them together or if i have one user who's diverged because they were you know offline or had two devices that were doing updates
00:15:44
at the same time and there was latency or any of these cases we can stitch them together it doesn't matter the system doesn't care which scenario scenario you're in it knows how to do this reconciliation
00:15:56
um and because it's multi-user uh we need a way of sharing things when they're not online which is pretty simple we create a deterministic naming system encrypt things do a key exchange
00:16:11
offline key exchange and then let that as an entry point into some encrypted data they can decrypt it with their asymmetric key and then unwrap whatever data you're sending to them the
00:16:28
downside of this today is you have to know that there's some data over there that you want to look at and where we're going and i'm going to touch really briefly at the end on this distributed database is that ends up as
00:16:40
a behaving as a message queue and so you can uh gossip hey there's going to be a message over there for you you should go look at it but that doesn't exist today today you have to actually know to go open up the specific app and then it'll
00:16:52
do the automatic check for you um talked about this yesterday as well um obviously there's a hierarchical encryption so if i only give you access to my photos directory you only get access to that you can't see my documents you can't see the root of the
00:17:07
file system and this is all done just with um symmetric encryption each of those keys whichever one i give to you you can see everything below because that will give you the key to the lower directory which has the keys for all of its files
00:17:19
so if i give you only one file at a single version superversion right um you'll only see that one or i can give you a directory at a certain version you'll only see that one or we can say this directory for all time and you can
00:17:30
then see reading forward and that's the the time one there um actually we don't really talk about the skip ratchet um and that's it rediscovering that
00:17:44
structure we are rewriting it in rust um as the the kids say uh for portability uh and webassembly uh so that this can then run everywhere so we the last couple years
00:17:55
have focused mainly on the web um but we want this to run on desktops on phones on everything um and we've also found massive speed ups for a lot of cryptography uh switching from javascript to rust
00:18:07
uh and and webassembly um like 100x improvement in some cases uh so uh would recommend and then i said i'll just really briefly touch on we have this distributed database coming
00:18:20
um it is essentially based on property graphs plus encryption um and we take the same basic idea as the file system where we blow away these links
00:18:31
put it in a flat structure it's actually a set now instead of a tree the order does not matter um and we get a bunch of really nice properties like we can reorder all of the rows you can pull only certain rows if you if you
00:18:45
need them we have explicit dependencies uh on uh time and ordering uh that only appear if you care about those things you can write a query that isn't against time and
00:18:57
different devices can hold on to different literally down to the single row level you can just grab exactly what you need so if you're in a low power device it'll still work
00:19:09
and then this is going to get us all of that nice broadcast and soft real-time stuff that was talking about before that's it thanks [Applause]
00:19:36
um um so we've had people experiment with a bunch of stuff um [Music] uh so tiddlywiki uh uh has an integration uh which is nice
00:19:50
we can you know essentially bake down uh um html files uh and also make use of the file system and uh all the data in there uh which is nice so it can hold arbitrary data
00:20:02
including websites right so all of that stuff um we've uh one of our team members uh so a lot of people we've hired uh started writing demo apps and we're like you seem
00:20:14
awesome come join us um i built a uh dynamic wasm loader as well which is pretty cool so actually really excited about the like the dynamic programming
00:20:27
uh capabilities and so well not not dp but you know being able to load in stuff um statefully really interesting um the main thing that's uh held us back
00:20:39
right on the flip side um is we put the line in the sound early on saying if we're uploading over http and we're downloading over http we've just created a web server with extra steps right so let's lean into ipfs and that
00:20:52
has been both amazing and extremely painful right uh when it works it's awesome right but it doesn't always work i think that's actually a really good point and maybe something for everyone who talks to highlight so
00:21:06
we run js that's in the browser uh and also we have a cli where we can add everything locally and then use that's natively to actually pull from
00:21:19
the developer desktop um as an example and this is how we've ended up essentially here i mean literally here in this room of life oh we need to slightly
00:21:31
improve parts of the ips protocol so i think learn from us gs ipfs in browser is not really viable yeah yeah well and and it's both
00:21:45
like browsers themselves you know their their overall security model um and you know as things are getting more secure say you know firefox is adding more security features apple just announced a bunch of security features that makes it more
00:21:56
difficult the real pain is that they all have slightly different apis supported right so like web workers just like they work one week and then they stop working in one browser right with like 20 market
00:22:08
share like well what now we literally yesterday with the team but kind of late here uh i think we're basically gonna say chrome sparring because basically firefox has made it so
00:22:22
that just various things don't work and we're like brooks of firefox primary user yeah we're like i don't think we've supported plus great but rage is essentially chrome
00:22:33
but one asterisk on that uh so i'm giving a talk tomorrow the day after uh on carpool so we're also writing protocols and implementations into kubo and other places so that
00:22:47
it can be like we can use other transports than bit swamp right so biswap has a bunch of challenges especially for the web um not just the fact that you know the latency of all those round trips um and if you
00:22:58
can start pushing car files over uh http with deduplication you gain a lot right and so a lot of these problems go away so that's right a bit of both uh so there's um yeah uh
00:23:19
time on the dht it can be pretty bad sometimes sometimes really good the some browsers will just suddenly drop support for some features right so not
00:23:32
having web worker support for jsi bfs can be very painful uh you you look at content addressing and think uh obviously we should be able to share apps on the same system and not go out
00:23:44
to the internet and come all the way back and that's in a lot of browsers not possible currently right so um or you can push around small chunks of data but they're slowly locking down
00:23:55
different parts right or at least firefox and safari are um so or uh you know apple a year and a half ago announced that they'll just start randomly
00:24:06
uh clearing local storage and indexeddb right so even if you're loading stuff and it could happen at any time so uh not when the user is actively using it but as soon as they uh navigate away
00:24:19
they might clear that cache because they don't want you to get tracked right specifically right now jsfpfs works in demos that you will find on the internet for arbitrary peer-to-peer use cases
00:24:31
to run it in production we have to run ourselves a bunch of infrastructure which only links the fission infrastructure by default run um
00:24:42
uh secure yeah uh we need a tls proxy in front of uh websockets because that's not baked into kubo ulcer.js ipfs uh doesn't use the
00:24:56
main dht by default uh so you have to talk up to a specific instance you can network gsi pfs instances in the browser against each other over
00:25:08
webrtc but now you need to run a webrtc signaling server which should just get baked in i think into uh all of these nodes so that you could then just point out any of them but that doesn't exist today and actually matrix
00:25:20
has some apparently uh solution for this like very recently in the past couple weeks they just started on it um that we could maybe bake into um into things uh just book transfer support
00:25:32
sorry web transport oh great all right because that's coming a little bit we should allow attention to those without ssl support okay does that exist sorry when's that has that already shipped uh uh we're
00:25:45
kind of working on it okay the idea is we need to find the connector amazing so i'll put this in the parking lot but as an example i think one of the things is like hey wouldn't it be awesome if
00:25:58
the default experience is developed it just works okay hey who wants to run jointly run and webrtc servers like i'll run one like
00:26:10
ceramic piano right like exactly like we can get an equal five people running a webrtc and doing their community bootstrap that way and if you're excited about that you
00:26:22
should you should come to the uh content address alliance talk on friday ready yeah my understanding is that you can use the signaling particles like we don't have the
00:26:35
ephemeral uh uh so all of these things are how do you do it in such a way that a front-end web developer just has a default experience of network yourself where stuff works
00:26:52
it doesn't have to run in front of themselves the point here is that like everyone on the network everything else is running one of these servers basically so there is no need to do that so like this is we're currently using this for
00:27:04
like for kugo into like oh my god i'm running i'd get that's a production help please if i don't get wrong i don't know it's just yeah right and so can we like take a new initiative like this and be like sweet
00:27:30
you should do things it totally includes focus yes in ux i think that kind of came up as well again unlike your users awesome um
00:28:00
which has to make it all the way into our uh all of our um default app code examples that are like fill this out and this is a default thing that will give you a thing that will just
00:28:13
work or in your docs you like connect ours or whatever in this case it shouldn't so like the idea is that you would just probably get to a set of bootstrap works and the good job is i get it like literally that's the thing
00:28:26
that either has to be fully made into the library by default so that when someone includes the npm thing it just works and then we document hey by the way the point is that your pinata i get
00:28:39
that's gateway in this link if you want to also support like yeah right like yeah well and uh so maybe not about this specific um transporter or signaling
00:29:06
system but it like in general like kubo takubo works like a bazillion times better than anything involving a browser today [Laughter]
00:29:38
it was designed for browsers but then for some reason we decided to add support running in node because why not um but like it's not um more resources that has better
00:29:55
yeah this is a probably a semi-controversial statement um but uh i would love to see high quality wasmable rust
00:30:06
libraries so that people could construct their own uh nodes so that we're not spending time spreading across all these different implementations right and then we can get it into browsers into edge services
00:30:20
into all of these places right and that's not to say that it has to has to use all of these pieces but at least it gets you up and running and we can have really high quality libraries for this stuff so there is lib um live ipfs is it
00:30:30
now um in sego i think um which might work i haven't played with it yet though there wasn't time to get go into browser with assembly i don't know
00:30:44
how at that point that month um i know someone was using the uh go to bp um essentially we have this vision of putting together with web investigators developers essentially this has been
00:31:08
um extended troll applied research where we first had to admit protocol so our biggest adoption is actually did by peers so you can and their method has been
00:31:20
adopted so um uh hugo from the um they've gone all over twitter blue sky
00:31:32
jack's block team and the thing named red5 it's also nothing you can't because everybody needs to decentralize off layers for these sorts of things and then next up there's a number of people who are very interested in doing private
00:31:45
encrypted data um on ipfs and platform um so i'm looking at that we're happy with protocol adoption that's what you're doing and as it hardens it actually becomes
00:31:58
usable for the just works for friday [Applause]
End of transcript