Video: Space, Pixels and Cognition (DocDrop)

00:00:00

foreign [Music] ERS linkers and mappers we are very happy to host a series of conversations around the topic of tools for thinking our longer term goal is to spark a

00:00:18

diverse connected shared memory that will help us make important decisions together our near-term goal with these podcasts is to interest startups in being part of beta Works upcoming accelerator think Camp betaworks is a New York City based

00:00:31

incubator and accelerator they've run seven camps before on topics from Bots to synthetic media and voice interfaces always sort of Leading Edge stuff you can find out more about think camp in this domain by going to betaworks.com

00:00:43

Camp all lowercase I'm Jerry mikulski your interlocutor and obsessive mind mapper our topic today is space pixels and cognition my guests are

00:00:55

Yi Liu Shen Burke and John undercover if you've watched Minority Report you've seen John's work he invented the interface that Tom Cruise uses to find

00:01:06

pre-crime and uh didn't just do sort of fancy UI design actually invented a usable and useful uh gesture language that works called g-space and then eliu

00:01:21

has a soft software startup called Soft space and is working in AR VR XR sort of domains to try to figure out how data and experiences fit in that space

00:01:33

so um John what how did your life change at The Minority Report marker because I'm I'm I'm assuming that that that the popularity of the movie and the the simple

00:01:47

attractiveness of the interface that you helped create uh must have done something pretty huge for you it without question and you know it's it's easy to feel Petula and say oh I'm

00:02:00

sick of Minority Report but it's been a Perpetual gift and so in fact I'm I'm really grateful the truth too is that when you work on a film and I've I've worked on quite a few now even though I'm not really a a filmmaker principally

00:02:14

and I guess I'm a designer and a technologist but is that you don't know day to day as you're working on it during pre-production production post-production whether it's going to turn out whether

00:02:25

it's going to be any good I mean you can be on set and just watch unbelievably brilliant actors doing incredible things or watch the monitors and see a you know world-class cinematographer capturing

00:02:37

amazing images and the thing that comes out is still sort of a dull wad of damp dryer lint uh and and conversely so I was really excited when Minority Report

00:02:48

you know was a sort of sharp and uh and even political uh and and beautiful and Elusive as it was and uh right away I knew that that all the time that I'd

00:03:01

spent designing that UI and training the actors because it was not just Tom Cruise it was also Neil McDonough and Colin Farrell and his interesting stories there um had had really paid off um and you know people still talk about

00:03:15

Minority Report which is amazing to me and so there's there's something about the process that we undertook that was really dedicated to an actual various similitude not a sort of you know Cinema

00:03:28

level thing which is entirely uh the consequence of the production designer the great Alex McDowell's World building approach there so it it Minority Report led directly to a whole

00:03:43

bunch of things including my um eventually long tooth startup oblong Industries and frankly it was in in in describing what we were trying to do

00:03:54

with oblong it literally saved dozens or hundreds of hours out of my life in my colleague's lives because trying to describe an alternate UI and here we're getting right into the meat of the thing is

00:04:06

ridiculously difficult people are so kind of blindered into the increasingly incapable uncapable uis that we're all settled with today that say no no this is the thing where you use your hands

00:04:20

and there's screens all around you it's not just one little rectangle it sort of gives rise to cartoon question marks floating above people's heads but instead we were just able to say have you seen Minority Report and most people had in bang that that was it the weird

00:04:33

thing is that even though we and the writer and the and the director and everyone involved in the film knew that we were making this deliberately kind of gray ambiguous obviously Noir inflected

00:04:46

and sort of political you know query like is it okay to give up all of your all of your your your privacy in exchange for convenience in exchange for Commerce and the rest of it the tech

00:05:00

world didn't take it that way the tech world has used Minority Report as a shopping list basically of startup ideas and there are literally hundreds of stars like oh what if we did predictive policing and just arrested People based

00:05:12

on an algorithm right that's all over the place but if we installed cameras everywhere what if we made robots that go try to find perpetrator you know like so and we're like no no no that was supposed to the point was that was a bad

00:05:25

idea but anyway there you are the power of Cinema that's super interesting I hadn't quite thought about it that way but it's sort of like occasionally there are object lessons that totally miss their Mark somebody says something about something that is misinterpreted

00:05:39

entirely or or were the messages just missed right indeed um and and as we look at Mark Zuckerberg's renaming of Facebook to meta and his pitch for the metaverse and I I've critiqued this a little bit

00:05:52

online it's like uh that what he showed was would have been embarrassing for second life to show a decade ago uh or 15 years ago it's like hmm intriguing like have we made no forward progress on

00:06:04

this and I guess he's forgotten as well that the metaverse is actually the medium the vector by which everyone is infected with this horrible kind of cognitive virus in the book right oh

00:06:16

never mind the negative thing or snow crash right yeah exactly you Luke can you just tell us a little bit narrate for us how did you get to soft space what like what was what was

00:06:27

your path to to the startup sure thanks for that intro Jerry and uh thanks for that very insightful behind the scenes um snapshot John of what it was like to work on Minority Report which I did also

00:06:41

re-watch recently and thought there's a lot more in here than just the floating UI like there's a lot in there so you know the self-driving cars the um genetically modified plants that like

00:06:55

you know can poison you uh amazing stuff um the backstory to softspace is that I was working in this design and art studio here in Berlin uh as a research

00:07:09

residence this was in 2016 when the Oculus DK2 had just kind of come out or that was sort of a state of the art in terms of VR hardware and the studio had bought this headset to

00:07:20

try to see the potential of this you know exciting new technology either as a medium for artworks or as a tool uh for the production of other non-digital artworks and when they bought it they

00:07:34

realized oh there's not like you can you can't download an app and just try it out there's nothing obviously so they somehow um found me um you know I was willing to take a break from architecture school all the

00:07:46

time to come for a year and just play with this device I've been wanting my hands on one for a while I had sort of been teaching myself coding throughout architecture school and this was a good a good opportunity to get

00:08:00

paid to learn to code although they now they know this I guess but they didn't know this at the time um it was a really fantastic year um I had you know like one Sophie or once every two weeks sort

00:08:12

of prototype recycle I would build something showed around people like the time and at some point the idea started occurring to me that um a lot of these prototypes were

00:08:23

building where we were showing uh 3D models of physical artworks and people were able to look at them or we were showing um just photographs reference images reference materials

00:08:35

um in in a virtual space a lot of these experiments were starting to hint at the possibility of a new way of working that actually grew a lot from

00:08:46

the practices and the advantages of the physical Studio that we had around us it's this old beer brewery in the middle of Berlin and sweet it sounds like a great space it's it's incredible it's it's absolutely incredible workspace

00:08:59

um and and so much of the success of the studio comes from its uh from it's it having this physical environment where different teams could materialize their thoughts they could externalize their

00:09:10

ideas passers by would be able to sort of stop and say oh look this reminds me of something I'm working on downstairs like why don't you come to the welding you know shop and we can sketch out something or try this that idea and um

00:09:22

and this idea just just sort of started uh infecting my mind like a mind virus and I would wake up at in the middle of night thinking oh you know I I really like why doesn't this exist I really want to build this

00:09:35

um and in parallel there was a moment where the main artist of the studio um he has a secondary studio in Copenhagen he was lying between Berlin Copenhagen a lot and at one point he came in to a meeting sort of like really tired from

00:09:47

the airport and kind of joked like oh if only we could just you know put on these goggles and just have this meeting while I was still in Copenhagen and then it really clicked for me like okay this is this is something that's

00:10:00

worth pursuing this is something that's worth at least you know exploring and it was an art studio so it wasn't the right context to really pursue what ultimately is a a software product

00:10:12

um and so I realized okay I have to take this into a different context which I guess is you know um Silicon Valley and the tech startup World um

00:10:26

startup scene and a hot art scene so you're kind of situated in a really good place there it's been a very I feel very fortuitous to have had this spark you know in a

00:10:37

place where there were people around to to pick up on it although I do have to say that at the end of the day for my for my limited experience of such things there still is no place like San Francisco when it comes to really

00:10:51

building things at the edge of what's possible that's its own conversation I think we could have a a riff on that as well yeah Silicon Valley has this very strange and interesting effect on everything and and

00:11:03

I want to bring us back sort of to this idea of alternate user interfaces and and also um not limit our conversation to the present or to past experiments or to or even to code being written today but

00:11:15

just kind of let us wander a little bit into what we think is possible and how it might work what the limitations might be but what the opportunities might be and I want to add to the mix something that's become really important for me

00:11:27

which is uh memory or permanence or persistence I call it and one one of my problems with cyberspace right now with the web with the intertubes with whichever one you want to call it is

00:11:40

that it's all uh flow and not not very much stock we're drowning in the info torrent every year somebody invents some damned new social media thing like it was Snapchat then it was Tick Tock and who knows what's you know who knows what

00:11:53

the next year is going to give us but but I I every day I have to do my sweep it's like I I'm on a bunch of different media and I have to go figure out what somebody said on each one and if somebody says something memorable or

00:12:06

useful or in fact something I need to reply to like must reply to I have to now remember which damned medium was it in and it's all lost in this in this all siled up kind of my experience of

00:12:19

cyberspace right and it feels like that and so many other things are big opportunities to change how this works and how we think about it um so John I don't where has your mind

00:12:33

gone since Minority Report and sort of that marker in the sense of uh and and I love that your experience there was so deep in the sense of you know you can Stand and Deliver a talk while giving

00:12:46

the interface and showing what it does and how it actually works that they're they're gesture Primitives that you created and internalized and so by doing that practically I think you hit a bunch

00:12:57

of you know barriers and overcame them and then hit some walls that you knew you couldn't overcome because as that's where the limits of that were what else what else have you sort of started thinking in terms of

00:13:09

where this were these uh types of augmented or alternate interfaces could go I think any discussion like this can either proceed by nominating a bunch of deficits

00:13:22

and holes and absences or just skip that and and talk about the the stuff that would fill them in or you can kind of do both but it for me there's

00:13:34

I think there's a couple of Pinnacles that you can kind of see even from our kind of squashed to the ground position where we are right now um one and maybe this is sort of the the

00:13:47

catch-all is just generic agency and efficacy like that should be the metric by which we uh assess a UI and even that sentence already like I don't know how

00:13:59

far back I'm gonna push myself here because we have to go back a step to say we don't talk about UI God bless us all here and and you especially Jerry for like for creating uh you know a little

00:14:11

90-minute space-time window where we can talk about but by and large it's not a it's not a a fit topic right and we can all talk about it uis UI is Bruno we

00:14:23

don't talk about Bruno either but uh yeah but a few of us do and uh so if we're going to talk about it then the first thing we have to ask is how much agency does it afford you how much efficacy

00:14:35

like how what you know what does it let you do that you couldn't do by yourself and I think it's not it's not even particularly cynical to say that the last 15 years of of concerted and

00:14:48

well-funded development has all kind of pushed toward a narrower and narrower and smaller smaller efficacy set of tools instead of ways of thinking

00:15:00

about what digital stuff even is like what is the substance of the machine and therefore what is the substance of our minds uh conversely my view has always been that the

00:15:13

that the machine the computer systems that we build on it should be like an exoskeleton for for mines like Sigourney Weaver like Sigourney Weaver and the max yes exactly

00:15:26

except software we're getting xenomorph is ignorance well I yeah I guess forget bicycle for the mind because that's a metaphor that never quite worked for me but an

00:15:39

exoskeleton something that lets you extend your reach in a benevolent not xenomorph smacking way necessarily but you know uh something that lets you extend your reach something that lets

00:15:50

you transduce your ideas and your human will into a larger pool of ideas and in some cases into physical reality so that's the efficacy piece 2 is

00:16:04

collaboration right the um the personal computer that begat all this and we haven't really gone past it we just carry it around with us now is arguably too personal it is in fact

00:16:16

entirely solipsistic uh both by Dent of the way the operating system works I mean the stuff the you the indictment can start at the UI but it gets all the way down to the Silicon right because you the the UI is the part

00:16:29

of the computational world that you get to see and touch and feel and experience uh and it's supported by and therefore defines and inflicts what's at the OS level

00:16:41

uh and so all of these all of these machines all of these systems were defined solely to support the activities and the thinking and the work of an individual person at a time and that's

00:16:53

so un human uh I I I really want to to get back to eliu's early days in architecture because I think that I mean I I don't know but I have to think that

00:17:07

that all of the great stuff that's kind of pouring into soft space and the sequence of prototypes starts with starts with space I mean it is soft space after all so uh you know physical

00:17:22

architecture is is inherently collaborative unless it's an isolation tank or a coffin just thinking about that for the first time but otherwise Like We Build architecture for people to come together in and to

00:17:35

accomplish certain tasks we're eating together we're watching a talk we're sitting around a table debating um we're throwing cream pies at each other in the war room uh to call back to something that

00:17:48

happened before the recording started and so forth right like the architecture kind of encapsulates and reflects the activity and we haven't built uis and operating systems in that mode so from

00:18:01

my point of view all all systems all operating systems all uis and all application Level systems should be inherently collaborative and that can't happen unless the OS and the

00:18:15

the basic UI already has that idea like this you know this sort of video multi-communication thing it astonishes me that this is a business

00:18:28

like this should be just like what a computer does Bingo every computer just does this because of course you want to do but no you know and I'm happy for Eric and he's got a

00:18:41

you know I don't know 100 billion dollar market cap company whatever but but that's that's sort of uh that's sort of an indictment so efficacy collaboration and then the third one

00:18:54

that I'm going to nominate is exhilaration it seems to me that those of us who were you know who saw the stuff climb out of the primordial slime in the late 70s and early 80s can

00:19:07

remember clearly the excitement that surrounded the idea of technology and you know we can say that it was overly utopian uh or we could say that what we've got

00:19:20

today is a little bit overly dystopian but something about how you know the the Giant katsumari Wheel has rolled along and picked up more and more um it's not picking up everything like

00:19:33

it's picking up stuff in a single axis and the exhilaration is gone I mean you know Google or meta or whoever can or apple can have their yearly conference or a big announcement and try to make

00:19:46

something pretty dull ultimately something like in the grand scheme pretty dull as exciting as they possibly can with the aid of expensive agencies

00:19:57

but you know it's like we're just in one corner of Toledo and there's all of the continent and all of the world still out there with all of its stuff and we're not allowed to talk about it right like so the excitement the the exhilaration

00:20:12

still exists it's just not being implemented and I think that um just just as I would propose to measure a UI

00:20:24

by its degree of uh efficacy I would also propose to measure it by its degree of inherent exhilaration and and the the kind of uh precursors for that the the analogs are the things that we do as

00:20:38

humans that are you know sort of cognitive and physical at the same time so playing an instrument or scuba diving or dancing like all of all of that all of that stuff that's mind and body at

00:20:51

the same time that's what acceleration feels like and I think all of us on this column know that a digital system can do that also we've we've seen it right we've built it we've

00:21:03

experienced it we just don't build like that anymore exactly um you know before going to you I just want to put I want to make explicit something that is a a force or an energy around this topic that's almost like

00:21:16

background radiation from The Big Bang for us and it's it sort of starts with engelbart's 1968 mother of all Demos in which his whole thing is about collective intelligence the whole thing

00:21:28

is about collaboration everything about it isn't just about window overlapping windows and mice they invent the mouse for God's sake um but it's all that and then a bunch of people interpret that vision and turn it

00:21:41

into single player single user personal productivity systems jobs and Gates both Geniuses in different ways had to be blackmailed bribed and threatened into multi-user anything

00:21:54

they they were like no no this is this is a bicycle for one mind or this is like a personal productivity tool and yet and there's a bunch of people grousing I think very appropriately that that so much was offered to us by

00:22:08

engelbert's Vision that it still has not been really tapped we have not gotten in there and then and then at some point we wasted a whole bunch of pixels and we we actually turned this into the wimp interface the windows uh Mouse pointer

00:22:21

Etc forgetting exactly what the what web stood for um but um but that is now the tractor beam that draws everything back into it so the desktop metaphor is like this this black

00:22:34

hole with an event horizon that eats startups and I was a big fan of Prezi I use Prezi as my go-to storytelling stool for tool for a long time and um Prezi redid itself twice once to look

00:22:48

more like PowerPoint but I could still do with the original magic with the zebra menus and then twice more recently they just lobotomized themselves they just killed off the endless whiteboard that I could tell stories in and I now

00:23:00

have no tool to go tell endless stories and I'm I'm searching trying to figure out why did they do that well they did that because there's this uphill climb to get anybody to use anything that doesn't look like office right and so

00:23:13

and so this is a tractor beam that pulls all startups toward it and it's the Brave and unusual startup that can ignore that and go someplace else and I think xrvr AR is an attempt to LEAP to some new place and maybe if we stand in

00:23:27

some new place and then I see augment like augmented reality things with desktop metaphor sprinkled in and my heartaches my heart bleeds so you how do you how do you cope with being a startup

00:23:40

on that event horizon in this environment with with do you detect the force I just described and then how do you either fight it or ignore it and just leap to some new place yeah I mean absolutely you're not

00:23:55

imagining it I think my my biggest Shield against it might just be my ignorance I literally don't know how to write uh JavaScript I only know you know DC sharp so I I'm incapable of taking

00:24:07

that that um that direct line into the Event Horizon where I just you know make make a really fast and dirty uh and functional 2D UI I have to invent some

00:24:19

you know convoluted 3D way of doing it um but just there's so many interest so many really really uh really juicy points here and um so I've been thinking a lot recently

00:24:32

mostly because I have to put this down in writing for a couple of different uh things I'm working on but um I've been thinking a lot recently about what the actual what the concrete benefits are of

00:24:45

having an immersive 3D uh interface you know like I I get this question a lot especially when I share my work on Twitter and what you see inevitably of when I'm building is just a 2d representation of it you know uh through

00:24:59

the lens view uh something in a video and I get this question a lot from from the internet and from other people uh not on the internet like what is this really adding Beyond maybe the novelty

00:25:12

of of the third dimension and I I I'm I think of it the answer to this in in terms of three separate pillars so the first has a lot to do with Jerry

00:25:26

what you mentioned uh early early on um memory and persistence um and you know there is now this um finally finally there is it has been

00:25:40

established that a spatial canvas can be a legitimate Paradigm for building a serious software tool right it took a really long time for that to happen and and I know we in our in our other call

00:25:53

we had sort of mixed feelings about the figma acquisition but I do think that figma at least for people in my position who are trying to build something that um is legible to you know capital and

00:26:05

Gatekeepers that is also trying to push the the envelope something like the figment acquisition just takes it cuts it Shades like 15 to 30 minutes off of every conversation you

00:26:17

know people ask like can UI Innovation actually be worth all the headache won't someone else to steal it from you want to W steal from you actually no it can't like you can make this work within the you know or existing flawed uh system of

00:26:30

allocating capital and and funding Innovation and all that um and so just to say that um we finally recognize now that something that you know every human being knows all the time except when they're using uh you

00:26:43

know their computer which is um spatiality matters it really you know the way that our brains and bodies have evolved over millions of years uh literally if you

00:26:56

could not remember where like a hundred different things were in your environment um at different scales and you know that some of the things are moving and some things are seasonally you know if you if you couldn't sort of Juggle all that in

00:27:08

your head you would starve to death or you would like fail to find your way back to the Village right so we have a much greater capacity for remembering and reasoning about things in space than

00:27:20

basically any software interface right now is really you know taking advantage of um any any sort of like mainstream popular software interface and so for me what something like figma or canva or

00:27:33

all these other up the upcoming Apple free form app what that's doing is legitimizing this idea that uh space is an important part of uis and I'll look I can talk about this a little

00:27:46

bit more um uh later on as well is and my contention is that uh canva does not go far enough right if you want to think what the ultimate interface for

00:27:58

harnessing all of our superpowers um of spatial perception of spatial memory cognition would be short of a brain computer interface like a shunt um it would have to be an embodied

00:28:13

three-dimensional interface so that's one the second thing is um and this this I think connects John to your point about efficacy or agency within a UI and

00:28:25

I know this connects to things you're working on right now because I've seen some examples of this the second Trend that I'm seeing is people building software taking seriously the idea that certain kinds of knowledge and possibly

00:28:38

all knowledge has this underlying associative Network structure to it and this is something that you know going back harkening back to the earliest days of the internet um people knew like Ted Nelson knew that

00:28:51

forever ago right and we also know this because the internet every time you click on a link that's how that's what's going on but the power of the UI is such that the

00:29:04

uh the way that these links are represented to us in the user interfaces of a web browser completely obscures this this underlying reality and therefore people you know if if even today I mean like today I tell

00:29:18

people oh I'm building a spatial Knowledge Graph um and then they'll say what's that and they say oh well you know like all knowledge is all information is sort of like in the form of pieces with different relationships and they're like

00:29:31

that's wait what no and they think it's like no all knowledge is an Excel table or all know which is like a giant Word document right because those are things that or maybe all knowledge is like a 10-point pitch familiarity is because

00:29:43

that's that's that's what they know right really no and so my contention there with um a 3D interface is that um we have forgotten that all knowledge or

00:29:55

we kind of like have this double thing where we do we don't do know that all knowledge is in the form of a graph and we don't know at the same time because we don't have the interfaces that can actually show it to us in a way and I

00:30:07

say this I we almost don't have any so there are things like um so Rome has really popularized the idea that everyone can work very directly with a Knowledge Graph and then there are you

00:30:20

know plugins and and sort of like clones of Rome that are then creating this graph view in in your in your computer that shows this like pretty hard to use GUI

00:30:33

of a Knowledge Graph and I've seen versions of this for a more specialized use cases like I think palantir's product has some sort of like gooey like that but my contention there is still that you know a computer even a huge

00:30:46

monitor monitor display is just too small and flat to really let you work with an all draft and it may not even be the case that representing a full knowledge graph so literally is the best way to go

00:31:00

um but at least with my prototypes what I've been discovering is that when you can actually step into such a structure and look around and sort of like focus on one piece that you're working on and then just look up and see oh what's around uh what's what's over

00:31:13

there wasn't it further away um and then to go see the other piece you don't uh destroy your view of your the current thing you're looking at you just move over and if your brain is building up

00:31:25

this spatial relationship between the first thing and the second thing um already goes a long way to making these structures giving us great greater agency in working with these structures as I would say and then the third piece

00:31:37

is just you know as you like collaboration social co-presence meta has gone on a lot about this and I do believe that actually that is a major superpower of uh a head mounted 3D

00:31:50

displays that you can represent other people remotely in a way that is just much more just like emotionally impactful um than on video um in scratching sort of palantir you

00:32:04

raised another type of UI that people are quite familiar with and is completely quirky which is murder boards basically if you watched home Homeland or uh you know other sorts of things when there's a crime and somebody starts

00:32:18

pinning photos to the wall and then they're scribbling and then they take news clippings and they put them on and then they take yarn and pins and make connections between like people know what a murder board is I'm not I had I just heard the term recently but but

00:32:31

it's like it's a map you know sometimes with with more Dimensions than just a bulletin board but but it's it's a map of what's happening and then I'm reading right now a book about song lines that isn't Bruce chatwin's book that kind of

00:32:44

puts song lines on the map but it's a called song lines the power and Promise by Margo Neil garudwara and Lynn Kelly and it's a beautiful book and I bring this up because

00:32:57

song lines are many song lines are basically memorized oral culture only stories about the landscape and Aboriginal Australian culture is embedded in language and landscape and

00:33:12

the song lines include A Narrative of navigation of how to get from here to there the story of how the landscape was born that over there is a series of lizards like the seven lizards who did this who did that

00:33:25

a series of easements and property rights that are baked into the song lines and who gets to do what where a series of totems or taboos for the different tribes each tribe has a taboo animal of

00:33:38

some sort that they don't hunt they don't eat but it's different from the next one all these things are woven into song lines that are memorized and passed down for it could be 40 000 years it could be sixty Thousand Years we're not really quite sure when this all started but

00:33:51

it's intricate and it's all about space and when you and this happened for all indigenous tribes around the world sorry forget all for many so many indigenous ways of knowing were tied to the land that when they were

00:34:05

torn from the land their systems were shredded and really endangered and part of what's happening now as we're starting to wake up and go indigenous ways of knowing important we should do something it's like guess what

00:34:17

we've pushed everybody off onto the worst bits of land in each of the countries that we that we colonized and so and so maybe I'm thinking here this this thought hadn't come to me until this moment maybe we're trying to

00:34:29

decolonize idea space and we're trying to create a space in which it's okay to hang things and have discussions in some sets of new spaces

00:34:43

whatever they are and I'm really interested in what spaces we're talking about because it doesn't all have to mirror actual geography and physical space it doesn't have to do that at all right there can be all sorts of abstract spaces what do these those spaces look

00:34:56

like and how do we deal with this in a way that gets us beyond the tractor beams of not just the desktop UI but also sort of colonialism in some in some weird way how do we free ourselves to

00:35:08

experiment in those spaces man there's a lot there [Laughter] um yes uh I think that the you you mentioned

00:35:19

abstraction and I I I've as time goes on I become more and more convinced that modern Western Society especially around technology

00:35:30

gets the balance and the apportionment of abstraction and literalism exactly wrong every time uh the you know there's a lot to be said about video games uh we

00:35:44

I think in our in our prep call I mentioned that in my view the the best new UI work consistently for the last 30 years has happened there it's just that it doesn't get credited because

00:35:56

academics don't like to associate with video games and that sort of uh fluffery but in fact it's really really important but the great um the the great trans media uh granddaddy

00:36:10

Henry Jenkins I think it was him he who said nothing has held back the advancement of video games more than the drive to photorealism and uh you know like if you if you if

00:36:23

you just look I I love either that you were you were kind of going to Capital allocation and you know Marxism is just over the horizon there but uh like where has the money gone well the huge amount of money has gone into figuring out

00:36:35

subsurface scattering so that the skin looks better and blah blah blah blah but the content and the experience and the exhilaration does not lie there and you know 98 of all video games are are

00:36:47

representational in a kind of art history sense and a few of them are not and the ones that aren't I think have a lot to teach us um all the way back to Tempest my favorite of the early uh you know arcade

00:37:00

video games which is pure like it's a narrative of pure geometry and Peril and stuff but it's pure geometry like well what is that what am I I don't know but you know uh and and on and on like that

00:37:13

I think that the um maybe we should rummage around in space for a bit I and to start it at the very beginning the the single best theory of the emergence of Consciousness

00:37:26

that I know of is um by a great still I think living uh neurophysiologist called rodolso yinas who is retired I think but was at

00:37:39

NYU and he has a fantastic and and super accessible book called eye of the vortex that's I like the first person pronoun in which he describes uh Consciousness as a mechanism that arose

00:37:53

out of the need to move period right like if you're sessile if you're a plant stuck in the ground you don't need to plan because you kind of can't and so you don't need Consciousness but as soon as you're one or a few cells going you

00:38:07

know through an ocean you need to know what to swim toward or away from or what to try to eat you know or cidle up to or or whatever and to do that you need an internalized model of the external

00:38:19

world and in his view that begets Consciousness if that's true that means our brains and you know a lot about what we are and how we think and how we understand the world is about space it's

00:38:34

about the movement of bodies through space causality that happens in space and time so for every programming language in the world to ignore that to ignore both space and time as basic

00:38:45

constructs for UI to effectively water down space until it's like the shabby's possible interpretation of someone's awful desktop and thank goodness you can't even see mine here like for all of

00:38:59

those things to be true it's like you know missing the the gigantic pair that is so low hanging and so enormous that it's actually touching the ground and

00:39:11

it's right there and almost no one's you know even nibbling at it right so I I think space is the place and this is the place there we go we have a tagline for the call

00:39:24

right like so we we can you know it's we can literally imagine um reifying The Memory Palace which was kind of a metaphor or an analogy but we can make the Memory Palace real there's

00:39:37

the uh the highly eccentric German art historian Abby Warburg whose final work in about 28 or 1929 was a thing called the Builder Atlas the

00:39:49

picture Atlas which was incredibly personal and is a gorgeous work it was about 60 panels about six feet high four feet wide on which he had tacked murder what is it murder chart murder boards

00:40:02

murder board style uh basically reproductions of works of art and newspaper clippings and everything panel by panel it was his representation of the world so his view alone but he

00:40:16

thought it might have you know use utility and currency for other people and the Warburg foundation in London occasionally mounts exhibitions of this stuff but in order to see it you have to

00:40:28

rent a warehouse and you know stage these things around so he's literally situating ideas in space juxtaposing them the relative positions of of each board and each item on each

00:40:41

board is not incidental is in fact the substance and the meaning the meaning comes from the connections and the associations so many cool things here the meaning is between the pieces I I I've that

00:40:59

absolutely resonates with me one one thing I would later on there is and that's a good jumping off point um at Abby Warburg uh essentially sort of analog sort of soft space is

00:41:13

unfortunately how I'm going to think about it um and and this reminds me a lot of a lot of the uh walls and desks and working spaces at the the artist studio

00:41:24

where I was working before on this um one thing I'll layer on that is that software does have all these advantages that um add to these analog working methods

00:41:37

right these analog working methods are actually super cumbersome and and it's a testament to their value to us um that we go through all the trouble of like clearing out this amount of like

00:41:49

wall space or or Borg space or whatever and then printing out or however you know you have to manifest in physical media these ideas and then pinning them up and then if something changes later on to spending hours or days like

00:42:02

rearranging things um and and and a lot of these operations I mean all all these operations are are trivial and um you know can be too trivial they can sort of like lose their meaning if if they're too automatic in

00:42:15

the software environments but you can save a lot of time I think analogously to how um the digital spreadsheets uh grew out of the paper spreadsheet

00:42:28

um but then because of the speed uh with which you could run calculations and changing things around enabled a whole new way of using spreadsheets which was this sort of like speculative fiction uh Corporate Finance thing you know where you could like do do models and and um

00:42:41

tell stories about what what could be if if x y z um and uh I I think another maybe more fundamental

00:42:53

or deeper question um that the murder board to software murder board raise it that transition raises is what are the underlying

00:43:05

uh semantic Primitives if we're going to talk about the meaning is in the space okay great I mean you know Abby warbrick was an artist I think artists um have a lot to teach us about things

00:43:16

that we just or they're just beyond our current capacity to articulate right that's like almost like kind of the artist's job to point like there's something there in that in that general direction I don't I don't know what to call it we don't we don't have to think

00:43:29

about it yet but like go look there um I think this is maybe a similar area where we don't I mean I have some ideas you know that are working hypotheses that I have to incorporate into the software I'm building but I don't think we really

00:43:42

know yet what The Primitives are of spatial semantics are of like associative knowledge graphs um the and because we don't know and a lot of these applications what happens to just have this sort of

00:43:55

undifferentiated reference right it's like oh this link points that thing and then you build it all up and it's like oh just like a giant spider web of things that point to each other um within really specific domains of course you can have like a controlled

00:44:08

vocabulary of these are the three ways that things can be can relate to each other um you know if you're like if it's literally a detective's office with a murder board it might be like these people know those people those people spoke to those people et cetera

00:44:21

but in a more General sense I don't think we the thing is we there may not be right there may not actually be a like a general purpose Knowledge Graph uh relational semantics

00:44:33

or whatever but um we know so little about how how these things even work um and it's only through the process of trying to build software that reflects

00:44:44

that Abby Warburg Style murder board this term I'm not sure about it but okay um because to build this upward you have to you have to Define it explicitly like

00:44:56

what is that connection and therefore now now we have to think about it yeah well certainly the the surely the best way to figure that out is to try as many experiments as possible which is a little bit anathema

00:45:08

to the current kind of startup feel right and kind of Academia has been hollowed out by the the pull of the Bay Area and by the nonsense of people like

00:45:21

Peter Thiel saying no the the less you know like the sooner you get started as a strapping young capitalist the better for everyone right but at seven start at seven no no five in neutral that's when

00:45:36

real creativity begins uh like but so that there's something structural here as well I think that um that that could use a little work you know where I that's not how all of this started

00:45:50

weirdly right I mean it sort of well no it didn't right you you've got you've got engelbart like in he's like a one human Cambrian explosion right and everyone's like oh thank you

00:46:05

very much we'll take the mouse and he's like no no but uh you know oh no just sit in the corner Doug you've done well thank you very much right so there's that and there's Xerox Park yep no Labs

00:46:16

but it's just a few people in in each case I give a speech to the senior leadership of Xerox when Paul o'lear was CEO invited in by their CIO uh when I was writing for Esther uh and I

00:46:31

basically said to them hey you guys you who have invented half of the Tinker toys that are on the table right now there is an opportunity for you to dive deep and sort of fix how we communicate through ideas through documents and all

00:46:44

that and then uh shortly after that not be because of me at all they rename themselves the document company and went right back to business as usual but the first sign I had of trouble in that meeting was the guy who introduced me

00:46:57

put up a foam core board that was like six feet wide and a foot big a foot tall and had on it their strategic planning process which was a timeline lasting two years and it showed like a little flow

00:47:09

diagram of it and there was a little arrow in a piece of it that said we are here and I was like OMG the web has just broken into the world thing all bets are off if you sit here and hang on to your

00:47:21

two-year strategic planning cycle you are dead people like and I felt like my talk went into an anechoic chamber like like I I could hear my ideas just sort of be absorbed into the sound deadening the soundproofing material in the walls

00:47:34

and not make it into anybody's heads it was crazy and so so there's this real danger that we we wind up losing so much of what's just on the table right in front of us as we started saying and um either to go back to what you were

00:47:47

saying a bit there are a lot of philosophers and deep thinkers who've done some really good work on this a simple one is Scott McLeod's book understanding Comics has the great the big the big pyramid which and I'll do a

00:48:00

really quick uh screen share because I pulled it up uh looks like this and basically says Hey weirdly sometimes the more abstract the comics character like in 1010 which is one of my favorite

00:48:13

cartoons in the world the scenes are lifelike but 10 10's face is always this oval with just a couple dots for eyes and you're like wait what a how does that work as art how does it work that I'm always seeing Tan Tam but not

00:48:25

worried about his expressions and B uh part of what McLeod is saying is that the more abstract the character the easier it is for us to slip inside them and to and to sort of adopt what their point of view or see what they're thinking whatever and there's this

00:48:39

problem with the boundary with with realism that John that you put in the conversation earlier so there's partly that but then also it seems like we're we're not going to end up with one winner for what the right philosopher is

00:48:50

and what the right abstraction is and what the right set of protocols and Primitives are but I think we're going to end up having a series of protocols and apis or agreements or something that allow us to to honor different

00:49:04

perspectives and ways of seeing but then have this conversation in the middle and I'm really interested in what that middle is because if we're going to have group tools for thinking multiplayer tools for thought then we need to have

00:49:16

an arena where we're thinking together and playing together and in a fit of peak a little while ago I about the domain the bigfungus.org to give it some kind of playful name because I feel like a lone ant feeding

00:49:29

the fungus that feeds the hive uh when I add things to the brain that I that I'm constantly curating and I'm looking around going this is really fun where is everybody and a piece of what what the challenge

00:49:42

here is is describing where the space is like like where are we putting this stuff if you ask me where Wikipedia is I can tell you where the servers run how they're funded what software they're

00:49:54

running where the data is which is all open source Etc et cetera Wikipedia because it's an encyclopedia and it's using a conventional model and it's insanely popular is really easy to explain

00:50:05

you bump up a level of abstraction from there and it suddenly we're lost we're unmoored we're unanchored and one of the things we need in this space is I'm going to borrow here from yoga the sense

00:50:17

of adrishti and when you're when you're standing when you're doing a one-legged Asana and you're trying to balance and you're like uh you pick a point someplace and that's your drishti that's your Anchor Point for the pose and it

00:50:29

helps you completely settle down and hold hold a difficult pose we don't really have drishti's in these new cyberspaces or metaverses or whatever or

00:50:41

we're not paying enough attention to that somehow I don't know but but we need to to ponder some of these things and then drop them down into the Arcane World of Concrete code right I really

00:50:52

like that um I I will I will meditate on that uh analogy so um I'm not sure if this is too concrete a um an interpretation of of what you

00:51:05

were just describing Jerry but your question does bring to my mind a lot of interesting projects some of which I believe will be joining us at beta Works

00:51:18

um that are trying to [Music] that are there are already are proposing an alternate answer to where is that digital thing I'm looking at um beyond the sort of like

00:51:31

centralized uh Corporation owned not your data model that for better for worse is the only model up until this point that has at scale provided the sense of oh I know I know where that is I know what I also know like that will

00:51:45

that could disappear if the company goes bankrupt or whatever but at least I know for now where it is um and you know uh I don't know if it's obvious but a lot of these models do rely on

00:51:56

um web3 or a decentralized um technical infrastructure and value like economic incentive structure to keep these things alive and around in some immutable address and location

00:52:09

things things like and as ipfs the interplanetary file system Yeahs nfts all that kind of stuff keep going I think specifically ipfs is is at least being explored very

00:52:22

um actively as a possible model for um a way for me to put something on you know into this virtual sphere into this higher plane where everyone

00:52:35

everyone else can also access it can can um then uh incorporate some uh version of it or not even a version of it I can incorporate a view of it in

00:52:47

their own tool that of their choosing that either they built or you know that they um decided they want to use um and as a builder of of uh maybe something more like the middle the coming together the space where

00:53:01

um this information is then made accessible to a group of people I'm very very excited about all this activity um in this direction because otherwise my only alternative is you know using

00:53:12

Firebase or using AWS or using one of these um more standard models and then also you know at least offering a local first storage model so the user can

00:53:26

always like have access to yeah have access to a copy of their data if the server shuts down or they close their account or whatever right right not that it feels like through all of this there's a

00:53:39

there's a job there's a job description that isn't currently advertised or filled for that matter like there's there's some superb meta work to be done here I mean that was a lowercase n in the

00:53:52

original sense they're not copyrighted sense of that yeah a Meta Meta classic I think we need to we need to have a acronym or something like that now uh I hate that the appropriation of

00:54:04

important words anyway um which is and and who knows where this takes place and again that probably gets to the sort of structural questions about is it the job of governments is it the job of Academia is it the job of

00:54:16

consortia uh is it the job of Citizen scientists who knows but the the meta work that I have in mind is to say like okay the best new experiments right now are soft space and and this thing over

00:54:29

here and this really interesting project out of uh Slovenia and right like and so what what is the what's the what is a conceivable lingua Franca connecting all

00:54:42

of those like what's the interoperability layer both technically and cognitively that would make it so that you know Jerry inside the brain can meaningfully have a conversation with

00:54:55

you inside a soft space construct right so that the stuff can flow back and forth but so that you can also sort of see it in your own preferred relative context but with you know like with that

00:55:09

sort of well this is in this block but I can see clearly down Broadway in the next block is some brain-like stuff right and I can see how it connects and you know that's there's a philosophical

00:55:20

element to that there's a there's a kind of Standards protocol level element to that and there's a lot of kind of architectural and design stuff yeah I'm totally on board with the vision you

00:55:33

just painted and struggling with it because for me in my imagination it means that when I reach a particular Edge or boundary in the space that I'm exploring and it's clear that

00:55:45

on the other side of that boundary a different kind of tool is necessary how do I make an elegant transition into that other tool and then some of these things are power tools how do I get good at that tool or how do I get good enough

00:55:57

at that tool where it actually works right it's the difference between mcdraw and illustrator or photoshop I just never learned illustrator or photoshop because I took one look at them and said this is our cane and tear and like it's

00:56:09

great if you're a graphic artist and it's a money money mint you know it's a it's a money making machine for for Adobe but I'm not going to learn it but I used to draw all my own things in mcdraw and at this point I'm looking

00:56:21

hard I'm using excali draw which is like this little piece of Open Source Code that works with obsidian that I can use with markdown files in really intriguing ways to try to emulate what I used to be

00:56:32

able to do in 1990 by myself right and so so we're kind of lost in the middle there and then I love that there's a jobs there's a framework called jobs to be done yeah um which is really interesting it's like

00:56:44

you hire a milkshake to keep you satisfied for the morning because you're in your pickup going from job site to job site right your things and and jobs to be done is often often leads to unusual insights because you know you

00:56:57

hire a politician to get something done you need in your life uh and it gives you a very different perspective and I think you're right there's there's a job here to figure out and unfortunately it's not a job that

00:57:10

Silicon Valley is going to reward very well because it's not a job for locking down IP and milking it for all it's worth it's a job for Building some set of Commons conversations and Commons protocols and agreements and norms and

00:57:24

practices then it's a job for going out and talking to the bonfires on the horizon that represent the different communities working on interesting but different and siled parts of this

00:57:35

problem and encouraging to come play in the general purpose Arena because it's going to be better for all of us if we do and and that's hard I think I think there's a to the very

00:57:48

beginning to be back to the beginning of that paragraph your paragraph just now Jerry I think part of the answer is pseudopods pseudopods baby pseudopods I like that or slime molds yes well they should look

00:58:02

just like that like if you get to the edge of your domain or demean you should be able to like grab like just a chunk of the soft space stuff next door pull

00:58:13

it into your domain I actually think that there is a kind of software meta protocol an interop protocol by which systems could advertise to each other their their capabilities and their sort

00:58:27

of sounds fantastic right and so like pull a little soft space pseudopod in volitionally right it's not coming at you but you pull it in and you can play with it like I don't have to learn the whole thing but like wow I see I see so

00:58:40

I can make this a spiky thing and actually actually can I show you actually please do please do and while you're looking for the thing to screen share we haven't talked really about machine learning at all yet here and

00:58:51

clearly it pays it plays it can play a huge role and one of the questions I'm interested in which I'm going to ask in a future podcast episode here is hey what are the many different ways in which machine learning can help tools for thinking in particular Collective or

00:59:04

group you know multiplayer tools for thinking but one of the interesting things that I don't think is on anybody's radar is what you just said John which is there could be a a companion app that

00:59:16

says oh hey Jerry you're walking into a setting a situation a question a part of your inquiry where you might want to run a simulation or you might want to do

00:59:27

systems analysis using kumu or a kumu like tool or and it would know that you're passing a a boundary into a new domain and it would then know to coach you or help you or find an expert in

00:59:40

that domain and make those boundaries actually very fruitful because otherwise most of us aren't trained in the variety of domains that exist and we don't know that we've hit a boundary where there's

00:59:51

something that could help us solve this problem better we just don't even know to ask the question yeah right so so A system that can have some of that awareness and Coach us through could really give us a step function up and

01:00:04

one of my theories of progress and this has really played out a lot in software is that it's a step function it's a step function every now and then somebody invents a breakthrough breakthrough changes everything and then and then suddenly we're at the end of its S curve

01:00:17

and everybody's like gosh this feels really old but I have no idea what's next and then boom somebody invents something new and now nobody can can imagine the world before the iPhone which launched in 2007 right it's like

01:00:30

really hard for us to imagine a pre smartphone world and yet somehow people used to meet we used to like navigate in cars we like do all these things pre-smartphone but but so

01:00:44

so I think we're we're sitting right in front and we're talking about here the next step function of what this is and and it's exciting to be having this conversation it is for sure um I can share my screen if please go

01:00:58

for it and I'm sorry that was a much longer uh digression than I that I thought I was gonna no that was right at least your screen great I'm I'm we're we're eager for the step function and maybe it can be an even bigger step than than in recent time so what you'll see

01:01:10

in a second is uh purely a prototype kind of uh an illustration of some of the things that I just said but the idea was that um if you have uh

01:01:22

if you have the idea of kind of uh code permeability so that running systems can kind of jump through the air and land on larger display surfaces again always

01:01:35

with an eye to collaboration uh then you can enable a new kind of collaboration where people with individual systems on their individual devices can can physically and therefore literally come

01:01:48

together in a shared display and interaction space and if at the same time the the systems that they're running have this capability of advertising their own

01:01:59

structures and and their own capabilities then maybe some kind of automated minimal interrupt as possible even when systems haven't been designed to work together so let us see then

01:02:14

what we can see uh better turn the audio down right so this is um a thing called frog and you see that as each of these three

01:02:25

researchers approaches the giant display the the program that's running on unfortunately in this case his uh laptop jumps across and begins executing on the

01:02:39

big board and the tendrils there are are meant to indicate that there is you know the sort of transaction of meaning between these systems and your stuff follows you around so there's a kind of big spatial obligation that's fulfilled

01:02:53

here um there was a whole kind of big you know urban planning and emergency first responder backstory to this to this scenario um and again you know this is elusive

01:03:05

meant to be Elusive and and nothing else um but again highlighting the idea that uh that ad hoc collaboration is easy for people and with the right

01:03:18

instrumentation uh of of running executable code it ought to be minimally possible for digital systems as well it's just that no one's ever built one of these but I think it's accessible

01:03:30

right now it's really interesting uh the sugar OS that was on the olpc did a little tiny bit of not not the standing in front of a display and sharing stuff but you could intersect and interact with people

01:03:42

in the OS in ways that were pretty clever and then Jupiter Jupiter notebooks which is like a lab notebook thing that has a python runtime basically under the hood you could imagine that our browser

01:03:55

becomes more like Jupiter lab notebooks and and then that the code simply lets uh a simulation or something moved from screen to screen and be shared more

01:04:08

effectively and those are like really simple primitive examples of what you just showed but it's this this is a juicy uh juicy World I'd like to live in yeah yes yes please I mean do you think that soft space

01:04:21

could become also a meta environment could one of the things that I've um one of the main things I admire about your work and the way you work is the kind of dedication to rapid rapid prototyping

01:04:32

which is really unusual in the startup world where you're like I'll go and you get entrenched and you build hard and you know and then it's just sort of shellac layer after shellac layer but

01:04:45

your willingness to kind of build and toss part of it aside and build a new thing feels like that it if you in the best possible way institutionalized that practice you could give

01:04:58

you could give coders and experiment as a bunch of bricks so that they could do their own little evolutionary explosion and try out all of the experiments that

01:05:12

I think are going to be necessary to Define The Primitives uh as you said and the language and the vocabulary and the grammar and the syntax and the rest of it um which which is not something that any one person no matter how talented can do

01:05:26

or any one company for that matter because companies are a really bad structure for that kind of thing yeah I mean to you know to be fair the prototyping phase that um were in the

01:05:38

middle of right now um because of the the limitations you you just pointed to it's necessarily uh a bounded period so it started earlier this year it came out of my frustration

01:05:50

with um how much experimentation and how quickly I could experiment um I basically how experimental I could be

01:06:02

using a more standard software development product development playbook for the preceding you know number of years and it probably has to come to a close toward the end of this year and you know and and something that I'm I'm

01:06:15

very curious to hear your thoughts probably after this podcast episode um it's podcast recording on would be how to maintain um some amount of this sort of like

01:06:28

destabilizing open-minded open-ended play while also you know serving paying customers and solving real problems and having people who are using your thing uh hopefully to to

01:06:41

solve real world problems but therefore are also reliant on the Dependable you know sort of concept architecture and all that um

01:06:52

the question about whether uh you know or who is building an interop layer for or or all these um uh spatial software experiments you know I think ultimately that just that

01:07:05

does have to come from uh the organizations that have the resources and have the the power here which are the you know as of today the hardware

01:07:19

manufacturers so they would have to be the ones to say we're going to build the OS later such that it has all of these um capabilities that were that were hope you know that we're dreaming of in this

01:07:31

conversation um one interesting uh heuristic design heuristic that that this reminds me of when I spoke to someone uh many years ago who who was an

01:07:44

interaction designer at uh then Oculus um one thing they told me was uh a rule of thumb they that the organization had for when it came to how hard to try to

01:07:58

um or how much effort to put into making uh virtual behaviors [Music] uh visible or synchronized across

01:08:10

multiple users was um based on our expectations from The Real World so my expectation if I am sitting with you at a table and I pick up a piece of paper and I you know move

01:08:23

it like this my expectations that you would see that um that that transform transformation um but then from your point of view right it would be very strange if I would be

01:08:35

able to scroll a a sketch on this piece of paper and you couldn't see it even if I was holding it right in front of you and so that place is quite a high sort of threshold or quite a high standard on

01:08:50

these um immersive spatial interfaces uh to make everything into it to to make

01:09:01

everything meaningful sensible across different software right different across different um um uh paradigms even and so yeah somehow it has to map somehow or it

01:09:16

doesn't map at all but then you risk like me doing something which I think is important and meaningful and you can't your your version of the app doesn't even let you see what I'm doing right which is someplace with some points is gonna just gonna happen it's going to break because other software doesn't do

01:09:29

the magic thing that your software does or and vice versa isn't the first cut at that again to be completely literal that it's just that you know you're doing your thing and

01:09:42

you're you know your your floating orb of activity kind of bumps into Jerry's like the the minimal thing is just to literally see what he sees maybe backwards I don't know but you know like

01:09:56

and that can be done at the pixel level it doesn't even have to it doesn't require at that at that stage software interrupt so so you're right John that um the then the lowest the

01:10:09

lowest common denominator becomes uh what you would expect a physical version of this thing to to how it would behave um I'm already running into design

01:10:22

decisions where I have to make a decision a trade-off between what is um I'll give you really really concrete example say the three of us were standing in a

01:10:34

um soft space prototype and we had a few images floating around us and in the current version of the Prototype the images automatically rotates to to present

01:10:45

their you know themselves to you head on and that's very helpful because then you don't even think about whether I have to move to a certain place to see what the image is everything is sort of just displaying itself to you what do you do

01:10:59

when there are two people in the space or five 500 people in the space right so the default would be that okay things have a canonical rotation um which means it's sub-optimal for the individual viewers but then if I pick it

01:11:13

up and move it everyone and I I think I am moving it uh or turning it like this everyone else sees it being turned that way and therefore we can communicate via its sort of one standard spatial

01:11:24

semantic rotation right spatially meaningful rotation but then you you trade off against um some of the uh Power of of a software

01:11:36

environment which you know could show you things um it could adapt itself to make it more visible to you um and the Dilemma you're printing is is almost funny because um anybody who's used Google docs to

01:11:49

collaborate online with somebody knows that staring at the same document and being able to edit it with other people is really really powerful right and it's it's sort of more powerful than five people standing around a screen trying to collaboratively edit a document and

01:12:02

talking to the Scribe or whatever else so in some weird way when we're re-instantiating physical space with an audience or a group trying to collaborate online we've like thrown ourselves back into the stupid problem of hey who's the screen facing and how

01:12:16

does that actually work in the space which you've you've described one fix for that which is nice because because really I I love Google docs for collaborating on writing stuff it works really pretty clearly and there's a bunch of other tools now that know how

01:12:29

to do right I I think there there are simple at the first layer at first level there are there's a set of small a small number small integer

01:12:43

number of simple heuristics and approaches that can address the problems that you've raised you and I think I think all that stuff I actually have another video that I could show that

01:12:55

that that comes at some of it but let's not waste time with that like there's I think a little tiny bit of additional uh vocabulary like basically diacritical

01:13:08

marks that let you know whether a particular element is being universally seen like this thing you know you're seeing it backwards because you're on the other side right and so it doesn't

01:13:20

have the diacritical mark and then a version of it which you can toggle where uh now everyone can see that you're seeing this piece of the environment the landscape the the topology the

01:13:32

topography in a way that privileges you so just the like this unbelievable power as we've all been saying in the simple idea of co-seeing if we stand in a physical

01:13:43

space we look in the same direction I don't have to wonder if you're all seeing the Rabid Ocelot that's coming toward us right like we all know that and that's that is an incredible

01:13:54

cognitive capability in all of this junk it's it's a giant bunch of Merc like this junk right here that we're using right now if there were six or 16 of us

01:14:06

let's do Hollywood Squares or Brady Bunch or whatever it would be the simplest thing in the world for this software to assure that the layout is the same for everyone but it doesn't they haven't figured out that

01:14:20

little trick which means that even simple things like eye lines are always messed up right you can play the game where if you've got nine people on you say everyone point to Jerry and everyone's pointing in some other

01:14:31

direction like so that's that's Ground Zero then ground one I think with a little bit of visual and and geometric annotation you can make clear which

01:14:43

elements are being universally seen and identically seen even if in weird orientations and which are being seen in some way that privileges a particular viewer or every

01:14:56

viewer or which things are redacted because you're not allowed you know I don't want you to see my email even though I'm you know I've exploded my 2D screen and you're seeing all the elements yeah

01:15:09

I I think it's just it's just I feel like the word is permission right like you just need to take for yourself and nobody go and give it to you take the permission to just invent all that

01:15:23

stuff I'm I'm speaking to all of us not you know like I think yeah there's that kind of like we're we're under this very low

01:15:33

ceiling of of implied non-permission and we've all been crouched down so long that you know like it hurts and straightening up is gonna is gonna take a little doing but we're all we're all on floor nine and a half and Being John

01:15:47

Malkovich it's like you have to you have to kind of Crouch and walking around you're seeing through John malkovich's eyes which is really disconcerting but but that's a form of Co seeing in some weird and twisted way

01:16:00

um we've we've um almost eaten all of our time we're close to 90 minutes and I'm wondering what um what questions are lingering in your head and if you had a research agenda or a startup agenda what

01:16:13

would you propose just a some kind of a green field question uh to take us out of the call it can be either way like like what questions has this raised for you that would you like to see answered or uh if

01:16:26

you could set up an agenda for what to build what to do what would that look like uh so John the last thing that you said about taking uh about taking action

01:16:39

let's say um and not needing permission I think um I think the question I would leave this call with is how do you invent truly new things truly new and

01:16:52

good things not just novelty for novelty's sake but truly new and good things in the face of myself included people being quite lazy and

01:17:03

unwilling and unable to invest the necessary time and effort energy often to to learn those new things because I think John where you were you were describing there but like you know why

01:17:15

do we feel so pinned in by these like um these these ghosts that there's no one stopping me from from just doing it I think the thing that's maybe stopping me for example is this wisdom this wisdom that has been that has

01:17:29

accumulated that people um will like unless you make it super super easy for people to to adopt it's not gonna work you know and and so the question would be how do you actually

01:17:43

um work from first principles and develop things that push the envelope when humans are often the hardest part of the envelope to push

01:17:54

human's hard to push I okay uh so I will dock my comments with with that stuff in you first off I don't think you're particularly lazy in fact I think you're

01:18:09

doing some of the best work in the world in but he's trying to he's trying to build for lazia for other lazy humans yeah you know when I when I when I open my you know sort of like I don't know banking app I'm also not excited to

01:18:22

spend 30 minutes learning some novel UI for managing money right it's like I kind of wanted to just show me a bank account and hopefully it's like a large number if you had to do like a Pokemon go in order to get interest collected on your account that would be an unhappy

01:18:35

moment so just admitting that I am lazy when it comes to other people's Generations yeah well maybe if they sucked less you'd be more inclined to to deal with that and to that point you know one answer and it's only a tiny bit

01:18:48

facetious is to maybe put a bunch of new ideas inside video games right like that's sort of The Last Starfighter idea rather train people to to use the digital world in a new way

01:19:02

um not to fight battles like in that movie but to do to you know to do mutually beneficial things I think so my my outro here is to say is to wonder what the new human

01:19:15

organizational structures which is a big theme for me these days are going to be need to be but then can be implemented like what what are the new accessible human organizational structures that will let what you

01:19:28

asked about to happen right um Bell labs and Xerox Park are gone and the behemoths that should be making the new ones of those and we all know them

01:19:41

uh aren't right like the whatever the political Financial whatever even kind of philosophical structure no longer exists so that's uh governments I think have a really

01:19:56

important role to play but it's not it's not at the the kind of sitting in the lab working on stuff level it's more at the kind of you know high level

01:20:08

structural approach and funding and you know darpa-ish maybe so what is what is that thing because I want to do my work cheek by gel with

01:20:22

your work even if you're a startup and I'm an individual or I'm I work for a hospital but I have this you know like hobby building this stuff or whatever it is right like how do we get together

01:20:33

here's the big word Community or around these ideas Because unless you know unless we can pull out the control rods keep the density really tight like get these ideas together and not working in isolation

01:20:47

then I think the best we get are sort of fizzle yield blips where the the extant Mega structures like acquire little ideas and extinguish them or absorb them

01:21:01

in some useless way into something else amazing how quickly things are extinguished yeah so yeah let's build that that new megastructure I have you know I'm a little bit obsessed with

01:21:13

Black Mountain College oh fabulous there's there's one of those there are several of those to be built but there's probably a really good one to be built with with these ideas as one of its kind

01:21:26

of core tenants so I was just learning about the Nordic folk schools do you know about them yeah um they're quite amazing like 10 of the population of most of the Nordic countries went through a Folk School

01:21:39

which was kind of like a citizenship School and one of the reasons we think that that the nordics have like some different sense of quality of life and policy and all those kinds of things is

01:21:51

is this thing uh which was invented back around 1870 1850 somewhere around there and then grew and then shrank and is still alive a little bit but it really isn't isn't dominant in any way but it's also

01:22:05

in that same realm of Black Mountain College these are these are spaces within which people were able to really be generative and um they were also really playful spaces

01:22:16

it's a little bit uh this is maybe a bad analogy but Bell Labs during its golden age which was full of scientists and Geeks inventing things like the transistor but was playful also in in in the sense of

01:22:29

Highly capable people who would just sort of meet and mingle and have lunch and suddenly run off and and invent something new and we don't have that but John as you just said basic research has been gutted uh corporations uh are are

01:22:42

like have thrown most of that investment overboard colleges and universities are teaching for basically they're becoming trade schools everybody's focused on stem and what will get me a job

01:22:54

and the humanities are being deprecated uh we're sort of losing people who care a lot about art and Technology even as I can point to hundreds of Art and Technology movements struggling to stay alive around the globe but they're

01:23:07

really struggling to stay alive in many ways right and then where they succeed they get co-opted by the industry of Art and suddenly and suddenly there's too much money involved and it's like all screwed up and it's not the thing you

01:23:18

actually want to happen either so what about that you know that's another black hole that's absorbing energy and and Innovation so somehow in the middle of all this we need to stake out curate or

01:23:30

or somehow stake um of vibration and energy and a tractor that brings people in without co-opting them without destroying their models or their goals or their incentives but

01:23:43

enhances the collaborations of the groups and as a result leads us back toward Reinventing the things we've been talking about on this call if that makes sense here here okay so let's just kill

01:23:56

one all right I'm ready I'm in physical location as well yeah exactly exactly it's it would help to have a drishti or a or a platform or

01:24:09

a grounding Point somewhere in our real world in meat space good old-fashioned meat space uh which we have to background in as well we have to sit here oh you mean real life right yeah yeah real life

01:24:22

funny we have to we have to coat classic everything I think it I think at this point we're just going to call a halt because we I could do this all day with you all this is just uh phenomenal I really I really love this call and uh

01:24:34

want to do this again so I will head toward our our exits and then we can figure out what's next thank you all for listening to tools for thinking a new podcast that just might help you with your thinking if you're

01:24:47

part of a startup in the sector please knock on our door at betaworks.com Camp otherwise keep listening to these podcasts and join me in thanking uh John Emilio for a fabulous conversation

01:24:58

really appreciate it [Music]