Video: Machine Learning for Thinking (DocDrop)

00:00:00

foreign [Music] ERS linkers and mappers we are very happy to host a series of conversations around the topic of tools for thinking our longer term goal is to spark a

00:00:17

diverse connected shared memory that will help us make more make important decisions together our near-term goal with these podcasts is to interest startups in being part of beta Works upcoming accelerator think

00:00:28

Camp betaworks is a New York city-based incubator and accelerator they've run seven camps before on topics from Bots to synthetic media and voice interfaces you can find out more about beta works in the think camp and this domain by

00:00:42

going to betaworks.com camp I'm Jerry mikulski your interlocutor and obsessive mind mapper our topic today is machine learning for thinking it's going to be really cool it's our first Peak

00:00:55

Under the Tent of machine learning in this little sequence of podcasts and I'm really excited for the topic and my guests are Alice Albrecht and Sam arvestman we're staying in the A's for this episode uh Alice is the founder of

00:01:08

recollect and Samus with flux capital and as I tend to do here I will ask you each to just sort of talk yourself into the into the topic uh Alice how did you find your way to to recollect and uh and

00:01:20

this tool this crazy little tools for thinking space yeah well thanks for having me first of all um so my background originally um I was an academic actually I was in

00:01:31

cognitive Neuroscience so I have spent many decades now thinking about mine um some what about thought uh but lots about the mind um and then I've spent about a decade in Tech then after that uh doing data and

00:01:45

machine learning work in various settings in various ways um so recollect for me um when I sort of set out to found this company was a way for me to uh kind of

00:01:59

combine my interests and what I thought would be in the future what was coming next so I spent about a decade doing machine learning work in Tech um in various different roles um and then

00:02:10

bringing those together when I set up start a company I was really looking at something that was a combination of the things that I've been really interested in for a long time and also where I saw the greatest potential moving forward

00:02:23

um so for folks that haven't heard of recollect I'm the founder of that company and what we do is we are trying ultimately to augment your memory and creativity as humans um we do that to start by taking all the

00:02:36

things that you read notes you take um and bringing those to you automatically when you are ready to work with them so if you are ideating so putting your ideas together if you want to understand what you know about a

00:02:48

topic more deeply um and better yet if you're creating something with that so if you're writing a newsletter or tweet even we want to be there for you to let you utilize your

00:03:00

electronic memory as it were um and so you know bringing all these things together recollect is really uh for me a passion and I think that is a direction that we're heading more generally in

00:03:12

terms of people meeting their creative thoughts to be ready for them thank you and you um you were on a great panel for us at the render session a few weeks ago in New York um and I felt like you were giving us

00:03:25

kind of a Cook's tour behind the scenes of what uh what somebody who's deep into the algorithms and the capabilities of the machines uh already assumes as possibilities limitations and all that

00:03:37

and I want to come back to that for a second but I think that one of the influencing factors for recollect is the extended mind thesis uh the idea that that there's sort of more than just what happens inside one Cranium that's

00:03:50

manifest in many different ways do you want to just Riff on that for a second yeah I mean the term uh can be really specific right so Andy Clark did a lot of this work uh and we can think about sort of that work earlier honor in the

00:04:03

realm of psychology um um but it's been used a lot outside of that as well so I feel like at a general level what we're talking about when we talk about an accepted mind is um having even you know as simply as a

00:04:15

sheet of paper and a pen that is an extent of mine right so you write something down you've externalized your memory you know it is there for later there are lots of criteria for that um but um some of the criteria are that you can

00:04:27

reliably use that again right so um I could write something down but then if I throw it away in the garbage it's really not an extent advice anymore so um when I talk about an extended mind at the highest level what I'm talking about is

00:04:39

um are we able to not even just offload but sort of co-work with our thoughts um outside of our own heads and how much of that is about collaboration between humans and how much of that is just the

00:04:52

extended mind for the individual Jewel out into media and artifacts for you for me it's more for now it's about it's more about the individual but if you think about the way that we're even talking right now

00:05:04

um I am putting my thoughts and ideas out verbally and you're picking them up and your friends like them kind of we're having this back and forth so I do also think a lot about um in conversation or as you read people's work right if as

00:05:17

you are influenced by other people um and you put that back out into the world I think about it in that sense too of sort of externalizing that process thank you makes makes great sense um Sam you're with Lux Capital it sounds

00:05:31

like a light in the capital world but I'm not sure that you're in the lighting business so uh um yeah so Lux is uh it's a vendor Capital firm that invests in like emerging tech companies or um I guess

00:05:43

companies that are kind of playing with science fiction ideas trying to kind of bring things from the science fiction world into science fact um and uh and so my own um I guess my own path into this I I also got my start in in Academia I have

00:05:56

a PhD in computational biology I got my start initially in evolution evolutionary biology but then quickly moved more into understanding complexity science kind of trying to understand big complex systems whether or not they were in biology whether or not there are

00:06:09

social systems or technological systems um there's and obviously all these systems are different in their own special ways but there are ways of abstracting the details away and actually finding certain mathematical and computational regularities behind

00:06:21

all these and so um and one major area is that of network science trying to actually understand the relationships between the pieces of the system and how they all interact and so I kind of moved more into Network science got very

00:06:33

interested in interdisciplinary research trying to find commonalities between different areas um and so after kind of spending time in Academia um spent some years some time in the foundation World um and then eventually found my way into locks and so I'm I'm

00:06:46

Their scientist in Residence and basically my my role is to survey the landscape of Science and Technology and find areas that could be of interest to Locks and sometimes finding companies finding people to build companies around

00:06:58

connecting the areas I'm exploring to our portfolio companies that we've already invested in and sometimes just engaging with the public through writing and speaking or connecting um to various communities that are maybe not as connected to the Venture world or

00:07:12

Lux and kind of bringing them into the the orbit of locks but what that means though is kind of on a regular basis I spend a lot of my time thinking about a new thing every hour or every few hours and I'm really interested in kind of where

00:07:24

um the uh the intersection of ideas happened and kind of where different domains and Fields collide together and so I guess so in terms of my interests and a lot of these things I guess it's both professional in the sense that I'm always trying to find new and

00:07:37

interesting things to try to connect them together and kind of bind uh find different areas but it's also from kind of my roots in complexity science and network science which is how can we better use kind of the science of how

00:07:49

different ideas are connected or people are connected and navigate this kind of high dimensional space of ideas or people or or papers whatever it might be and so I think a lot about how can we build better tools for Serendipity or

00:08:02

finding new interesting ideas and both in terms of helping me find interesting things that could be overall language blocks but also just because I think we need better tools to allow better Innovation and more Innovation to actually happen and I just have to say

00:08:15

it is such a juicy moment in human history to be doing what you're doing and both of you I mean like like we have superconductivity of ideas Publications people we're sitting here on a nearly zero

00:08:28

marginal cost very nice resolution and audio quality Zoom call that would take for granted now which you know a couple decades ago would have been like how are we gonna do this um and and then there are so many people doing interesting work where the

00:08:41

experimentation is their time but there's these insane power algorithms just lying out there in the field so to speak for you know for people to tap into so partly I wanted to I want to start with a little bit of just what is

00:08:54

what is currently under the the machine learning machine intelligence umbrella what are the kinds of uh sort of uh both from a technological perspective but from a like what problems are people solving and the one the ones that are

00:09:06

easily visible are deep learning in different indifference and senses uh some piece of that is turned into text uh to image generators so the the dolly too and and other sorts of things uh

00:09:20

happening and now there's like a half dozen of them changing what we think of how how painters and and artists are going to work and if you're a painter or an artist you're scratching your head going wait Doom is this bad

00:09:32

photography was going to be bad but it turned out okay um you know before a little digression but before photography like the representation of a battle was some painter who was told what happened on

00:09:44

the battle and painted a heroic image of the battle collapsing in time and space because they wanted to show when when the hero died and is bleeding over here in the arms of his lieutenant and over here the charge of the Cavalry and back

00:09:56

there the castle that was stormed none of which necessarily sort of coheres or makes sense and and it isn't until Matthew Grady takes pictures of the battlefield at Antietam with bloated bodies lying in ditches that the

00:10:08

American public sees those in in in newspapers and goes oh wait a minute like like battle is not glorious and wonderful battle is dangerous and and disgusting and there's lots of dead

00:10:20

people here and I'm sitting here thinking okay that was one of the spillover effects of Photography which is a while ago right 1860 50 uh somewhere thereabouts and now we're talking about I think things that are at

00:10:32

least as dramatic and and different and interesting but but I'm curious what's what's in the tent and you know there's also text generation the image generation text generation text comprehension similarity seeking what what how do you look at the the bag of

00:10:45

tools what's in the tool kit pack up tools is very interesting um I like that you started with deep learning because I feel like people forgotten about it now they're like oh yeah I don't know um such a shame too I really like deep

00:10:58

learning well yeah I mean I think an interesting it's not even that deep learning doesn't exist anymore it's not like we've replaced it with anything really it's really uh imbued and all these other things but um the problem with deep learning was you needed so

00:11:11

much label data most of the time it was a thing that was difficult for people that didn't have lots of computational resources to actually use um it even as it did um finding a use case for it that was

00:11:24

really going to be applicable inside of an organization I mean uh we were talking right before this like I was at fast forward labs for a while um we worked a lot with big companies and little companies too um to help them understand what they

00:11:35

could do with things like deep learning um deep learning is definitely still under the tent um I think that there is uh you know anybody who's been on the internet recently or Twitter or these places

00:11:47

probably Community listening to this has seen the really really interesting demos with natural language generation natural language understanding and now the multimodal models right where we're

00:11:59

getting images and text and we're crossing over those things I very I personally am very excited for the multimodal piece it's something I've been put like you know advocating for for five years or so trying to get

00:12:12

somebody really excited about it um as somebody from the sort of kind of Neuroscience space it feels really obvious that we start to combine these things um so those are certainly Under the Tent I think

00:12:22

um there are lots of other tools and techniques that don't get as much attention I guess um and I think some of that is research interest

00:12:36

um so the people that are creating these things then some of it is like how much of it is publicly available and easy for people to pick up that are not necessarily machine learning researchers you know today I think for the average person trying to implement something

00:12:49

with machine learning we still have less complicated models that you can use for things like um you know fraud detection you're not necessarily you don't need a dolly or something like that so there are lots and lots of use cases where people are using machine learning today

00:13:01

it's invisible to the end consumer um but it is powering a lot the recommendation systems are something we've become completely accustomed to but that's generally a machine learning model under the hood it's really interesting because I have a

00:13:15

funny connection to this field in that my first job in Tech was back in 1991 and I joined a little company called new science associates in South Norwalk Connecticut and I joined their AI service and I became their neural

00:13:28

networks analyst uh and and deep learning is called Deep because it's multi-layered neural networks and back then what I was doing was explaining the difference between rules-based systems and neural networks and how this one can explain to you

00:13:39

which rules are fired and that one over there cannot and this could become a problem but blah blah blah and there's all sorts of really interesting issues that come out of that and back then there were other things like fuzzy logic and case-based reasoning and there were all these other

00:13:52

things that didn't really make it through right but but we were writing about them we were interviewing the companies that that did them and I don't hear much conversation now for example about rule-based systems either they got so mainstreamed and they're just sort of

00:14:04

vanished into the background or they didn't make it or I'm missing something like what for example what happened to normal what we used to call Expert systems my sense is that was I mean I could be wrong about this but I feel

00:14:16

like IBM Watson included some sort of expert system piece within what they were doing IBM was like a polyglot mashup of a bunch of different kinds of systems so in in myself and like more broadly there's like yeah like the whole

00:14:29

like symbolic approach I think the symbolic approach and obviously um it's not on the rise right now but I think a lot of people or I wouldn't say a lot of people but my sense is that there are a number of people a number of researchers who think that to kind of

00:14:41

take some of these ml techniques to kind of build Next Level some of these more symbolic approaches need to be combined with them I feel like and Gary Marcus I think he often writes about those kind of thing so I would say he's probably the most prominent proponent of kind of

00:14:54

advocating for um or at least recognizing the importance of symbolic approaches alongside some of these other things um so I think it's it's not gone but it's maybe kind of either been

00:15:06

incorporated into other things kind of in like alongside like the Watson approach which my sense is yeah it was just a lot of things that kind of threw a lot of things together and kind of hoped it all worked out um and then there's other people who are kind of beginning to revisit some of

00:15:18

these um Alice is your sense that does that Accord with with your sense of all these things I think so and I I'll take I like whenever I'm on a podcast to like make a bet and then you know years later I'll just do it again and see if I was

00:15:31

right um I'm sure I made a bet on the multimodal piece um I think we're gonna see a combination of these is people like right now we're talking about just machine learning how can we apply it and like kind of put it into a system um there's a whole group of people that

00:15:44

like to talk about AGI and I don't think we should take that left on this podcast um sorry um but I think that when we start like you know Gary Marcus I think

00:15:55

um tannenbaum's Lab at MIT like there's these people that think a lot about like you know how do we have symbolic systems on symbolic systems um uh at sort of like a less technical level like how do we start to like have

00:16:08

things that we can bake into these things that we already know right like physics right like there's laws of physics that we can start to put into these models that don't need to be learned by like lots and lots of Trials necessarily right like so we can refine

00:16:19

those but there are pieces so my my take is that like yes I agree with uh your account Sam of that this is sort of where we're at now I think um we will see a Resurgence of of adding things in intentionally that we know to be true in

00:16:33

the world um and I think some of that will be driven by the folks that really are trying to shoot more towards an AGI um these would be the same kinds of folks that when I like teach this as they would teach a baby right they're

00:16:44

looking at the ways that babies learn they're looking at lots of different Fusion algorithms um but they have a really different Focus I think which is very interesting in the machine learning space from someone who's like I want to create art

00:16:56

with this or I want to like make a better risk model um yeah I would say in one area that I think is also beginning to um become like have I wouldn't say a Resurgence because I don't think it actually really went away but the whole Space of

00:17:09

evolution evolutionary computation um it was very popular in maybe like the 90s early 2000s um and then with certain other like the Advent of other techniques it kind of couldn't quite compete in certain optimization techniques but I think now

00:17:22

with just so much more computing power and data and processing power um they are able to do certain things and so and especially the interesting anyway so one is I've seen this kind of been being used in tandem with other

00:17:35

um ml techniques I think it's or sometimes for optimizing hyper parameters and things like that um but in addition the interesting thing with evolutionary computation is that it can um at least in the genetic programming

00:17:46

kind of version among others it involves manipulation of symbols and so you can actually I mean pour in a whole and go back to like what you're talking about with physics you can just pour in a whole bunch of data and say it's like provide some equations that actually

00:18:00

explain this kind of thing and my senses in certain cases there have been situations where these systems have popped out results and the scientists are not entirely sure like the the formulas work but they're not entirely sure why they work

00:18:13

um so it's kind of so it's an interesting thing where there's I mean sometimes the Deep learning techniques have a certain amount of non-explainability but even sometimes it's about more symbolic approaches might also have that as well which is uh kind of kind of interesting I I don't

00:18:26

know where to take that but it's intriguing and and I think you said in your intro you you at some point were on sort of evolutionary biology yeah so um evolutionary algorithms and machine

00:18:38

learning are nowhere as or the last time I looked into them were not nearly as complicated as Evolution probably is um but they take some ideas from that which is really interesting um and I think the sort of Applied place

00:18:51

I saw this that I think Uber was doing this for some folks at Uber were doing this um and maybe out of the University of Boston and Texas um this is a few years ago now one thing to Loop this back to our conversation

00:19:03

though with the evolutionary algorithms that I really loved when I looked into this um was that uh sometimes they'll introduce sort of like random or they'll bias the models towards like really a

00:19:16

little bit more aberrant like the Next Generation that it reads um for me I think when I sort of went through how the algorithms were designed and why would you do that right like if

00:19:29

you're trying to get to a specific thing it's not a great idea to introduce lots of you know chaos into the system um but if you're looking for more Creative Solutions that's very interesting to me um so like the idea that you could have

00:19:42

this style so in evolutionary algorithms you have generations of algorithms um and so as somebody who is really excited about you know creativity and machines Beyond even like painting

00:19:54

things and making text um I think it's very interesting to start to uh think about how we would bring that in to evolutionary algorithms and we think about things

00:20:06

um yeah just generally I I love that about them yeah so I I think you might be talking about something like the like novelty search kind of algorithms like that I kind of are and where it's like the um I guess the uh the thing you're

00:20:18

selecting for yeah yeah yeah the objective word rather than it being um some sort of yeah some sort of objective it's just more okay is this new in some Dimension or some way of explaining it and so like um especially

00:20:30

so Ken Stanley and Jill Lane before I think yeah so they they worked on some of this they have a book um called why greatness cannot be planned and their whole idea is like when you are searching in some massive High dimensional space then it becomes very

00:20:42

hard to say okay here is a specific thing we want rather instead the idea is to just optimize on interestingness or novelty or creativity and the idea is that when what that will do is we'll

00:20:53

we'll create a set of Stepping Stones um that can then all be recombined in some sort of productive way and then we'll will yield kind of this like like expanding boundary of interestingness that then will hopefully get you eventually to the objective that might

00:21:07

be very hard to actually otherwise um describe or formulate um so yeah and I think yeah it goes very well with like tools we're thinking of like how do you optimize for interestingness like find the kinds of things that people want to

00:21:21

actually be shown or the kinds of things that are going to be most useful for actually generating new ideas so yeah no you're you're yeah that intuition of going in that direction was exactly spot on I love that and I love sort of the the

00:21:34

mixing and remixing of ideas and Technologies that's happening that you're both really fascinated by because I I love that as well I want to scroll back for a second and ask a question explicitly that we kind of looked over the the fence at which is do either of

00:21:47

you think that computers or software can be creative what does that mean for you I guess I kind of I would say yes um in the sense that a creativity I think is like a combination of like like there's like novelty and surprise but

00:22:01

like it's like novelty and use or maybe like surprise and usefulness or I'm not really sure of the right combination but um maybe it's one of those kinds of things like you know it when you see it um I do think they can be creative at the same time though I would say and and

00:22:13

there is kind of the the um the view of like oh like these mlts and they're just like stochastic parents they're kind of like recombining things exactly yeah exactly at the same time though I mean that is certainly one really important

00:22:25

of like one important component of creativity which is recombining and remixing so I would I would not say that uh machines and computers can be creative in all the ways that humans can be creative at least at this point but

00:22:38

certainly in some of the ways that humans are creative I think we're there and and that makes it and I would say it's the kind of thing where I yes and machines can be creative at the same time though like whether or not that's a cause for worry

00:22:50

um and this might be hopelessly naive but the way I kind of view it is like and humans have still made these systems and so we can kind of take a certain amount of pride in that kind of the way I view it is there's um this Yiddish term called noxus which is kind of like like vicarious Pride or joy and the

00:23:03

accomplishments of like your kids or like when they um I don't know when they have their Bar Mitzvah or get married or go to college like it's not your own um not your own accomplishment but you can still take a certain amount of pride and joy in these

00:23:15

kinds of things and I feel like um we should be having more for our machines in terms of their creative abilities um I I would say they're maybe creative still in somewhat simple kind of like

00:23:28

stumbling ways I'm the same way like maybe like small kids are kind of creative but they're also like when you kind of and like I've had situations where you know I like my kids will come home and like show me something that seems really really creative and then when you start asking them like you kind

00:23:40

of realize that they sort of took something that that I didn't know about and then kind of just like tweaked it a little bit um and so it seemed like it was entirely novel but it was less novel than I expected that being said they were still being very creative and I don't want to

00:23:53

shortchange what they were doing and I feel like that's the same kind of thing with machines where yes we can caveat away all the accomplishments in these of these machines and these computer programs but they are still at some fundamental level being creative

00:24:06

um I I would not necessarily say they're more creative than humans um but they are certainly creative in some big big sense well thank you Alice yeah um I loved your answer

00:24:19

um I have uh what sound at the surface like some conflicting ideas about this I guess um one thing I would say just to figure out what you just said Sam was uh you know are they more creative than humans or not I think that's maybe the wrong

00:24:33

question it's like they have a totally different kind of creativity and that's fine like when we're trying to design these I don't think the goal has to be let's get to human level X or Beyond it um and so when I think about these

00:24:47

algorithms and creativity um I I think when we Define creativity the way I look at it is from a psychological perspective or a psychological testing perspective because of my background

00:25:00

um and so I kind of can't tap it but um you know in cognitive science or in Psychology when we think about creativity what they're really when people test your level of creativity with their testing for a couple of things like can you come up with novel

00:25:12

ideas so those are things that maybe didn't exist before either for you or for everybody else which is an interesting distinction um and are they useful so are they grounded in something right like I could create

00:25:25

an algorithm that just took two things and squished them together and maybe there's millions of them and they just squish them together but that's not creative if it's not useful it doesn't have isn't grounded in something um not that all creative acts need to be

00:25:38

useful but there's a context and there's something that the either the human is trying to convey if they're doing some creative act um uh or when we talk about machine creativity like we're missing this intense piece right like usually there's

00:25:51

some human behind it that designed the algorithm of course but also is deploying it and putting it in the setting right like even if I use these new algorithms like stable diffusion I'm writing a text um right I'm giving something to the machine

00:26:04

um I think where it starts to get really interesting for me so if we take that kind of working different definition of creativity um I think we could program machines to do that type of creativity right and that's something that I think about a

00:26:16

lot for our product and for the work that I'm doing now is um you know if the name of the game is to help people come up with you know creative ideas can we like you know create this artificial Serendipity can

00:26:27

we take things that are connected but not quite right on on the nose when we bring them to them um but the way they're connected I think is really important um so machines I feel like um if they're being creative they're programmed to

00:26:40

connect things in certain ways and humans connect things in lots of different ways like uh I think things happen around the same time do you have some like disapparent memory from a long time ago that you're connecting and you're bridging all these things that's

00:26:52

much harder for machines to do so um I think machines can produce outputs that would meet the criteria for human creativity um but I think that right now we're not

00:27:05

quite there where we would say oh they're just being creative on their own it's more of the symbiosis of you know human creating a thing machine doing you know creating some versions of that helping them

00:27:16

um and going way back to the comments about kids um I have children also and I feel like sometimes I'm like oh my God you're a genius look at this crazy thing you came up with and um but there is usually something

00:27:28

underlying that and I think that's true with all humans where you know they they had some spark of inspiration uh they attended to that thing like somebody said a word or they thought of something and then they're kind of building on that so when I think about creativity I

00:27:41

also think of like um you know it's building on other pieces that are already we're pretty good at remixing what's in front of us sorry yeah I was gonna say yeah there's all these raw materials yeah and yeah to what you're to your

00:27:53

point Alice like the question is whether or not the computer is remixing these things on its own um without a prompt um for something that actually is kind of creative like feels like useful or valuable in some way and you're right

00:28:06

we're still not quite there it's much more of like a human machine partnership as opposed to just kind of allowing these computers to go off and be creative on their own and and sort of to just walk back and pull Evolution back into the conversation nature evolution

00:28:17

is creative it's just that it's stochastically creative and the things that don't flourish die off uh the things that flourish look really creative and in fact create new niches and or dangers for us Etc et cetera

00:28:30

we're watching that at work with the with the pandemic we've just been sitting through in Zoom like locked into little Zoom rectangles and then there's a whole bunch of interesting research over in Evo Devo and all these different areas I don't know how much of it is

00:28:42

making over making it over the fence into large language models and uh deep learning and so forth but I'm very intrigued by that because like you both I love the multi-disciplinary the Nexus of innovation is coming from lots of

00:28:55

different places makes me really uh really happy um are there places you've seen where we are overestimating the capacity of machine learning are the things you've read where like

00:29:07

no we can't do that and if so what what what areas would you mark off with yellow tape yeah so I uh one area that's really I don't guess Salient right now um is is these large language models and

00:29:21

is all of this text that's being generated and um I think there is in it depends on the person and really the background of what they're trying to get out of these but I think it's easy

00:29:34

to overestimate what those large language models can do and what they are doing um uh I think that they're fantastic but I think that it's probably going to lead

00:29:46

us astray if we think oh they deeply understand right they they're the understanding is something we don't understand because it's it's not super easy to inspect these models those

00:29:59

models are created in isolation so um when humans learn things right we're in an environment so we're getting information in we're updating things um reinforcement models you know follow this Paradigm

00:30:12

um but large language models on their own don't really um they learn regularities there's lots you know if it's a mass language model it's predicting the next words that lots of data that's fantastic though human probably could read all those books and

00:30:24

all that text and pull out all that data it's crazy it's crazy and it's fantastic um but I I worry that we're overestimating in that sense like how much these these uh

00:30:37

models really understand um and what's possible from that standpoint like if I believe the model understood everything like oh my God there's so many things I could do yeah like the understanding

00:30:48

piece I think is still something that um it can simulate a feeling of understanding somewhat reasonably well but yeah we're not there yet and it's and so and because of that we have to make sure we're using these tools like kind of with that understanding of it or

00:31:01

without with the understanding of the lack of understanding in mind it's like for example I was um I'm very interested in kind of like like alternate history stories um where it's like oh like what if something were different in history and uh and so I

00:31:13

started um I was kind of feeding some like summaries um I think based on like Wikipedia summaries of like novels and books and movies and things like that of all certain histories and giving them to gbt3 and saying okay now I'm going to give you a new a new what if and kind of

00:31:25

seeing what happens and um and like the summary the outputs of like oh like one what if Jimmy Carter were re-elected or whatever and kind of seeing that kind of thing um and the summaries that came out were not terrible but they also yeah

00:31:38

belied this like clear lack of understanding of the world that being said though because these large language models have this like massive Matrix of associations of different concepts it's actually really good for helping

00:31:51

um I guess in terms of like Divergent thinking you're kind of like helping under like saying okay here are the kinds of things I need to be thinking about so when I try to understand the world what are the features that are actually Salient because too often I feel like humans are we have so many

00:32:04

biases that we forget all the different things that we should be considering and um these machine learning techniques and they have many biases in their own ways but sometimes they can help you kind of expand the set of things that you should be considering and so I think um even

00:32:17

without that understanding they can be useful for helping your own understanding but you're right you don't you want to make sure that you're like very clear on the fact that they cannot understand these kinds of things and and we humans are such good salience detectors that often are salience

00:32:31

detectors throw away things that we ought to be paying more attention to we dismiss evidence that doesn't fit our our filters or our biases or whatever else there's plenty of research in that area but but it's really interesting and these computers are just kind of

00:32:42

innocent they're like woohoo uh the the thing I point to when when talking about the creativity question uh is Alpha zero uh where alphago beat Lee sedal the world's top-ranked uh go player awesome

00:32:56

then they said well let's not let's not train this thing on all the go games we can find you know played by Champions let's just give it the rules a go and luckily the rules of Go are 19 Lines by 19 lines black and white stones and

00:33:08

here's how you kill something so they're really simple the domain is incredibly constrained but then there's a chart that blows my mind where you watch the performance of alpha zero blow past Lisi doll blow past alphago and there's a

00:33:22

piece at the end where it's where it's we're above whatever anybody's doing um and that I I see that right there that that's creativity within the boundaries of the rules of Go it was inventing things that

00:33:33

ages of go play you know maybe four or five thousand years worth of go players hadn't discovered to try to do and that that's very interesting and go Scholars are busy studying those new moves and trying to figure out what that means and I was just before this call looking at

00:33:46

the the cheating and chess controversy with Magnus Carlson resigning from a chess game and what's happened with uh chess software and all of that with freestyle chess it's sort of this sort of like a mess in that space that's really really interesting and I think

00:33:59

good fodder also for the kinds of things we're thinking about um let me step over for for one second um because like you're actually like neuroscientists um

00:34:11

I have this feeling in neuroimaging and all the reason for all the research on how we can see what we know and all that that we're really working with uh 8-bit graphics and like it's really Minecraft

00:34:23

it's worse than Minecraft images that we're getting the brain is very loosey-goosey and things move or things actually shift around in the mind and stuff like that so so if I were to point to places that I would rope off I'd be

00:34:36

like we really don't understand how the mind works is nearly as well as a lot of these papers coming out would seem to make us think or want to claim to is that sound reasonable I'm going to deconstruct before it yeah okay that's

00:34:47

less my old advisors you know come back from come at me um which paper are you talking about machine learning papers you're talking about cognitive science papers or Neuroscience papers or like I'm talking about what mines what winds up hitting

00:35:01

the news in science digest or wherever else that oh look we can we can read we can mind read now or we can place memories you know into people's heads or we can whatever there's a bunch of things like that yeah yeah some people I

00:35:13

mean like if somebody said oh I can I can read your mind or I can put false memories in your head let's say like that's science fiction that's not happening right now um I think that uh you know when you

00:35:25

talk about like eight bits or pixels I mean part of what comes up for me is like the resolution of the images right so we're seeing these images of the brain that resolution is they're called voxels and I mean we're trying to image this system

00:35:37

um and what that represents is a population of neurons so already we're at a place depending on how good uh or how much resolution we can get where we're getting information about a population around so we're doing non-invasive they're Imaging

00:35:50

um so it's it's correct in the sense that um you know we're not getting to the individual neuron recordings unless we're doing invasive techniques and then we are but we're getting them a really small part of the brain and it could be

00:36:03

something to have the patient you know that our patient would like that um so I think that uh the other piece of what you said was you know uh you kind of use brain and mind uh interchangeably

00:36:15

and then they're really separate so I think that distinction is really important but we think about as the mind um you know I am on board with saying that it gets contained within your body really like I don't believe uh anything

00:36:28

further than that but but um understanding the way the mind works is very different than understanding how your brain works um so I think those two come together obviously haven't was in cognitive

00:36:40

Neuroscience that's bringing those things really explicitly together and saying okay um these models of the mind can be informed by what we learn about the way that your brain works but they're informed about lots of other by lots of

00:36:52

other things like your behaviors and psychophysical testing and things like that um so I think that we have a ways to go it's what's very interesting for me is the interplay now between these machine

00:37:04

learning techniques and Neuroscience um and that unlocking a really new things that we can do and when you add language into that like back you know a while ago when I was in graduate school

00:37:18

um I took a course in I think it was called computer modeling or something at the time um as it's like 2008 baby but uh you know it was taught by a psycho linguist and we learned about these like language trees and I thought oh my God this is

00:37:30

horrific I don't think I would ever work out anything to do with language because it was so arduous um but when we think about combining again cross-disciplinarily right we've got these neuroscientific techniques that um unlock a lot with neural Imaging

00:37:42

so one really interesting thing that happened at home about a decade ago now um was that we started to be able to predict what people had seen right so if I am feeding all these images to you and

00:37:54

I get a pattern of activation this was back in the day um then we were using this sort of multiloxal pattern classification to say oh we can guess based on this narrow image what image you saw and that was

00:38:06

really that felt like it unlocked a lot and I think it's gone from there but if we take the sort of how machine learning is interacting with the neuroscientific models get and we're getting better data Those sensors get better but also now with the language because I think it's

00:38:19

very interesting um because we can sort of do some things around okay you know language is a little bit harder I was more on the vision side like it's a little bit easier to understand in terms of the biology of it right we have a

00:38:32

visual area it's pretty well organized we have a good explanation our working theory around how this stuff flows through language is much mass here in my mind um so when we talk about thoughts like somebody like interjecting a thought into your mind like where does that

00:38:45

thought go um it's a complicated network of different areas that come together to bring up and it's something that I thought it's really a memory or the production of something um so I don't know what your question was

00:38:56

no but something to do with like the resolution of these things I think it can look really like it's at a sort of a gross level that we're like you know it's really rainy and we don't really know much um I think we know a little more than that but I don't think we're

00:39:09

in a place where it's you know tomorrow I'm gonna say yes uh and I and I'm very much an optimist in this space I love neurotech but like I don't think we're gonna be uh saying like I'm gonna

00:39:21

alter this memory for you um and just like fix it a little bit Arnold Schwarzenegger will be very sad about that yeah um and thank you for indulging me in these excursions because we're still

00:39:34

sort of talking sort of Neuroscience and algorithms and and all that kind of thing and I was trying to explore the boundaries and sorry for my sloppy use of language you're totally right I was just flipping back and forth between brain and mind but I'd like to take us

00:39:45

sort of uh toward the entree for this conversation which is where does all the stuff that we've been turning the soil in effect tools for thinking and what are the opportunities

00:39:57

what are the places where you know we could start Alice with with sort of the pieces that you're using and finding success with and recollect but where else and I kind of want to want to figure out what does the Horizon look like for the things that are possible

00:40:10

and and I think there's lots of interesting things I know that in my daily use of the brain I would love to have a smart assistant sitting next to me going oh Jerry looks like you just put a book in your brain would you like me to tag it up with metadata as a book

00:40:22

and it's author as an author and find you know her other books and add them form for you and I'd be like thumbs up you bet because right now all of that is manual for me and and that that's a kind of assistant you know

00:40:35

machine learning informed assistant that would be completely brilliant but I'm interested in more about this space partly so that we can find the other players and and see how this stuff all fits yeah so

00:40:46

um I think the the world that you drew out Jerry where you have this assistant and it's helping you do these things like um you and I have talked about this before but for everyone else listening um I think the path the path I think we

00:41:00

should move is a little bit off from that um like I think that with machine learning and with um not even just machine learning but you know interfaces that are like changing and all the time um we have an opportunity to move Beyond

00:41:13

an assistant that would be able to tag things for me and sort of like organize them more nicely um which is great I think it's a stepping stone I think it's a really um it's a useful thing to have

00:41:26

um but I for recollect and otherwise sort of want to bless past that um and have a system that can be a lot more flexible so there's no hard coding right if you think about the mind and how they

00:41:38

inform these tools right like so you have lots of things that are associated for you but they aren't Heart Association so far I stick two things together they're just stuck there right like those change over time as we have more conversations even I have had a few

00:41:51

um uh all of those associations that we think of as like okay if I tag this and look at this thing and I organize it this way then I'll be able to find it maybe um yeah there's evidence against that in some cases

00:42:04

um but what I think the power like a powerful thing in these new tools would be utilizing the fact that these models can adapt to sort of changing environments right like do things are changing they're evolving over time it's

00:42:17

learning more information um and it's changing representations in ways that maybe the humans don't need to see all the time so I don't have a physical piece of paper or I'm not looking at two books and saying I want to see those two things together

00:42:30

um so I think there's a lot of opportunity there um and that's part of you know the of what we're building it recollect and what I'm interested in there is can we start to automate some of these pieces that I don't think people need to

00:42:42

actually do they're just a sort of question in the middle that we've used in these physical spaces um but then the other side of that which I think will you know continue to be really interesting is

00:42:54

um can we not only organize those books for you Jerry can we start to like suggest how they're connected right um in this vast space of possibilities that we talked about you had said I

00:43:07

think you know people are really good salience detectors and I would say yes and um they're easily distracted and it's hard to keep all those things in mind so that goes back to your memory so

00:43:18

um you know if you're working memory is limited if I put these two books together it'd be great if the assistant could also kind of suggest again how they're related why they're different what other things um so I think there's a lot of

00:43:30

possibility in the tools for a thought space coming back to that for um us to use these algorithms in ways that are not just automating the process that we're already doing right now

00:43:43

so I'm I'm super helpful for that area um I agree entirely and and thank you for sort of shining a light on the fact that my use of this tool is so manual that I would be happy to have minimal assistance to do something like tagging

00:43:57

which is all which is the example I gave and it's it's clear that there's so much more possible and it's clear from where I'm sitting chugging away with this simple tool that I'm not seeing that because because I would just like that

00:44:10

next little step and what I really want is what you just did which is okay okay like like that's great we can do that what's next what's out there and Sam if you want to sort of take a swing at framing this please yeah and so and I think of it a lot from like the

00:44:23

perspective of um like when you have and as a society huge amounts of information more information than anyone can ever consume and even within the Sciences like there's and not only can you not know

00:44:34

all of science you cannot even know your own little tiny specific sub-specialty that you might be trained in um there's just too much being generated and so um the question becomes what are the kinds of tools that can be used to kind of help navigate this help make discover

00:44:46

degrees and so um I think one of the earliest examples of this of like someone trying to Grapple with this was actually uh it was in the mid-1980s uh it was the information scientist Don Swanson um he wrote this paper called undiscovered

00:44:58

public knowledge and the idea behind it was um it was kind of like he started as a thought experiment essentially he said imagine there's some paper in the literature that says a implies B and there's another paper somewhere else in the literature maybe in your specialty in somewhere or somewhere else entirely

00:45:11

different it says be imply C because the literature's so big chances are no one's ever read these things and so even though it might be true by combining them that they that implies C no one knows this and so at the time he actually used like manual search tags I

00:45:24

think in like Medline databases um and then found actually he found a relationship between um I think um like consuming fish oil and actually helping alleviate some sort of circulatory disorder then he published it in a medical journal even though he

00:45:36

had no medical expertise and since then though we now have ways of automating this like and so actually um in the past few years there's been um there have been papers that have looked taken I literature in a specific area and then done sort of this kind of

00:45:50

thing word embedding and kind of put it all like all the knowledge you know it's in some sort of latent space and I said okay where are the different concepts where do they lie but then not only that can we actually figure out um

00:46:01

like what are like when these discoveries were made like when these advances let's say in Material Science were made but based on these embeddings could we have actually made those discoveries earlier so like if like we know when the paper for some Advance

00:46:14

happened but based on all the previous literature if we had poured it into these kind of ml techniques could we actually look at that space and find where these gaps were or where these connections should have been I mean it turns out you actually can do that kind

00:46:26

of thing and then so there's that there's people have actually also done similar techniques of helping to try to predict um what kind which papers are going to actually be the most impactful earlier than it eventually became clear

00:46:38

um there have actually been a number and there's um this uh non-profit organization called Ott that has is working on a tool called illicit um like kind of like umht ought okay yeah

00:46:51

like like what should like what ought to be done um and then so the tool elicits um builds it's based on kind of like gpg3 and some other large language models helps you actually navigate the scientific literature and I'm saying

00:47:03

this all from the scientific literature space but I think these kinds of tools um are really valuable and so and one of the things that illicit does is you can ask it a research question it will show you a whole bunch of relevant papers to that research question but then using

00:47:16

these large language models will actually help summarize the findings and like and help but like summarize defining relative to the question you had and so um we're beginning to see a lot of these ml techniques help us stitch together

00:47:28

all the knowledge that is out there that no human can ever possibly know and actually bring it all together and so I feel like like we are helping discover all that undiscovered public knowledge and so I'm I'm very hopeful for for what the what the future holds here

00:47:42

um I love that scenario and I think that properly applied a lot of these things are are going to be really fruitful in the next couple decades I mean it sort of has to be that way and then in the back of my mind I'm hearing the the the

00:47:54

song skull crusher Mountain by Jonathan Coulton which is about uh you know evil mastermind who you know welcome to my layer on here on Skullcrusher Mountain I'm like man I hope I hope that the the the darker side of people out there

00:48:07

doing stuff don't get there first because this is It's really powerful right yeah I hope we don't get to the darker side of things um I will just give a tiny plug for our listings I've tried it and it's pretty neat

00:48:19

um uh for even putting together these ideas to create more things than Alyssa I'm sure um so that's really like already here today thing too right yeah we're yeah we're there and I mean and yes

00:48:32

to your point Jerry like yeah a lot of this like the ability to kind of speed progress and Innovation um is very much a double-edged sword and yeah and at least my hope is that we are

00:48:43

going to make good advances faster than people can kind of use them for bad uh and I think that's more of a hope than necessarily A truism because and we don't know but yeah but I I'm hopeful that these kinds of tools will help us

00:48:57

kind of better Advance the the frontiers of knowledge rather than just exactly democratizing mad scientists or Evil Geniuses or whatever I love that and it sounds like we're sort of lifting the lid on a topic for a future uh podcast

00:49:09

episode because I think the idea of good AI friendly AI ethical moral whatever you want to call it how do these things work um both when you send off the machines and they start to do things autonomously but also when their tools for thinking

00:49:22

coupled with humans uh and you know what does that mean how does that work uh all together so so we can set that aside what other kinds of uses do you see for machine learning and the tools for thinking space what other what else

00:49:35

could we just describe here that that's a possibility I can talk about sort of a farther out one because it was something I was really excited about still very excited about it um

00:49:48

so I think the other thing we can start to think so we I talked a little bit about how do we start to connect this information automatically how do we kind of push creativity in that sense um that again is already here right like it's already functioning in our product

00:50:00

I think it's like this is very much now moving forward I think it's going to be very interesting when we start to bring in other types of data so if we think about wearables if we think about information about the individual as

00:50:13

they're interacting with this information and as they're manipulating it I am very excited for that so um one of the very early versions of recollect uh was a little fighter out there

00:50:24

um and was scaled back a little bit um but it involves sort of a you know could you bring in a brain computer interface I studied attention um so could you start to understand what people are attending to in these uh when

00:50:36

they're thinking about things how do you preference that information um in these models um in these systems um but also how do you like catch when somebody's ready to do creative work or ready to do stop work

00:50:49

um and how do you start to push them into that state if you want to or if they want to and that gets into ethical territory but um I think there's a lot of opportunity in that sense once we start to combine really data from

00:51:00

different places so um we can do this without a wearables to some extent today um if we think about interactions on your computer what what kinds of things are you reading what are you attending to are they like what's the sentiment of

00:51:12

those things like um but I think that'll be a big unlock when we get to a place where someone these wearables are more ubiquitous once we get that data and then I think the other place that I'm very excited about again that is sort of like an initial

00:51:25

version of Reckless but like when we are using um spaces outside of these rectangles I'm very annoyed by the 2dness of this uh I feel like especially for uh abstract things when we think about like

00:51:39

I want to think about something if it's hidden if it's buried in a file structure or I can't really see it immediately it makes it harder to work with um so I'm excited about some sort of AR coming on that people can use more

00:51:50

easily um again another thing I thought would be here already and it's taking a little bit longer but um in that sense you can start to think about okay if I'm if I'm building a tool for Thought can I start

00:52:02

to put my memories places or things I want to think about in space um like I would a book or like I would uh pieces of you know paper that I put together um can I start to alter the environment

00:52:14

to facilitate that kind of thinking so if there are associations that I've made and I've started to put those together um um and then they're the obvious piece of like can I manipulate things that are not hidden now they're not just in this completely flat thing that I can't

00:52:28

really reason about as a human who didn't evolve for all the stuff to be in the flat thing um so I'm excited about those two areas but those are really I think going to require other technological advances

00:52:38

before they're adopted vibes me I love those ideas brief note on the second one you you just posited which is one of the earliest of these podcasts that I recorded with this was with eliu Shen

00:52:51

Burke and John undercover and YuYu is the founder of softspace which is busy working on AR uh xrvr that those kinds of things and John undercover was a name I didn't really know but uh you've seen

00:53:04

his work because in Minority Report when Tom Cruise is doing the gesture interface to find to use precog information to find pre-crime uh John invented that language and it's a

00:53:18

functioning language like he he can actually you know multiple people can approach with different devices which sync up with a big screen which then can be moved around can be shared to other screens and across devices and you can

00:53:30

zoom in and open and query with with the language that looks like you know like that it's super interesting so we had a good time with that I'm not sure we answered that question fully because we did like an hour or 90 minutes or

00:53:43

something like that so I I think there's lots more to do in that area I had a call I had a coffee with an engineer from Google Glass after Google had deprecated glass and I was really

00:53:54

interested in the brain and a heads-up display because if you if you could feed my brain my geolocation and identify a couple things in my field of view people or things there is an awful lot you could do that would be super super

00:54:06

interesting right and we rapidly figured out that that glass was never going to have the resolution to be able to display text like that in front of me but but then we had a really juicy conversation about what's the potential

00:54:17

of augmented reality in space when you've got some kind of large memory with with a lot of stuff that matters or that could be relevant to bring in and I love that particular area um other zones or Sam any comments on

00:54:31

the the ones that Alice just mentioned oh those are great I mean I I know I would say I mean I guess um maybe one one near-term thing which I guess is already happening which is kind of interesting um to kind of like continue on with hgpt3 and some of these

00:54:44

language models is the way in which people are using them to collaboratively write um like stories and fiction and things like that when I I so whether it's a pseudo right or some of these other tools I think there's some really interesting things happening there

00:54:56

um longer term although I guess there's some near term stuff happening as well is around um not just like tools for kind of helping us navigate um I guess large bodies of knowledge but

00:55:08

also kind of helping tools to help us actually learn those areas and like and so if there's I mean helping essentially a tool that would kind of adapt to the way in which each of us learn as well as knowing the body of knowledge we already

00:55:19

have in order to learn some new field of study or whatever it is I think that'll be really interesting to see and you see that there's already a hints of like things around like space repetition um in terms you're just actually learning a new language and making sure you're actually kind of learning that kind of

00:55:32

stuff but I think there's a lot of potential there in terms of using ml there's the scene in The Matrix where Trinity is in front of a helicopter and she calls tank and says I need to I need to know how to fly this and downloads the the capacity to do that I'm like I

00:55:46

want that I want to be able to do that that sounds great that's I think it's near a term though I mean like if we look into this oh well okay so I'll posit one solution to that problem and then maybe it's wrong if

00:55:58

someone actually tries to work it out uh but if we have all this information about what we're consuming or what we know right um so we have some what's called scaffolding right so we've got some basis for what somebody might know

00:56:09

generally and we know that one of the ways people learn best is when you apply something or you attach something to some structure in their minds what they already have right so if I am trying to teach somebody something they're definitely you're absolutely right like

00:56:22

there's different ways that people learn right you can adapt things to be um you know more in the audio more video whatever there's lots of ways that people learn best um and if we could say take some text

00:56:34

and generate a video out of it great a person like gets something out of it but I think even and a lower hanging fruit really is okay so we know this person knows these couple of Concepts probably a lot of

00:56:46

Concepts those concepts are a little bit fuzzy they're not necessarily keywords but we know that they have some information about those can we see how far away this other new thing is and try to see what the bridge is like what are

00:56:58

the things generally and then like language models would be excellent for that um so I think that's the way we can start to tailor learning today if somebody wants to build that um in a way that would be really powerful like of this other sea of

00:57:11

information if I want to learn something new about like I don't know cat species um like there's lots of information out there it's too much I can't figure it out like okay find the stuff that's nearest to what I already know and then like use that as the bridge I was gonna

00:57:24

say it reminds me actually there's a paper um that was co-written co-authored by a friend of mine a number of years ago where he was looking he was taking uh different scientific fields and looking at um the jargon like like the different

00:57:36

kind of keywords and terms that are used by different domains and then what he did was they created this essentially like some sort of a topographical map of different fields and so it was the kind of thing of saying okay let's say you want to move from I don't know

00:57:47

theoretical ecology to molecular biology or whatever the examples are I don't remember exactly um you would it would kind of give you a map of saying okay you wouldn't necessarily go directly you would go through these other fields because those are the ones that have the most similar language and so I think what you're

00:58:00

saying is exactly like there's a a lot of similarity there in general how do how do people learn what are like how can we figure out what what kind of Concepts what kind of jargon do people already have because and because jargon

00:58:12

is actually a very problematic thing because it's like you might have the exact concept you need but it's a very different term and so you can't search for it and you don't really know even what to look for and so helping having some tool that can kind of help you

00:58:24

navigate and say okay this thing actually means this other thing that you already know about but you don't realize that um can be very powerful I love that along the way for what was what you guys were just saying um I was thinking wouldn't it be cool if

00:58:35

you could have um some machine learning basically tell you what are multiple different connective paths between these two sets of ideas because it's one thing to say what is the shortest path what is the quickest way what is the what is the obvious Bridge it's another thing to say

00:58:48

hey did you know these things are connected in these six different ways because sometimes it's the Obscure or the less known path that actually opens a a deeper connection or the right kind of answer or whatever it might be and we

00:59:01

don't have time permutationally to do it our brains usually can't handle like setting things aside in that orderly away and that there's a whole lot of reasons why it would be hard for a human to sit down and do that unless that was

00:59:13

the specific goal of their of their Endeavor but but machines could probably do a very nice job of that I think that kind of comes back to those evolutionary algorithm I think we made a little bit earlier around like what's the objective of this right if we if we

00:59:26

say the objective is to find things that are less common right let's find these paths that are not the most well-worm path right there's not the most number of connections between these but maybe there are fewer um and then I think you're right about

00:59:38

humans too right we we're unable to keep so many things in mind and we think we can we really can't um we need an extended mind of some sort don't know why yeah um but yeah being able to plate all

00:59:50

those all of those and and cut through a lot of the cognitive biases we have and our own attention right like if I'm really focused on this thing I'm I'm really not I mean to be creative you actually have to be mind wondering which is a really interesting piece I think

01:00:03

um so if you look at studies of creativity um when your mind's wondering it's not super focused and so if we're really focused on solving a problem really focusing finding the solution um we really cut out a bunch of the world necessarily

01:00:16

um so we don't even even if we could keep all those like nine paths really in our mind and we were like super genius in that sense uh depending how long the patents are um yeah I don't even think we would get there so I think that's really another

01:00:28

place we could bring machines about anything else coming up for us any other um I mean I I had made a little list before the call there's uh from partly from what we were knocking back and forth the sort of the ideation idea uh

01:00:41

there's the idea of increasing Serendipity or Divergent thinking Divergent ideas uh there's how to navigate through gigantic bodies of knowledge and make your way through which we've talked about a little bit

01:00:52

here uh suggesting uh connections or summaries or insights uh that that's actually pretty uh a Chris Pedregal uh who was your your interviewer in the panel at render uh the thing that caught

01:01:06

us when we were talking with him sort of earlier was he had been taking audio notes transcribing those feeding those into gpt3 and then querying gpt3 about his own notes and gpt3 was doing a very

01:01:18

nice job of summarizing things for him and I was like I was like oh damn and he just hacked that together the way Linus was just sort of hacking things together for his own pleasure right super

01:01:29

interesting and and that settles over into this idea of having space to play with these things because I think a lot of the insights and a lot of the important work will be done by people who were just messing around uh just

01:01:43

taking some of these tools and recombining them and putting them in some doing multimodal in some way that's unexpected and suddenly it starts coughing things up right um I think we have some of that ahead of

01:01:53

us as well there's an old Mac app that some people still like like Stephen Berlin Johnson is a big fan of Devon think and what you do with Devin think is you feed it a whole bunch of documents and it suggests connections it basically has like a relevance or

01:02:07

similarity ranking I'm forgetting what the algorithm is that does similarity ranking but um and it's you know been around forever yeah I think the summary thing is an interesting point it's something I've been thinking about a lot in terms of

01:02:21

these tools so I can just take this that direction for a second uh so we have this product now that is really powerful and that it brings all these things back for you um that are related it can bring things back that

01:02:34

are inspirational that's amazing uh I suggest we made that true I feel like I was struck by this problem again of oh no there's way too much information I've got all this stuff I've gotten on my board what do I do with it now

01:02:47

um and so uh I am very interested in this idea in like uh probably maybe a little bit of a principled way of trying to figure it out but how do we start to

01:03:00

get this summary or this gist or this um like tldr for people around all of these disparate pieces of information which is not an easy problem it turns out

01:03:12

um so I've been experimenting with some things but it's really about um you know that is a place where we talk about we want to get to human quality in a sense right let me check a summary of something or like a takeaway

01:03:26

um we don't want a thing that's like what in the world is this summarizing is both urgently important and very dangerous Dave Snowden has a bunch of philosophies

01:03:37

about data collection and the use of data and Analysis and one of one of his insights is you don't want people in the middle summarizing and collapsing the the raw data to sort of tell stories or whatever else you really want access to

01:03:50

the raw data because the summarizing tends to really warp and bias things but at the same time for reasons you just explain really nicely we absolutely need something to digest filter

01:04:01

synthesize you know deglaze maybe these huge bodies of knowledge one of my amateur design ideas about this is what I call levels of Zoom and so one of the reasons the brain

01:04:14

works really well for me is that it has a pretty high level of Zoom but not the highest the highest level of Zoom conceptually is where you're up looking at the big ball of twine of entities in their interconnections and I have never

01:04:25

done that with my brain data don't want to not interested there might be a couple nuggets there but but the next level down or maybe it's a couple levels down I don't know but the the level that I'm at that I really like is that it's

01:04:38

just words or phrases that are easy for the mind to parse because they're either up down or left right and that's the only set of relationships that that are possible in the brain that turns out to be a really nice organizing principle for how to how you parse visual cues and

01:04:52

how you make sense of things and so I get this very deep sense of orientation and local structure from that that I don't get from other kinds of tools and then if you pop down another layer of Zoom you're seeing summary paragraphs this is a little bit like a stretch text

01:05:05

that Ted Nelson invented years ago and then all the way at the bottom is full bodies of text that might be linked up might have context might have have tags or whatever but the full text and when I when I see the full text and I zoom out

01:05:18

a little bit and there's no other other Q levels at the zooms I get lost right away because all blocks of text look alike right you start to just see a forest of blocks of text and it's like whoops I just I just lost my ability to get

01:05:30

around I don't know what's where and so how we markup navigate uh mess with that and some people are highly visual so they'll like the visual display other people are not at all they

01:05:42

just want show me all the things with this backlink and I'll be happy right which is sort of Rome mem Athens research all of the outliners with backlinks are kind of in that category and a long way of asking a question but

01:05:56

what do you does that sound possible what are your sort of approaches to solving these issues um Etc I'm not sure I have a good answer for that I I think I mean and certainly one

01:06:09

of the things that like it's I mean there are many different exactly what you're saying like there's many different ways that people kind of process lots of different information and like figure out kind of how to get the um get the lay the landscape I'll also kind of understanding the details

01:06:22

um and certainly I like one of my Linus's techniques of like trying to kind of like put words in either color them or kind of like provide them in some sort of like meeting landscape I think is really powerful

01:06:34

um because exactly what you're saying Jerry of like large blocks text you can't really get a sense of what's inside them um without any like just at a glance because it just looks like Words

01:06:46

um and then without reading the entire thing uh and so um yeah and I I guess I guess one of the things I'm not sure this is quite answering your question but like one of the things that it makes

01:06:58

me think about is um summary is right it can be it can be dangerous but I think it's so it's not necessarily summary that is necessarily the key it's more about how can I

01:07:12

um I guess to limit and like and and limit the search space or reduce the search space of the things that are potentially relevant and still allow people to then say okay and rather than some massive

01:07:23

High dimensional space that I can not possibly examine um saying okay here's the area that you that you should be looking in now of course it might mean because of whatever technique you're using you're going to miss certain things that could be

01:07:35

relevant um but at the same time I need to be honest with ourselves that if we were doing it manually we would also be missing things and yeah and just just because the algorithm is not perfect does not mean that humans in turn are perfect we

01:07:46

are all imperfect in our own special ways and so um but I think if we can have tools that allow us to limit the search space and then allow us to kind of see what is the underlying data that can be very powerful um so I'm not sure that quite

01:07:59

went in the direction that you expected but I would say that that is the way I kind of think about some of these these sort of things yeah thank you um that does uh harmonize well with what I was thinking else um I don't know if we talked about this

01:08:11

before Jerry um so uh my team all knows about the octopus or we try to turn it into a forest analogy because no one likes the octopus um when I was starting or thinking really about recollect

01:08:23

um I was really frustrated by this idea that I didn't have the ability to change levels and sort of the way that you're talking about but not in levels of the document which it sounds more like you're talking about

01:08:36

um I wanted to first of all create like a a algorithms to pull out actual concepts for people and not just the words but then once you have something in place um

01:08:48

to have the which is what we have right now in our product which is like the underlying sentence or paragraph of information you've got some information here up a level for me is in a sense of summary so if I have a couple of these and they're connected I want to go up a

01:09:01

level I want to just get like a summary of that going up going down for me that was not the full document going down for me is actually the individual sort of uh machine generated Concepts

01:09:15

um and so I have all these drawings of this and I tried to explain it to the early team when they were joining and I was like this is what I need it's an octopus it's gonna be an AR at some point and they're like thankfully they join the team still

01:09:27

um but I was like I need to be able to be at these different levels for different things too so I need to be able to take sort of machine generated like what are the components of this paragraph even and I need to be able to

01:09:38

move over to another not octopus the tree or Branch right and be able to like see that's a mycelial root system with the the flower the paragraph address are actually flowering fruiting bodies of

01:09:51

the mycelium known as mushrooms those are actually paragraphs see ah how do you do that in software and now like there's this great regular episode about this but now there are like in the

01:10:02

canopy of freeze there's a whole nother canopy There Were Trees growing in the canopy of trees way up high all these plants it's amazing um so so maybe even more like that but um

01:10:15

yeah whatever natural reference makes sense for people it's really this idea for me like it's about uh how like it really it's in some sense this is a level but it's not the level that whoever wrote the document made it's

01:10:27

mine yeah exactly and and there's gonna be there is a massive UI challenge here about how do you how do you become aware of what's available how do how does ML machine learning assist you in making

01:10:39

sense of it and adding to it um how do you switch views when you need to switch views like there's some data is naturally three-dimensional or you can change in dimensional you can swap in dimensions and then you should be in

01:10:52

AR and standing in the data and looking around for mean word spaces concept spaces don't work very well in Dimensions like I need kind of need flatland to to negotiate them usually but not always so how do I know when and

01:11:04

where in Apple's old knowledge Navigator video they have three little iconic assistants on the side that had different characters different personas I don't remember what they were let's let's call one like a professor professor and one was like a researcher and one was something else and they

01:11:17

would each kind of raise their hand or make a little gesture when they had something to say to you in this simulated user interface not a bad idea not a bad idea because they could represent your preferred ways of looking

01:11:30

at what you're doing and then when one of them says hey hey hey right now you would pay attention because they likely would have something really useful for what you're trying to get done we're getting near the end of our of our time and I just wanted to offer both of

01:11:43

you like like what questions would you love to see answered um where where might we all go next it sounds like all three of us are pretty hopeful and optimistic about this space

01:11:54

and happy to be alive at this moment because there's just so much going on so I'm I'm I'm good on that um but what does the agenda what was there maybe the research or the design agenda feel like for you

01:12:07

I mean so for me I I would I guess I would just go back to like how can we handle the increasing like burden of knowledge um both in the sense of um that uh there's like so much to know but like it's also the fact of like how

01:12:19

do we learn all this kind of stuff so that we can make advances at the frontier of knowledge okay and there's um there's some papers talking about like the death of the Renaissance Man the idea that like like no one can know all these different things because we

01:12:31

now have to specialize more and can we now use these kinds of tools to kind of overcome that and still allow us to not necessarily be Renaissance people but at least allow us to stitch together lots of different domains of knowledge and if

01:12:43

we can do this within some sort of kind of computationally assisted way I think we don't necessarily have to be stuck in like hyper specialization um we can begin to navigate lots and

01:12:56

lots of different domains of knowledge in a productive and fruitful way as opposed to just a sort of pillow talked way so that is the area that makes me hopeful among many many different ones love that

01:13:07

I love that too I feel like I get this image of like not flailing around and like you know you can't be you can be a polymath I guess today but there is way too much information um so like yeah that'd be lovely I love

01:13:20

that future um for me you know research agenda what I want to see next what I'll make true to some extent happen next in the World um I

01:13:32

I am really interested in this creativity thread um I don't think the applications we have now are the end of that and I don't know that the I don't know what paths

01:13:46

I'll take so I'm interested to see sort of how those evolve um but I am very interested in this idea that we can take um sort of personalize as an individual your knowledge right like there is too

01:13:59

much stuff in the world you're exposed to some things um you get some piece of that world can we help the individuals to sort of get that information to them which is what we're working on right now but then

01:14:12

um in the sort of like co-creation space um what are we going to see in terms of not just text generation of like oh I could like have it write a thing for me um how is it going to help us to sort of

01:14:25

like push our ideas forward I guess so um I love your learning ideas Sam and I think that's really awesome uh I I sort of want to like have a thing that pushes my it helps me

01:14:37

to sort of push my ideas forward um but also helps me to utilize the knowledge that I already have I don't really have access too so um

01:14:49

tbdm with the research agenda will be exactly for that um but I'm excited about this co-creation space exactly me too um what you were just saying Alice reminded me of a video that I posted it

01:15:01

looks like back in 2016 it's called why I do what I do I think what I'm about to do really quickly is in there and I I basically say thanks for clicking on it um I I basically say

01:15:13

um leibniz might have been one of the last people alive who was a polymathon who understood all the disciplines of his day and kind of had his head around everything anybody knew in Western Europe at the time and I picture him as like standing in the middle of this fast

01:15:26

plane where where there's disciplines forming up and then disciplines form up and they basically um start to get competitive and then there's too much to learn so here's mathematics here's biology over here is

01:15:38

the humanities and poetry and arts and over here you know all these different sorts of things form up and we start to get the academy and and uh all the all the ways in which we try to tackle the disciplines and the way that a new

01:15:51

person in the field makes their Mark is they go out to the fractal Leading Edge of their discipline pick a facet of the fractal face uh you know the the seam and then say no it's this way and if

01:16:04

they're really lucky can have a big effect then their discipline shifts toward you know that sort of thing like Margaret Mead had a big influence France Boaz had a big influence and they kind of turned their discipline a bit but still mostly the the Leading Edge is a

01:16:17

way of absorbing new ideas and not shifting very much it's a way of almost defending the disciplines barriers and then I think the part that all three of us like is that it interdisciplinarity and one day it dawned on me that maybe leibniz is not standing on a plane he's

01:16:30

standing on a sphere and that all these disciplines are sort of pie slices going away from him but we've passed the equator and they're beginning to converge and I was having the experience in talking to biologists and neuroscientists and

01:16:44

computer scientists and philosophers I was having the experience of having similar conversations with different jargon and different Framing and different backgrounds and so I I started getting this mental model of oh my gosh we're seeing this convergence of ideas

01:16:56

and things and so forth and then this little explanation begs the question so what's over here and then I have another amateur theory of History that's the thing that's over here I it looks to me like yin and yang and I'm borrowing yin

01:17:09

and yang from taoism and I'm going to overload it a little bit Yin is generally a feminine receptive Dark Earth Energy young is is generally masculine uh positive active bright

01:17:21

energy it's not that all men are young and all women are Yin but every healthy entity needs to have Yin and Yang in balance and what happens what's intro the interesting things that happen are at the inner interface of Yin and Yang

01:17:33

there's a creative tension there and my little amateur theory is that we've been suffering from a young overdose worldwide for 300 to 3000 years that's somewhere along the line Yong one

01:17:46

and we ended up in a command and control male dominated like if you scroll back far enough it's matriarchies everywhere everybody understands how to nurture the commons there's a whole bunch of other ways of living together on Earth and

01:17:58

being just fine thank you very much and we killed those off basically Young when it won at pursued a scorched Earth strategy and got rid of all traces of Vienna could possibly get rid of luckily many of them have survived and we are

01:18:12

now in this like very interesting era where we can see sort of what happened we're trying to piece the the pieces together but I think that we're busy renegotiating how yin and yang play in our lives and we're busy trying to

01:18:25

figure out what is the next platform that Civilization is going to run on is it crypto dials with nfts I don't know is it Zuckerberg's metaverse I don't know I don't think so but it's going to

01:18:36

be something there's no promise ever as humans evolve that the next regime is going to be healthy for address or better like we've been through so many just dysfunctional ones and and my thesis here is actually a very uh

01:18:49

pessimistic one is that first 3000 years possibly we've been suffering from a young overdose is not a good look for humans um but this whole little image of mine is like why I get up in the morning it's because I think that the Curiosity we're

01:19:03

showing here in this call and the interdisciplinarity and the ability to find how these things tie together for good is what's going to help that thing come about and balance out I love that I love the introduces the

01:19:17

pulmonary piece of it too um I think one thing that I'll just very quickly say on that front too is like you have two people here who went an academic pass in a really specialized field

01:19:29

um and ended up following curiosity and passion to other places for other careers and doing other things um but I think that is less common than the alternative of like

01:19:43

oh I'm gonna do this very specialized thing and I will make a market this very specialized thing that I'm very interested in um and so I think some of the shifting uh I mean is really at the institutional

01:19:55

level like having like I had to find this path of my own and it's not an easy one um but sort of like staking out a way that I could make what I want to see true in the world meant for me like I

01:20:07

don't know this seems like the best way for me to be able to accomplish that even though there's a lot of stuff that's very difficult about it um so I think if we are to make this shift sort of in a different direction or to more inter interdisciplinary place

01:20:19

I'll make a little call for like creating those spaces where people can be more interdisciplinary but also deeply curious um and the the rub there is always capitalism I guess where it's like how do you make that actually into something

01:20:32

that's real and productive and like you know contributing um but yeah I I appreciate that on the call you've got two people that could have gone the other way and I sense that I

01:20:44

like like like I hear it in your stories in in that sense exactly as you just described it thank you yeah I just I I would add to that like I I think I mean when you're saying something like spaces for this kind of thinking um like when people want to do research

01:20:57

kind of knowledge creation um they're often shunted into like a few different kind of modes of like academic or industry lab or whatever it is but it took a few points and some and what is really a very high dimensional space of potential institutions and organizations

01:21:10

and we need to explore that space more broadly and I and I think you're beginning to see new types of organization institutions arise but we need more of that and uh and especially ones that allow kind of more interdisciplinary

01:21:23

um thinking to uh to actually uh be valued so yeah I totally agree with it absolutely thank you um but I want to thank you both for for this conversation this has been super useful and interesting and fun too fun

01:21:35

besides so thank you um and uh thank you all for listening to tools for thinking a new podcast that might just help you with your thinking if we're lucky if you're part of a startup in the

01:21:47

sector please knock on our door at betaworks.com Camp thank you so much for for paying attention here [Music]