Select text to annotate, Click play in YouTube to begin
00:00:00
a top executive at the heart of Google's Deep Mind efforts to advance artificial intelligence joins us right after this welcome to Big technology podcast a show for cool-headed nuance conversation of
00:00:13
the tech world and Beyond we're joined by a very special guest today Colin Murdoch is here he's the chief business officer at Google's Deep Mind we're going to talk a little bit about Theory we're going to talk about practice um
00:00:26
and how Google is trying to break take The Cutting Edge and artificial intelligence and productize it Colin welcome to the show Alex it's fantastic to be here I'm really looking forward to this
00:00:39
conversation great so would you like to start easy or hard uh wherever you like Alex let's jump right let's let's just jump in uh why build artificial general intelligence I mean this is something
00:00:52
that is a stated goal of Deep Mind to build AI on par with human intelligence or even something that's surpasses it why do it that's right well I mean just stepping back for a moment I think it's
00:01:05
really important to think about what artificial general intelligence is or AGI is because you'll be familiar with a lot of AI systems today you build an AI system to solve a particular problem and that works exceptionally well we've seen
00:01:17
huge kind of breakthroughs in fundamental AI research which has driven really important impact in the world through this form of AI what we hope though with artificial general intelligence is to build a system that
00:01:30
can solve multiple different problems so one AI system that can solve multiple different problems and much like us humans Alex that means we can take learnings and the AGI system can take learnings from one setting and apply it
00:01:43
to a new setting and we expect that therefore uh means that this AGI system can create more creative and transformational solutions and we know this is possible actually because humans are a form of artificial well not
00:01:56
artificial in fact real general intelligence and look at the incredible things that we've been able to achieve and that's why we at Deep mine think it's a really worthy to pursuit to develop a form of artificial general
00:02:07
intelligence that will help hopefully tackle some of society's biggest challenges things like climate change and you know problems and questions in healthcare that's actually the core of what we're up to and it's incredibly important and I think incredibly
00:02:21
interesting yeah but it's it's going to do more than than just that I mean you know you can direct I'm sure you can direct it to do you know climate uh work and Healthcare but there's a whole host of different things that you know an AGI
00:02:34
you know will will I imagine will be able to do from your website you say uh by building and collaborating with AGI we should be able to gain a deeper understanding of our world so that's even a level deeper than you just mentioned resulting in significant
00:02:46
advances for Humanity how how will this technology a provide a deeper understanding of our world but B you know if if we can apply you know if we can think like a human then where are we
00:02:59
GNA how practically will it be able to achieve some of these you know goals that you just discussed so I think uh the application in science is a really interesting area to think about um science is a
00:03:12
incredibly complex area that we as humans over the years have made incredible advancements in and those advancements in science have really enabled us to build the societies we have today have the health that we have today have the food production that we
00:03:25
have today but at some level it feels like we're meeting the limits of the knowledge that human alone can create through this process and a concrete example of this is um proteins uh they're the building blocks of life
00:03:37
they're what makes you and I work alcts right now if we didn't have proteins the little machines in our bodies uh we wouldn't function and scientists for years have been trying to determine the structure of proteins actually because
00:03:48
if they go wrong if they're misformed um if the structure isn't quite right that can cause things like disease um and all sorts of um maladies across you know whole range of different areas the
00:04:01
challenge is for humans we've been trying to do that using experimental means so it takes years of painstaking research and millions of dollars of specialist equipment to determine the structure of just one of these
00:04:14
proteins which is why actually experts for about 50 years now actually these are really really uh Advanced scientists have been trying to use computational approaches to determine the structure of a protein because proteins are made up
00:04:28
of these little units that encoded for in our DNA and we actually know the sequence of those units the challenge is how do they turn into this three-dimensional structure so there's a really important problem there that is the core of a lot of really important
00:04:41
biology understanding that we've not been able to solve as humans and this is a good example of where AI has been able to step in and I can tell you a bit more about that you're building up too
00:04:52
yeah Alpha fold Bingo yes I mean so so I'm going to wa wait a moment because so Alpha fold we're going to talk about about it more in the second half but you've been able to decode proteins with uh Deep Mind research using this alfold
00:05:06
system but that is that is still a narrow intelligence so you're actually taking it a step deeper if you're trying to build an artificial general intelligence so talk a little bit more about why that's
00:05:18
necessary well there are many problems that uh we're able to uncover uh um begin to solve with things like Alpha fold but if you step back and think how humans we solve problems we come across one problem we solve that and that
00:05:31
typically opens up a new problem and if you step back and think about the universe as a whole and our world as a whole the size and scale of that problem space is immense and I suspect there are problems that we've not even thought of
00:05:44
today and I hope with a system like AGI artificial general intelligence is that once we solve one problem that typically opens up uh and shows this kind of branching set of new problems the AGI
00:05:56
system will be able to kind of trans translate across that problem space at a speed and scale that we can't even imagine today and and in many ways it's a hard question because we're bounded by our existing human intelligence so we're
00:06:09
abanded by what we currently know but if we can build a system like this I think we'll be able to explore this problem space in an exceptional way and the reason why I use Al fold as example is because I think science and scientific discovery is actually a really
00:06:22
fascinating and important area that is this kind of enless set of problems that we hope in AGI system we working with humans be able to help us cck doesn't this seed a lot of control or a lot of the sort of meaning of what
00:06:36
it is to be human to the machines what we see actually um in Alpha fold as a good example um and even when I think when it gets to uh AGI is that we'll see computers and people
00:06:49
working together um the great example that actually uh from a couple of years ago that still resonates to me is a system called alphao now go is a a board
00:07:02
game which very popular in some parts of the world I think our listeners are familiar but yeah you can go ahead yeah fantastic um and people studied this for thousands of years and we developed an
00:07:14
AI system that was able to play alago um and we set it against various human experts and you know in some cases the human experts won but over time actually
00:07:26
alphago became increasingly powerful what we discovered however was that once we then made alago more generally available the humans actually used alago to improve their performance and that's
00:07:38
often the case so the AI system help the human improve and you put these two things together and in many many cases it's the AI system plus the human that ends up being the most fruitful the most powerful outcome so there's definitely a
00:07:52
path there I think of a sort of uh co- Evolution or co-development between these two forms of intelligence yeah but again these are narrow intelligences right they are built for
00:08:04
one specific task so when we you know if we're able to achieve an artificial general intelligence which is basically going to be able to think and sort of perceive the world and plan on par with human obviously it will exceed a human
00:08:17
because it can have the entire power of computing behind it we're we're limited by our brains and we get tired um that's much more powerful I mean it does it don't you think that's like a completely different different level and and again
00:08:29
like you know we talk about a partnership but what what's there to say that that you know we're again not I mean there's a going to it does seem like there's a point where the machine will just like not need the human
00:08:41
feedback to make these discoveries so that's the questions are we going to see a good chunk of what it means to be a person to be a human to these machines and if so maybe that's good maybe it evolves Humanity to a different place
00:08:55
but yeah what do you think about that well I'm sly excited to imagine how an increasingly capable system will help help humanity and also help Humanity
00:09:06
expand our capacities I one way I think about it is that you know I've got three children um I'm an adult so I've been in around longer for them there a form of uh artificial general or general intelligence so am I uh but we can work
00:09:20
together even though I happen to know more than my children we can work together they sometimes teach me things that I'm surprised of as well so um I think these are different forms of intelligence that I hope can work together um and actually in Harmony in
00:09:32
many ways should so it's interesting that you mentioned your children it's almost like creating new life in some ways um should the entities being responsible for
00:09:44
creating these new life this new life in some way be be businesses I mean should we have businesses in trust you know trusted with the stewardship of this new you know fascinating form of
00:09:57
intelligence should we reach it there and if so why so I think way I think rather than life I think the way I think about it as a as a tool I think that's probably the framework I use when I'm
00:10:08
thinking about these sort Technologies and then I think as we think about building these these really powerful and important tools um then I think businesses do have a really important
00:10:19
part to play for sure um I think uh government has a really important part to play I think regulation has a really important part to play I think Society has a really important part to play this is a new technology a new set of tooling
00:10:31
that's going to have a really important and positive impact on the world and one we've got to develop in a responsible way because it is a very powerful technology we've got to take great care with it as well so that is why actually
00:10:43
at Google dmin um we have and it's always been part of what we do this uh kind of responsible approach or pioneering respons as we call it it's always been in the DNA of what we do I think all of these actors need to work
00:10:57
together in harmony actually to to develop something that's really important and impactful for society yeah but you did just compare the the intelligence to you know it's a tool but you also compare the intelligence to that of humans so it
00:11:10
can't be both of those things or maybe it can um it seems like in some ways we are like both um you know it seems like we're both in awe of like what these
00:11:22
things can do and still not fully in comprehension as a species of what we're working on do you think that's a fair assessment I think uh we're at the early stages of a very long ladder if you like so we're
00:11:34
on the first rung and uh predicting exactly where research itself will go is always a precarious task um I'm very hopeful if that's what you mean by an a
00:11:46
of the potential for this technology to really help lift Humanity to new levels to help things like uh climate change to help in things like health I think that's really important to keep in mind um we've got to continue to interrogate
00:11:59
these systems to understand how they work to make sure we're doing it in a responsible way to make sure we get the right review in place each step of the way so that we do understand we do roll them out in a way that makes sense for society overall then we got to do both
00:12:12
those things we're going to be bold in how we develop it but also responsible and take care okay so let me ask you a little bit about what the path is to getting there
00:12:25
right so you talked about how it you know a general artificial intelligence will sort of be able to um learn learn and apply lessons from different fields
00:12:39
or different areas of experience so what do you what type of uh research and advancements need to happen in order to get us closer to that point well we are still in early days
00:12:52
but let me give you some examples of what's currently working what we call generalization and I can talk about some of the areas of active research we think are going to need uh for us to get there so um just stepping back for a moment
00:13:05
what I what I what I mean by the ability to generalize and you'll hear there some a lot in this field there the ability to take learnings from one setting and apply them in another setting and we recently developed for example um an
00:13:16
algorithm called muzero which was originally developed actually to play the games of Chess and go and then we realized we were able to take this algorithm and apply it to I'm going to say the game of YouTube video
00:13:28
compression that's maybe funny I say that but we were able to take an algorithm develop for games that was a master in chess and use that to dramatically reduce the bandwidth requirement to stream YouTube videos and we did that by understanding that
00:13:41
actually a video is a series of individual pictures and if you imagine the transition between each of those pictures is like a step in a game that gave us the Insight that this algorithm would generalize from uh playing chess
00:13:52
and go to uh YouTube video compression uh another example actually is an algorithm we call flamingo um and we were able to uh use Flamingo as part of an app called Lookout which
00:14:06
is an app developed by Google uh if you're partially cited and you need help you can use your mobile phone to take a picture of something and then you can also look out what's in that picture and can help you can kind of be your eyes
00:14:18
and we realized that algorithm could could could be used and redeployed and generaliz uh into uh adding the descriptions into videos that are uploaded to YouTube shorts so creators
00:14:30
creating videos on YouTube shorts they have less time um they upload the video the algorithm that helps people find those videos need something to go on to help people discover them uh so we discovered Flamingo was also able to
00:14:43
look at that video for you entally watch all the videos for you and add data metadata to those videos such that when you're searching for those videos it's much easier much much easier to find the videos that you need when you consider
00:14:55
the billions of videos that are viewed every day on on the on YouTube shorts that's really significant so those are those are just I'm calling those out because those are two examples where we're beginning to see these forms of generalization and of course generative
00:15:08
AI is actually a great example as well so these new uh these new tools where you're able to interact with them in a way that is kind of Fairly conversational um and get a really
00:15:22
surprisingly and um powerful response that is one way uh we're beginning to see this move to morge more general intelligence and maybe I could just jump into tell you a little bit about how for
00:15:34
example uh that recent working generative AI is allowing us to move closer to AGI and then I can tell you a bit more about some that's fascinating fantastic so what's maybe surprising to
00:15:47
know if you've been following the field in generative AI you've really seen it burst onto the scene in the last you know 18 to 24 months is that some of the underlying breakthroughs were developed about about 5 years ago and what's
00:16:01
happened in the last you know 18 24 months though these systems have been really really scaled up and by that mean uh the size of the model the number of parameters in the model which contain uh
00:16:12
the model power has Dr grown dramatically um and these systems have been trained on kind of larger and larger data sets and what's happened is that by scaling up these models we've
00:16:25
seen these emerging capabilities appear and but that I mean it's not always possible to predict exactly what capabilities would appear but we've had these new capabilities appear that have demonstrated really powerful
00:16:38
generalizability so you can then take a system that's been trained in this way and you can ask it to summarize a document or write you an email and it wasn't necessarily trained expressly on these tasks but it's able to achieve these tasks because of the training
00:16:51
process and the scaling up has happened and that's been as as I'm sure you've been following and many others have been following that's been a massively uh important breakthrough in the last 18 to 24
00:17:03
months and but these systems are still not complete uh they still get things wrong um they maybe can't plan in the right way they maybe can't remember what you did yesterday and help you today so
00:17:16
things like memory the ability to remember between episodes planning the ability to imagine a whole range of different future scenarios and plan effectively in that setting those are two areas of active research that are
00:17:30
really important and at a kind of zoomed out level another is what we call kind of Concepts and transfer learning so as humans we're able to build this kind of deep conceptual understanding and that actually forms a kind of really strong
00:17:42
foundation for us to take knowledge that we generated in one setting and transfer that to another so Concepts and transfer learning planning and memory all really active areas of research which I think will help us push the next French here
00:17:56
and and and actually by the way we haven't necessarily reached the limit of making these models bigger either we're not no one's quite sure where that limit is and so that's also a really important active area of research just making these things bigger and bigger um where
00:18:09
will that where will that go and where will that land and what more can we get from that yeah we've had Yan Lun on the show and I've been speaking with Yan for since 2015 2016 so seven or eight years
00:18:22
on this point about what intelligence is from the eye of artificial intelligence researcher and he's always said that it's the ability to predict and to plan and it is very telling right now that the research now is is all about
00:18:35
teaching these AIS to predict into to plan it in fact speaking about the Gemini the new Gemini model um I'm pretty sure people from Deep Mind have talked about I think I'm going to just cite this that the algorithm should be
00:18:47
better at planning and problem solving so that seems to be where we're going so first of all I'm going to get you know I have a few questions for you about Gemini but just let talk about it on a broad level
00:18:58
how do you teach an AI to plan and predict so um there's a whole range of different active research tasks here and and to be clear there isn't an answer yet which is why it's still active
00:19:12
research but one of the ways we motivate This research is by making sure we have tasks that require planning uh so we spend a lot of time and investment in building a whole Suite
00:19:24
of different evaluations and tasks which then provide the target if you like for our research and our research programs to focus on and that's a really interesting definition of intelligence and one of the definitions of
00:19:36
intelligence that we use at Deep mine and was actually created by one of the founders of uh Google deep mine Shane lag is intelligence is the ability to perform well across a range of different
00:19:47
tasks and I really like that definition because it's I think it's very descriptive and it's very easy to operationalize into a research program and so this sense of building multiple different evaluations and tasks that
00:20:01
provide then a way for us to measure our performance and progress against whether it's planning or adding memory um is a is really Central to actually the way we conduct research and then behind that
00:20:13
it's a creative process uh so what you're trying to do is bring together people from a whole range of different disciplines from Neuroscience from different areas of AI research to uh come together and you have ideas about
00:20:26
how we can make progress and then use the incredible engineering Talent we have and the compu resources we have to experiment and take steps forward that's how we do it at a at a kind of meta
00:20:39
level and deep mine started largely with some breakthroughs in gaming so how is that applicable for because you know I think about predicting and planning and it seems like if you're playing you know sophisticated games like go then that's
00:20:53
basically going to take you in that direction yeah you great point because games are a fantastic Proving Ground for these algorithms they're fantastic because they're actually hard for humans
00:21:06
um they have a they have ability for us to measure how good performance is there's normally a score of some sort so we can Benchmark the algorithm's performance versus the humans performance and there's a whole raft of
00:21:18
different existing games out there that can push and pull the algorithm AI capability in different directions and by the way you can develop new games and I think maybe the third the third maybe
00:21:31
the final point is that games can run faster than clock time so you can do many many durations in a kind of simulated game much much faster than you could experiment in the real world which is why there's such an incredible uh
00:21:43
Proving Ground and development ground you're absolutely right for these algorithms and we continue to invest deeply in kind of game like environments for exactly those reasons maybe one other important point there also a
00:21:55
really uh useful way of testing algorithms out to test kind of their limits um and we can check their kind of technical safety to make sure they're doing what we expect them to do so they're a nice way of developing and
00:22:08
testing an algorithm before they break out into the real world and and maybe a nice example here actually we often use this technique of uh first developing an algorithm in a game to your point to do
00:22:21
planning um and there's an example here in robotics if you try and train an algorithm directly on a real robot it's going to take you a long time because a robot can take quite some time to complete the task and in the beginning
00:22:33
it may be all over the place like I maybe like a young child learning to walk what we what we do is we create a simulation of that robotic environment be a robotic arm stacking blocks and we
00:22:46
train the algorithm in that simulated environment until it gets good in that simulated environment and we take the algorithm and then we apply it to the real robot stacking blocks we discover it's actually then pretty good out of
00:22:58
the blocks and in the real world the robot can then begin to build in the training there so the way that I picture this happening is like it feels like most of uh the general public has started to get
00:23:11
a chance to like start talking with AI VI of these large language models so you know when I when I try to conceptualize like what this might look like down the road I start thinking that like when I'm speaking with uh Chad GPT or aard or a
00:23:25
bing it starts to remember who I am it's starts to be able to accomplish tasks for me it starts to be able to help me plan you know is that is that sort of like the next step here is that where this research is building
00:23:39
toward that's right so you're able to converse with these do agents today as you've discovered and you can have actually quite a meaningful important conversation but it might have not remembered what you did last week for example they've got limited kind of
00:23:52
context Windows as is as you may have heard it called but what you really want it's to remember as you've noted what did I do yesterday what did I last week what's my preference when it comes to kind of looking at a given film for
00:24:04
example um because next time I ask to watch a film you it wants to know what I watched before or maybe what my preferences were so that ability to remember more about our previous interactions actually becomes really important you want these things have
00:24:17
like the memory of a goldfish it's like you sit you're talking with it and then five minutes later it's like hey just remind me what what you said like totally forgot so that's like one step yeah yeah absolutely right it's a really
00:24:29
important area for us to kind of expand the memory of these systems sometimes we refer to this as episodic memory so they remember um episodes important episodes in the past so they they're able to
00:24:41
bring to bear that important um understanding then when it comes to planning about the Future these systems need to be able to uh stop and reason about the right sequence of steps to take so um you know for example I want
00:24:55
to plan a holiday you know I want to go here then I want to go here and I want to go uh there um and the series of things I want to do may change over time I may get on the flight I may if you
00:25:08
could ask one of these systems today catch that they can they can come out with a pretty good uh pretty good response on some of these things but they aren't able to plan based on what you've done in the past um and you know
00:25:20
what would what would a reasonable kind of itinerary be that's changing over time I'm actually going hot over my family very soon this is very live on my mind I don't think the AI systems can really get to a level that I would really want them to at this point right
00:25:32
and so we think about where this going we talked a little bit about um you know being able to predict and plan we talked about um you know I get we we sort of hinted at multimodality right like having a model that's generalized so be
00:25:46
being able to like do text but also and like a human we can talk we can read we can see you can process and most of these models have just been text or
00:25:56
computer vision or computational and it does seem like the next step is really going to be bringing them all together that seems like a massive technological feat but my understanding is that that is something that's being worked on inside Google with this new Gemini model
00:26:10
I mean those two descriptions that I just read are are both Gemini so talk a little bit about what Gemini is and how it's going to take us on on that road the Gemini is uh one one of our latest
00:26:22
research programs and you're absolutely right one of the really important errors it's touching on is what we call multi modality um it's a bit like the human senses you just described we can kind of use all our human senses together and
00:26:34
combined to uh achieve the goal we're we're setting out to so it will bring in things like text it will bring in things like images and it able to input those things but also output both those things
00:26:47
so you might have a question about something you can see you can share that image and you can also ask a question about it and you may want to then adapt some something in that image uh by uh
00:27:00
saying please edit this element of the image and it can do that for you so bringing together these different modalities is something that is a really cool important part of that Gemini program as well as the kind of memory and planning architectures that we
00:27:12
discussed earlier and and maybe a final important component is you know we're hoping to develop models of different sizes and scales so there'll be kind of different sizes of these Gemini models which can then be applied to different
00:27:26
use cases depending on what important I mean has there been anything about training Gemini that's that surprised you or is this kind of like where you think it's supposed to where yet it should have been going the whole
00:27:37
way um I'm I'm not deep in the Gemini research program myself um but what I would generally share is that not it's not Gemini specific is that when it comes to training these large models um I think people in general have been
00:27:51
surprised that as you make these models bigger they get more capable and they start to uh demonstrate these capabilities that you wouldn't necessarily have planned or expected and in the field this is generally referred to this there emergent these emerging
00:28:04
capabilities and I'm not sure if we've fully got to the end of that process yet so there's a there's a kind of almost a constant state of surprise as these new capabilities emerge right so I was speaking with some
00:28:18
folks uh at Google and trying to figure out like what to ask you about and someone brought up um talking about modes of training so I'm C I I want to ask you about the Deep mine approach
00:28:30
versus uh this new approach that's or maybe not new but definitely is gaining share and people's minds called uh constitutional AI so I'm just going to read you what constitutional and our listeners what constitutional AI is from
00:28:42
a recent New York Times article and I want to get your take on whether that's the right way to train these models so it says constitutional AI Begins by giving an AI model a written list of principles a constitution and
00:28:54
instructing it to follow those principles as closely as possible a second a AI model is then used to evaluate how well the first model follows its Constitution and corrects it when necessary you know I'm
00:29:06
curious what you think about this um this approach and whether that's something that you know Google would consider employing and if not why not so this this is kind of uh I would generally think about this approach and
00:29:20
other approaches like this there's a way of ensuring these models are behaving in the way that we want them to behave um and we think about do definitely think about that very deeply it's very important to everything we do there are
00:29:31
different ways to do that um one way is actually by having um an AI system like the one you've described provide feedback to the model that you're training about whether it's behaving in the way that the designers would like
00:29:44
their system to behave and that's that's certainly something um it's all part of the overall approach another important way actually is that you have humans providing feedback to the model this is a a process called RF that folks might
00:29:58
be familiar with where human human rers interact with these models and are observing the Constitution and provide feedback to the model on WEA and how well the uh model is performing against
00:30:10
that Constitution and actually at the moment that's a really important part of I think the core research process because humans are actually very good at this there's a kind of secondary benefit
00:30:21
of that is that we are um beginning to understand how we can begin to embed more and more human feedback into the model process so I think in general terms yeah this is a really important
00:30:34
part of how we approach research to make sure the model is well aligned with the sort of if you like Constitution that the designers and the society ultimately would like these models to be behaving in accordance
00:30:47
with one last question for you before we go to Break um why does everybody in this field or some of the leaders in the field just constantly compare uh this work to the nuclear weapon project I
00:30:58
mean I I can't go a day without hearing uh an AI illuminary talk about how like they're the next Oppenheimer for instance I mean this is from a New Yorker story where Sam Altman said uh
00:31:10
you know let's see he um he compared the company to the Manhattan Project as if he was chatting about tomorrow's weather forecast he said the US effort to build an atomic bomb during the second world war has been a project on the scale of
00:31:23
open Ai and he just tweeted that he was uh hoping the oppen Oppenheimer movie would inspire a generation of kids to be physicists and and sort of Miss the mark on that and he wants you know a new movie of of that of that scale basically
00:31:37
you know put me in a movie or something of that nature why why what's going on with all these these comparisons so i' I've not seen the openen H movie yet but I'm look I'm looking forward to seeing it I think the comparison that I often
00:31:50
hear is actually of the Apollo project the space project and I think the reason uh examples like that Apollo projects and don't hear that I mean you don't hear these these nuclear comparisons I
00:32:02
feel like they're all over the place I don't hear anyone talking about this as as a Space Project well maybe maybe I hear it more because that's actually how Google de we often think about it and talk about it I think the but I think the kind of fundamental first principles
00:32:15
analogy is that these are these are projects where you're Gathering Together uh large group of very very talented people with a very clear focus and a common belief um and if you can do
00:32:28
that then you can make incredible progress I think these sorts of projects may be kind of offering that sort of inspiration uh to you know folks working in this
00:32:40
area Colin Murdoch is here with us he's the chief business officer at Google deep mine when we come back we're going to talk a little bit about the business side of these models uh and especially how they're being applied within Google
00:32:53
back right after this and we're back here with Colin Murdoch he's the ch Chief business officer at Google's Deep Mind um Colin how what is um the state of the merger between Deep Mind and
00:33:05
Google brain which just came together to sort of be able to work together in a way that hasn't happened for years that's right so we've just formed uh Google Deep Mind from the team that
00:33:17
was at Deep Mind and the team that was at at brain and actually uh these teams have been working together for quite some time in the background I think the the recent merger if you like to form this
00:33:31
super unit has come a really important time in the overall development of AI we're kind of in this you know super era or Golden Era of AI development and we just thought it was a right time now for
00:33:42
that reason start to bring together the talent but also the compute and the resources so that we could make sure we were focused and organized in the right way for the next phase and I've actually been at well deep mine now Google deep
00:33:56
mine for about 9 years years and the pace and the change and the kind of Frontier that we're working at means we're constantly needing to refine the way we organize to make sure that it fits where we are in the kind of
00:34:09
Technology Evolution cycle um and you know it's going great um I'm really enjoying kind of getting to know the entire new team and we're making good progress right and so it's so interesting because you know deep Minds
00:34:23
era areas have been the gaming working on protein folding which we're about to talk about um Google brain you know maybe more search related so how much of your activities are now going to be focused on the core Google Business
00:34:35
versus some of this other type of research I I I think it was interesting to know uh even at Deep Mind and actually this is very close to uh my role is that we we have for a long time
00:34:47
being um taking the technology that's been developed in our fundamental research programs and apply that to Google's products and services so that's all that's actually been a cool part of both these groups and actually is now a
00:35:00
fundamental part of what we do at Google deep M so we're both advancing the state of the art in the technology applying that to really big problems in science and then using those breakthroughs to drive value and impact across these you
00:35:13
know was often bidon user products at Google and that's absolutely right Alex it's fundamental to the the new setup at Google demon mhm so um where do you see the bigger business opportunity is it going
00:35:27
to be I mean you're the chief business officer so is it going to be search or we talked a little bit about artificial general intelligence I mean you have Alpha F right now that's that's out in Market like where is where's the future of the business on this front so search
00:35:40
is of course an incredibly important part of Google's portfolio I I expect it I expect it to continue to be a very important part of Google's portfolio so we'll continue to do everything we can to drive value and search let me tell
00:35:52
you about how I think about it because this is a you know a technology trans transfer process and that's not easy going from research to real world impact at any means at all even when even when you're operating kind of Google deep
00:36:05
mine working with Google I think about it actually as a matching process so on the one side we have all this amazing research and these research breakthroughs um and there's a team of people that are developing those and on
00:36:16
the other side we maintain relationships with all the great businesses and business units across the whole of Google and in fact the alphabet group as a whole so we can deeply understand what's important to them and moving their business forward so we've got this
00:36:30
set of Solutions on one side and the set of problems on the other side and then we try and match these two things together sometimes joke is a about like running a dating service where you're trying to match problems and solutions um so we go ahead and do that um and
00:36:42
then we try that out and if it works we go ahead and launch so there's there's a there's a process there that's what I want to share it's quite important to share there's quite a systematic process it's one of matching uh technology solution and product problem as defined
00:36:56
by the business search is an important area uh we've done a lot of work with YouTube as well so uh for example I I talked about that a bit earlier we've worked with YouTube to help um create better tooling for
00:37:09
YouTube shorts so you can more easily find the videos you want we work with YouTube to reduce the bandwidth requirement to watch these videos we've even worked with internal teams to
00:37:21
create better coding tools so we can kind of really power up all the developers at Google um we've worked with other teams at Google to do things like predict the output from wind farms
00:37:35
that Google is part of so we can make more efficient use of energy so there's a there's a whole range of different applications and I think that's important to recognize yes search I think will continue to be really important um and also I expect us to
00:37:48
kind of weave the technology into all parts of Google so we can really help lift the whole business yeah so Tu a little a little bit about your process between taking you know a research breakthrough and
00:38:00
then actually putting it into production so um so it starts it starts in the kind of research setting we have a group of researchers that are working towards ultimately artificial general intelligence and as a result of that
00:38:13
they're developing all these new algorithms and breakthroughs along the way we have a dedicated team of um technical Specialists and product managers who are constantly tracking
00:38:25
that Evolution this kind of gold mine if you like of research breakthroughs and then those same people are also in constant contact with the business owners with the product owners that sit
00:38:37
across the Google and alphabet ecosystem deeply understanding their world deeply understanding their priorities and the problems that are important to them and trying to find a way of casting those problems in a way that match up with the
00:38:50
algorithm that has been developed in the research process the example I mentioned earlier actually is is probably a good one so muzero this algorithm was originally developed to play games like
00:39:01
chess and go um person in this team was talking to uh the Youtube team um and they were talking about how it would be valuable if they could reduce the size of a video because it could be streamed
00:39:15
to more people more places around the world um and it's not immediately obvious to anyone that a algorithm developed to play games of Chess and go can then be applied to reducing or compress ing a video this is where the
00:39:28
really creative component comes in because they realized if you think about a video as a series of images and steps in a game Sorry and steps between those images you can think about that as a
00:39:41
game with each each kind of uh picture in the video there's a stage in the game and each step between those pictures is a step in the game and so that was enough intuition for them to go why don't we try mu zero on video
00:39:54
compression so that starts the ation that's the initial kind of matching phase the next step is to take that algorithm and work with the product team in this case the YouTube team to take some videos and try it out try it on
00:40:08
real data because it doesn't always work it's a hypothesis you know we run a portfolio here of these things and some of them work out and some of them don't so that's the next step we used to call
00:40:19
it incubation we can still do and the final step is that if it works and if it meets the bar and important of the product team at Google then we work with them to help make it launch so we start
00:40:32
at the beginning with a matching process we then kind of incubate and prototype and then finally if it works we go ahead and launch and that's a kind of individual project but zooming out at a portfolio we have this kind of constant
00:40:43
humming and rolling portfolio of these projects to make sure we've got this flow back and flow forward of innovation between the two two areas cool so why don't we talk a little
00:40:55
bit about about we have 10 minut about 10 minutes left so I want to talk a little bit about how this works in a couple areas um I definitely want to cover protein folding which is like I think the most interesting breakthrough that deep mind has come out with um to
00:41:07
date and then um then maybe we can end on self-driving cars but let's start with with protein folding you know I think that Alpha fold and maybe we've done it earlier in this in this conversation sort of gets talked about as sort of a thing that's that exists
00:41:21
and you know okay it happened and then people move on but I actually want to hear a little bit about like what is going on with with Alpha fold talk a little bit about about the Breakthrough itself and and how it's being applied right now fantastic yeah
00:41:35
so we we we did touch on it earlier on but just as a brief recap Alpha fold is a way of determining the structure the 3D structure of proteins which ends up being really important in a whole range of different fields and this is
00:41:47
something that folks have been trying to do for many years but took years to determine the structure of just one protein but with Alpha fold we've gone from years to minutes to even seconds sometimes to determine the structure of
00:41:58
a protein so what does that mean in practice well um we've used Alpha fold now to map all 200 million proteins known to science or 200 million proteins
00:42:11
known to science we've made that available to everyone and actually someone estimated recently that that's probably saved about a billion years of PhD time uh because you know probably on average you
00:42:23
spend about a PhD to determine the structure of just one that's available to everyone now we've had hundreds of thousands of biologists and Sciences from around the world that
00:42:35
are now tapping that database to be able to advance their particular work and their particular domain let me tell you some stories about how people are using this um there's actually a team I think at the
00:42:48
University of Colorado that are using aftera predictions as part of their work so they're focused on the problem of anti itic resistance we kind of take antibiotics for for granted in most parts of the world and that's a great
00:43:02
thing um but the bacteria are developing and there is increasing cases I think in the us alone there is probably millions of cases a year of antibiotic resistant diseases and that's a an important also
00:43:14
quite a scary problem so there's a team of scientists working on how to uh address antibiotic resistance there's a particular bacteria involved in antibiotic resistance and they've been trying to determine the proteins on this
00:43:28
bacteria for a number of years a number of years quite some years but hadn't yet made an advancement um with Alpha fold and the protein structures from alpha fold they were able to solve that
00:43:40
particular protein in minutes they've gone from years to minutes really unblocking and experimenting that research I think that is a you know that alone is an incredible incredible example of how alpha fold is impacting
00:43:53
in the world there's another equally important example there's a group working on developing malaria vaccines you know disease that devastates hundreds of thousands of lives every year and they've been able to use these
00:44:06
alphafold structure predictions with different proteins but still Alpha fold structure predictions to speed up their Research into malaria vaccines so a couple of examples in in healthcare and
00:44:18
there's other groups focused on neglected diseases uh where you know would may have been too expensive to do this the traditional way but with Al F predictions they're making advancements um a slightly different example but I
00:44:31
think equally important and quite cool is there is a group at the center for enzyme Innovation which I think is at a university here in the UK which is uh focused on developing enzymes that can
00:44:42
eat the Plastics eat the Plastics that clog up our landfills and our oceans and they've been able to use these protein structure predictions to speed up their work into producing um plastic eating
00:44:55
enzymes so we've got we've got like and those are those are just a kind of sampling of the different ways that Alpha's been used today it's quite difficult to keep up there's a kind of new new group almost every week coming up with a way of using these uh these
00:45:09
predictions in their work yeah that's that's wild so as Alpha fold applications expand I mean as Google comes up with more you know programs like this does it
00:45:22
change the nature of Google's business I mean Alpha fold and Google search are very different so talk a little bit about how that fits together yeah and it's it's uh you know it's a good point these these two things are quite
00:45:34
different um and Alpha fold is a good example here of know I described some of the ways that is having impact in the world when I looked at Alpha fold to your point I was like well how does this work with Google search it's not obvious
00:45:46
right how does how does how do these two things knit together what's the match there um so took a step back with my team and um thought about other ways we could employe and deplo this and it
00:45:58
seemed we we Lo we looked across a range of different areas and business opportunities by the way from agriculture to all sorts of areas but in the end we concluded that actually there is a great opportunity here in drug
00:46:10
Discovery you know it takes 10 plus years to develop a drug and then often when it goes into clinical trials it fails there's a very very high failure rate and so you've spent all that time and money in investment and it doesn't
00:46:23
actually make it through and solve the clinical need concerned about so having understood this kind of scale and importance of the problem and the opportunity that then gave us impetus to
00:46:34
form a new company uh so we we formed a new company which is a sister company now to Google deepmind it's part of the overall alphabet group it's called isomorphic Labs it's about 2 years old and its mission is to use AI to
00:46:48
reimagine drug Discovery and the team is making fantastic progress I'm really excited to see how that work will help reimagine that whole process um now
00:47:00
there's definitely more research to do there that's not just kind of alphafold and done there are kind of alpha fold scale problems along the way that's a really good example of where we've been able to set up something new based on a breakr like Alpha fold and I think there
00:47:13
could be other advances that come in science that may trigger a similar sort of arrangement so then does Google just become like an incubator for AI companies or I mean what is the nature
00:47:25
of of Google again if it's like you know it's can it just keep existing across all these different business lines I I'm not I'm not well placed necessarily to talk about the overall
00:47:37
corporate strategy for Google right um but in in in terms of interesting because it gets into so many different areas yeah yeah I mean in terms of how I think about it what's important for me to be doing is really focusing on the
00:47:51
most important problems that Google has and the rest to the alphabet companies and making sure we're building and developing AI systems that can help solve those and being alert by the way like the isomorphic lab's example when
00:48:05
there is something that's important and significant and material enough that I'm also surfacing that and saying hey do you know what I think there's a really important opportunity in business here let's talk about it yeah interesting
00:48:18
okay five minutes left self-driving cars um I'm in San Francisco right now here for a visit and hopped in a whmo a week go uh and it was unbelievable I mean the way that these these cars can drive
00:48:31
around and no driver feels totally safe I mean maybe that's misplaced confidence but it drove really well um how how close are we to having these type of cars on the streets
00:48:45
everywhere and is it really a matter of needing more breakthroughs from the research side or is it is it just a business will I feel like you're perfectly placed to answer that one yeah yeah well I don't know that's tricky for
00:48:57
me because I'm not I'm not inside the I'm not inside the wayo business so I'm notar able to forecast or project exactly where where weo goes next um I I've also had a ride on one of the cars
00:49:09
and it was about gosh five or six years ago now so um and my ride was pretty good back then actually like so I'd like to have another go just to help calibrate myself on I guess the speed of progress and that would help me predict
00:49:22
where we're heading in the future um I'm here in the UK so weo hasn't hasn't kind of made it to the UK yet but I'm looking forward to have a ride then so yeah I I'm not sure about exactly AO but I'm looking forward to you know monitoring
00:49:35
and tracking and set of bracing their progress for sure and from a research perspective I mean it seems like I again maybe you're not you're not on the ground so you can't share this but it from a research perspective it it seems like that that
00:49:48
should be solved at this point I mean I guess if they're already on the ground yeah as you said I'm I'm not I'm not on the ground with weo so I'm not exactly sure where their research programs are they got a lot of smart
00:50:01
peopleo let's let's end with this we just had a year where people have been going bananas over large language models I got a question today when I mentioned I was going to be interviewing you
00:50:14
people want to know what is the next model breakthrough that's not an llm that people aren't paying attention to but will be as impactful as what we've seen with these models
00:50:26
I'm really excited I don't know exactly what the breakthrough will be but I'm really excited about the union of these llms plus reinforcement learning and I'm excited about that because I think there's a lot more to come from reinforcement learning and I know at
00:50:39
Google dmme we have great deal of expertise in that so I expect to see the fusion of those two things creat some really powerful and important breakthroughs K Murdoch thanks so much
00:50:50
for joining you're welcome great to be here all right thanks everybody for listening thank you Nate guatney for handling the audio LinkedIn for having me as part of your podcast Network and all of you the listeners great to have
00:51:03
this conversation with Colin here for you hope you've enjoyed and we'll see you next time on big technology podcast
End of transcript