Select text to annotate, Click play in YouTube to begin
00:00:00
[Music] [Music] B B Dev
00:01:37
[Music] [Music] to f um thank you for having me um as said
00:02:06
uh we are uh going talk about um critical thinking in the context of of um primary education in Uganda
00:02:22
um the this is um part of um joint work with um Nava ashra who's a professor at the London School of economics and
00:02:35
vessel nurani who who is a uh post talk at the University of Chicago and who spent a lot of time in Uganda making this experiment happen in in in terms of very abstract
00:02:51
background uh it's uh lot a lot of the um there's a large body of um evidence as you might imagine on teacher training
00:03:03
this is one of the biggest pieces of of um spending for any any education system once you leave out salaries it's often
00:03:16
and buildings is really the other very big piece so it's it's it's a it's a big and and I think mostly it would be reasonable to say that professional trainings have been disappointing that
00:03:30
when there have been rigorous evaluations most of them find that these EV these trainings don't do very much um now there's exceptions to that and the
00:03:42
the exceptions are also interesting and specific the exceptions are what what sometimes called um structured pedagogy
00:03:52
or in a sense versions of giving teachers a very simple uh and very uh clearly delineated program to implement
00:04:05
so you tell teachers do this then do that then do that and there different versions of that including some we have worked on which we call teaching at the right level these are successful they
00:04:17
they are often you know the the World Bank and um you know you and UNICEF and several other institutions uh recently
00:04:28
concluded a surve way of the things that work in education it's a panel called geap and the panel concluded that these two interventions were the exceptionally
00:04:43
the two two you know very coste effective and and successful interventions uh in education so they were really this sort of stood out so
00:04:55
it's not that so in other words what I'm saying is that there if there is evidence of success in in changing pedagogy which is sort of the goal or at least modifying pedagogy
00:05:09
which is the goal of teacher training the success is very clearly from being quite um let's say directive it's
00:05:20
in telling teachers do this then do that then do that then do that um that's based on I think a broader thinking that maybe the teachers are
00:05:33
insufficiently insufficiently trained or that they are you know they're unable to uh you know focus on the on on the right things so they need to be told what to
00:05:47
do what we're going to talk about is a very different uh kind of intervention a kind of intervention that actually
00:06:01
emphasizes uh sort of what sometimes we think of as being a primary quality of of teachers which is that their ability to to make
00:06:14
you think about things uh in different ways so it's an intervention where uh and I'll say a lot more about it um where the emphasis is precisely not
00:06:29
on content precisely not on the steps the teacher should take but kind of the opposite uh to give teachers tools or at least get teachers used to
00:06:44
the idea that their job is to to make people think to make them think about how to ask questions about how to how to pose sharp questions how to uh develop
00:06:57
uh arguments how to to um take things that are in your in your context and not in the textbook and turn them into uh some form of scientific observation so
00:07:10
very much engage engaging with um with the ground reality through a lens of science so very that's so the intervention comes out of
00:07:24
um um I think the motivation is uh sort of at one level very clear it's you know it's an invitation to think about um is it possible to get teachers to be more
00:07:39
more you know actively pedagogical you know being engaging with with the students uh make you know making them think rather than telling them things and that's a big that's a distinction
00:07:52
I'm going to emphasize the distinction between telling students things and making them think so I I'll come come back to that point multiple times it's
00:08:03
not that teachers don't often um complain so it's not that when you when in particular in the world of of uh you know structured pedagogy you you often
00:08:16
hear that you know teachers saying look but you know we would you know we don't particularly enjoy uh enjoy uh teaching students uh
00:08:29
in very um very uh you know ways in which give us no Freedom we would like to do the ways we things we would like to do things but this is not going to
00:08:42
work because this the school system demands that a certain material gets taught and a certain material gets taught uh requires us to follow a particular you know particular sequence
00:08:56
of steps so we we we better of sticking to the existing styles of teaching so it's a there's a very clear you know it's not so sometimes we we have used
00:09:10
the term uh tyranny of the curriculum es and I have used this term often and and it's certainly something that we're not going to get rid of so this is not in
00:09:24
other words we're not going to liberate teachers from the curriculum even though that's often what they say as a justification for being you know not being very engaging just kind of being
00:09:37
mechanical they often say that you know this is because we have to cover the curriculum we're not going to liberate them from that uh on the other hand there's another another body of work
00:09:51
which is which shows that maybe as a result of being very you know their falty to the curriculum they are often uh unable to use sort of obvious CH you
00:10:06
know for example Esther and others did these studies where uh they uh they prepared children very young children
00:10:18
preschool age children in um what are you know what you would call Pre arithmetic so they were and showed that indeed they can be prepared and they can be better at pre- arithmetic than than
00:10:34
um you know people who were in the control group so it can be done and this makes them more open to uh better at learning mathematical ideas and then
00:10:46
what they found was that left to itself that has no effect on when these children go into school they don't learn more because they have had this pre- arithmetic training so that that's a
00:10:58
kind of thing that suggests that you know teachers are uh sort of you know following a a track they're not engaging with the fact that these children actually could do more if they were
00:11:11
given a chance but they're not engaging with that so the as a background to I you know much of what I'm going to say is this uh puzzle can teachers
00:11:25
be encouraged Within the existing school system to be more engaging more engage with what the CH children know and think
00:11:38
and trying to get them to think more so is that possible within the school system or is it the case that given the structures that exist all you can do is change the structures you can introduce
00:11:51
structured pedagogy but you can't really you can't really get teachers to teach the same material differently so that's that's the question and it's sort of the it's a first order question because of
00:12:02
course uh if we think that teachers are a resource and in it's an enormous amount of investment that almost every country puts in they have they take some of their best talent they put them into
00:12:15
education systems that pay them a whole whole lot of money and uh and the question is can you make better use of them and is it or is it the case that basically you know you to all you can do
00:12:28
is you can move move you know tell them to do this or that or can you can you actually let lose their imagination and talents that's that's the question we're
00:12:41
interesting interested in going to tell you about an a randomized control trial of a learning tolearn program um the program is about getting
00:12:52
teachers to be um you know to to change the way they teach based on opening them up to ideas about you know how they themselves us uh learn
00:13:08
about the world so it's a in particular it's a teacher training program that and we look at whether a a teacher training program that focuses on critical
00:13:18
thinking on on being open being asking questions rather than a program that's about content so it's very specifically a program that's not about teing telling
00:13:31
the teacher here is more mathematics or more science it's it's very and I'll tell you how that's implemented but it's very explicitly not about not about uh
00:13:45
you know putting more content into the teachers it's and can we by just getting them to you know maybe articulate their thoughts differently or think differently get them to teach better
00:13:59
within the existing education system and generate better outcomes for the children so I'm not going to I mean I'll show you something on it but the focus is not
00:14:11
on how to whether by getting the teachers to do critical thinking or training the teachers in critical thinking or invite actually not training inviting them to be more uh
00:14:25
more involved in critical thinking can we improve the critical thinking of the students which interia would be a good thing too uh but rather on on can that
00:14:40
actually make the teachers better at the task that they supposed to do which is to get the the children to do better in standard exams so it's a very much of
00:14:53
focus which says we're going to it's not saying we're going to get the teachers to get the children to do something different it's saying we're going to get the teachers to do what they are
00:15:04
supposed to do and can they do it better it's going to be for so the intervention was um teacher training it was a I'll tell you more about the
00:15:17
training 40% of the children in each school on average were trained and we're going to look at the effect on upper primary children eventually I'll also talk about what happens with children when they go to secondary school but
00:15:31
we'll we'll come back to all of those things um so this is the headline result and it's it should impress you this says that 50% of the children pass the this
00:15:46
is a a board exam so an exam that the K the Ugandan government holds is not take a School exam it's an exam that comes is you know all children in Uganda in order to go go to high school have to take
00:16:00
this exam they have to pass this exam if you don't pass it you don't go to high school 50% pass in a given year in that's the control group mean in the schools that got this
00:16:14
training which I'll tell you more about 75% passed so that's that that's the headline result that you and you can see that that that involves if you look at the standardized scores it's a
00:16:27
distribution that's the blue distribution is simply shifted to the right everywhere it's not a it's not a matter of some children benefited all children essentially moved quite
00:16:40
dramatically to the right so that's that's the headline result and and that's the good news obviously um and when we look at these these um
00:16:54
in 4 years after they completed oper primary so they're now in high school uh we see uh that they are more likely to be in
00:17:06
school and uh you so they're not surprisingly since they more likely to have passed the exam um but it continues in the sense that they they in again in
00:17:20
Uganda people who fail drop out and you don't you see that the effect remains quite large it's it's still a um about um a quarter
00:17:34
larger the um so what what is what's the history of this program it's interesting the history uh history it in the
00:17:45
1970s um set of PhD students from the US who were mostly in the sciences and were frustrated with education and inspired by a Brazilian Ilan educationist called
00:18:00
Paulo Fier who who who of course many of you know about um Paulo frier inspired a lot of them by sort of suggesting that the education systems are broken and
00:18:12
I'll come back to Paulo freer's words uh quite often uh and inspired these these PhD students to move to Colombia to rural Colombia and start a an NGO called
00:18:27
fund the fund was a was a kind of a you know was famous um in in and but it was actually not teaching training teachers it was training youths to be better
00:18:42
citizens and better citizens by having a more scientific attitude towards uh questions to to be leaders of this communities for solving problems in agriculture there was a very much a
00:18:55
focus on the scientific mind mindset but a scientific mindset addressed towards youths in the early 2000s somewhat
00:19:07
entirely fortuitously I was on the board of uh a a funding vehicle that the hulet foundation had created and as a part of that we we funded an experiment for
00:19:22
moving uh this this um an experiment not a Riz experiment just an effort to move this fund program to to Uganda uh
00:19:36
the the kimanya NGO Foundation actually is is the child of that that set of efforts and um and so and then they
00:19:50
adopted the ideas of fund for training teachers rather than te training young men and women they were training teachers on how to teach what what do
00:20:03
they do so the the the uh it's a it's a quite quite an elaborate program in the sense that you have
00:20:17
um you know three three three separate phases Each of which you you have you go um once a once a week for training it's
00:20:31
it's it's it's uh for the full weekend so it's quite a lot of training what happens in the training what happens in the training is um an example is this which is that there are teachers first
00:20:45
make some shapes out of clay and then there is someone who's who is specifically designated as not being a trainer and the the the person who who
00:20:59
is their kind of their coordinator if you like is someone who is typically a young person with a high school degree often not less educated than the teachers who who's the animator for
00:21:13
this this Gathering of teachers and who ask them well what is that shape you've made and they say that's a cube and then they say well how why do you say it's a
00:21:26
cube what is it about what so what is what makes it a cube given that it's not entirely so so this is kind of a platonic question it's not entirely perfectly a cube it's not it has some
00:21:40
bumps what makes it a cube so those kinds of question through that's one example obviously uh you know to keep it going for a year you need more than one
00:21:52
question um so you you you get them to talk about Concepts to to be articulate about Concepts to get used to the idea that you know you can be asked random things
00:22:06
and then you have to figure out a way to have a vocabulary to talk about it so they get used to uh uh what the process is in a sense building a vocabulary for
00:22:18
talking about questions that don't fit directly into some curriculum so there's not a an answer pre pre-digested answers are discouraged and
00:22:31
they are encouraged to sort of formulate their own propositions so many many of the things that um that are emphasized and one is as I said prec using precise language
00:22:47
describing the world uh but also designing experiments you know and experiments doesn't again mean an RCT it means how would you know if this is true or not ask yourself the question how
00:23:00
would you know so thinking about those questions thinking about what are objective statements or subjective statement Etc so these were teachers were um you know
00:23:17
encouraged to to to I mean this sort of continues what I said before which is that um there
00:23:30
are they were for example they were encouraged to talk about people and you know and then you realize that when you say your your your perception of someone is different from somebody else's that
00:23:42
these tend to be subjective judgments as against objective judgments and so on the and uh they they're also encouraged to
00:23:58
understand that you know in by the by going through this process of asking questions of engaging with the questions rather
00:24:11
than rather than uh by uh you know reading from the textbook you you get not possibly the same answer but you get it in a different way that you come to a
00:24:27
and uh all of this uh is um is uh very much related to Paulo freer's thoughts that teaching should not be what he
00:24:41
called like banking he he he didn't like bankers and and he he he thought teaching was too much like banking meaning it's about putting things in an account in your head and they should be
00:24:56
about more like more about engaging with the material uh in a fluid way not about you know one person comes and puts something in somebody else's head so very much his you know if you read pile
00:25:09
of his he it's almost um the first thing he keeps saying um so it's and I think I think that that relates to these ideas
00:25:21
of you know not assuming an answer as uh focusing on asking questions not assuming that you know the answer and not assuming that there's an obligation to know the answer that the
00:25:34
answers are come out of our inquiries we don't need to know the answer exan so both the humility part of it but also the openness to data that you you once
00:25:46
you have the humility you can then say I don't know but we can figure it out so going from here is the answer to I don't know and
00:25:58
but we and it can be figured out so all of these are related thoughts um I'm going to let you read this it's uh not entirely
00:26:11
in perfect English but it's a transcription so um I'll let you read it um I think the this key sentence to me is even you
00:26:28
who you you who don't doesn't feel comfortable eventually by the end of the training you would have developed that confidence and comfort and uh and holding and so holding each other
00:26:40
because when you see someone is not very comfortable you you keep on you on trying to keep in and you find eventually someone comes in so it's it's
00:26:52
this idea that you know you you get to knowledge by exchange of ideas rather rather than by reading a textbook so that's very much this is
00:27:05
[Music] um all of this is a bit bit uh you would imagine uh hopefully these would be inter intermediate outcomes you would
00:27:18
think that the classrooms the the children would be more engaged they would be asking more questions so intermediate outcomes that will look at an engagement teachers would also talk
00:27:31
to the children more and then therefore and maybe talk to other teachers more because if the context is not one where we are supposed to know and therefore we
00:27:43
we don't dare ask questions because it will reveal our ignorance but rather we we know that not everyone knows everything but we can all figure it out then you can exchange ideas more
00:27:56
comfortably and uh you may actually in the process you may actually get to know more get to know more about your students as well because you know once you are open to the idea that your your
00:28:10
students are are um sort of you know they are they uh they are uh also potential sources of knowledge or they potentially you know
00:28:24
just another they'll have another view and you can learn from them you can talk to them rather than talk at them you can talk have conversations rather than announce things to them so by in that
00:28:38
process you might actually get to know more about the students so we look at if they know more about the students will know see if the CH the classroom observations are the students more engaged and things like
00:28:51
that and one hopes that they will spread to to other things where this attitude the the the students would bring this attitude home as
00:29:03
well it's a very small experiment uh it has um basically matched 28 schools matched um
00:29:16
and there were 80 teachers who were trained in each of two years the experiment was there was you know we matched so it's unsurprising
00:29:32
in a sense we matched Pairs and then we randomly chose one so it's unsurprising but you see that the treatment of the control are very similar there's no no no no no
00:29:46
uh so the P values are all very high meaning there's no close to nothing is even close to being um similar sorry dissimilar treatment and control
00:30:00
are very similar um each school was asked to send two to four teachers um it was the training was um 11 days as
00:30:19
a part of a the term during the term holidays they were structured into um typically into you know two days or 3 days at a time um and there were three
00:30:34
of such sessions so 30 days that's a lot of training that's that's the order of magnitude of training that many education systems will have which is one one month of training a year but it's
00:30:46
it's a lot it's not this is not meant to be a light touch intervention it was quite a lot of effort and the the effort was done um um 75% of the teachers did
00:31:01
all of them and 40% so that's the timeline uh it's the Baseline was in the
00:31:14
end of 2017 uh the first training started in the beginning of 2018 uh the second training uh so we
00:31:25
have data on student learning uh from December 2018 uh we have dece then we have a second round of training uh
00:31:37
unfortunately then you know what happened and so uh there's a we have school closures um Etc and so
00:31:48
when I show you results 4 years later this includes Uganda had actually a pretty pretty extensive School closure so you can see that schools were closed
00:32:01
for almost 2 years so it's this is all going to be when I show you the results of what happened to these children is going to be despite the fact that they were out of school for 2 years so you
00:32:14
were you might have thought all the learning might have disappeared so all of that uh unfortunately uh happen but we still fortunately we see some we see effects um
00:32:27
so that's the you know we're going to use the assignment not all teachers finish the program we're going to use the assignment of teachers uh uh for
00:32:40
the for the uh for the treatment okay so what I'll show you some results then just this is sort of goes back to what I said which is these
00:32:55
are results in the pl exam this is the public exam that you have to take to graduate from from uh primary school so in can in Uganda in East Africa in
00:33:07
general primary school goes till 8th grade so that's a so these are relatively older children so it's not they're not they're not they're not six
00:33:19
or eight or 10 they are more close to 14 and or 14 or 16 even they quite significantly older than what you would call primary age children in France um
00:33:33
so this is this is uh the effect on measured now now in in standard deviations uh you can see that it's 6
00:33:47
standard deviations Improvement in in science and English for example uh so these are for uh but you can see that it's you can look at the distribution
00:34:00
that's before whatever standard deviations means you might say but if you look at the distribution all the high scores come from the blues that's the core fact that you
00:34:12
should take away is that you really see that the the at the right tail so only the blues okay so that's that's where it's coming from it's coming from the fact that you some students just do much
00:34:27
better as a result of this um if you look at mathematics you don't see that and this is not accidental because Math teachers
00:34:43
were not trained the the schools were unwilling to let the Math teachers go so the Math teachers were not
00:34:53
trained I will uh so so social studies um the pass rate in the pl exam uh but you even with math you start to I I'll I'll come back to this point
00:35:09
you start to see maybe some some little dribbles of that because and that's not surprising the children are being uh asked to be more you know more open more
00:35:22
critical thinking on their feet more so they might actually since the evidence in mathematics and this is a some of our work with Esther for example shows that
00:35:35
for example we have some work which shows that uh children who are who work in markets in India who are who sell things
00:35:47
and therefore are extremely good at calculation you mental calculations they they are able to give change within seconds and give it correctly when you give them the math exams they
00:36:00
give up and they don't do it they they can't do it they don't do it and uh that's that's that's suggesting that there is innate ma mathematical ability that's not being used by the system so
00:36:13
if you think about what this program is doing even though it's not training the Math teachers it is training the it is training the children to be more confident of what they know to be to use
00:36:27
their own knowledge uh and so in that sense maybe you could see an effect on math and I'll come back to that point can you sort of see it a little bit little bit little bit uh maybe at the
00:36:39
right tail but I'll show you some results later now how does how do we then there's a then then the this sorry this
00:36:51
this is um this is what the kind of exam they had uh I don't know that it's particularly Illuminating um but it's it's a kind of exam they
00:37:04
have um these were this is what you know the curriculum says and sometimes curriculum say many things uh so to see if uh actually
00:37:30
something more specific than just you know you look at that exam maybe all you learn is to tick off boxes is is there something more that's happening is there a real increase in
00:37:44
scientific understanding so to do that we did something which was um I think um an innovation in Uganda we we organized a series of science
00:37:58
shows okay this was this didn't exist but it's a it's a it's a place where you go the we had some people who were I know generally recognized as experts who
00:38:10
who were were and I guess not hired were were brought as judges and then all the children from the treatment and
00:38:23
control schools were brought to uh this science fair basically and they were all told the children were made groups of um I don't remember how many actually
00:38:37
um but you know I think four or five and they were made into groups each School sent three or four groups to to the to the to the science fair and they were given and they had to come up with
00:38:52
projects that um for example uh either on conserving uh resources in the community or like or preparing safe
00:39:06
drinking water and they're supposed to do design a strategy and an experiment to evaluate it and all that so they were asked to do a science presentation
00:39:17
basically at this at this fair and that was true in treatment and control when the reason we did that is as as you might imagine we were worried
00:39:31
that maybe they are just getting a little bit better at you know you saw those you know very ambitious but maybe not very uh ambitiously implemented uh
00:39:43
science curriculum and maybe they get a little bit better at this but maybe they have have is there a a deeper change you see and the judges were you know told
00:39:57
what that they should judge the framing of the problem the designing of the experiment the hypothesis the outcome measure and the articulation of the of
00:40:08
the whole story um and so you get one if you had no idea so on these 15 I think on no on on 12
00:40:21
measures for example on measure three they were asked um to say the the judges were asked to say uh did the children have a clearly
00:40:33
articulated hypothesis and they could say no uh I had no idea what they were testing or they were a very clear hypotheses so that that's the judging
00:40:48
rubric so that's what we find basically almost all the winners in this came from the treatment schools so you can you
00:41:02
know you can see that the um the index is is uh hugely significant and very large but you can see the distribution the
00:41:15
distribution is as you as you move to the right uh you start to see mostly blue and that's uh in other words the
00:41:26
and overall all the only win all the school level winners came from from the the which was the average of the school all the winners came from the treatment
00:41:40
schools so you you know they were simply much better at this activity so they had learned something which was kind of the which match the intention of the
00:41:53
program which is to get them to be more more better at formulating hypothesis testing hypothesis articulating hypothesis so that and you see a a match
00:42:07
between them um and you can see these different competence categories you see sign significant results on most of them as well
00:42:25
okay the other things we also wanted to do were to use other more standard measures of you know of higher order learning and creativity so these are
00:42:36
standard measures taken from the from the educational practice literature so this is there's nothing these we didn't innovate on um
00:42:49
and on all of them you know the different measures we see again G very large and very positive effects so these are really uh you know these are now now
00:43:06
in standard deviations uh and you see you know half a standard deviation or 3/4 of standard derivation gain in these so these are very large gains and again if you if you don't like the standard
00:43:19
derivations you can see that the blue bars are doing very well m on all of them so that's sort of the first fact
00:43:36
which is that it it worked it it got uh the these young young men and women to be much more uh you know scientif you
00:43:49
know better at scientific method better at at uh sort of higher order learning better at taking exams you know at all levels it worked what what can we say something
00:44:03
about what happened actually that's you know is it can we see change not just in what they do but what happens in the
00:44:15
classroom and I mean it's uh so so this these come from um from uh classroom observations so these were
00:44:36
has the have the usual problem of classroom observation if somebody was sitting in the classroom and taking notes on what's Happening uh you have to you could worry about maybe these
00:44:50
children are more confident and therefore they are more able to uh they're more willing to ask questions in a in a when there is an observer so it
00:45:01
may I can't tell you that that's not true so I mean there's some some danger that what we are observing is a bit what we put in which is that these kids are
00:45:14
more confident and we they are therefore asking more questions so I'm not saying I can't say that that didn't happen but we do see again uh the fraction of uh
00:45:25
children children who are engaged who are by different different metrics of being engaged um are is is larger in in in treatment than
00:45:39
in control so if you look at the bottom right the graphic uh the the blues are learning kinds of things
00:45:53
um the pinks are doing doing something else chatting with their friends okay so that you can see that the chatting with their friends goes up goes down and
00:46:05
being engaged with what is being taught goes up so there's a this is sorry I should say this is when the teacher is not present when the teacher is not present are they
00:46:21
discussing what they heard in class or they're chatting about something else and they're more likely to be discussing what's happening in class than chatting with
00:46:34
what so we we asked students to recall whether they ask teacher with help and which teachers
00:46:51
um and we also ask them if they got beaten and I can tell you that so
00:47:04
that corporal punishment is very common in Uganda I'll show you the means in a minute so this is not this so in other words the fact that the children were
00:47:18
intellectually more engaged didn't change the disciplinary climate in the school I can you see that in a minute corporal punishment didn't change at all
00:47:30
53% of children uh classrooms had corporal punishment um but um and that didn't change at all on the other hand um children asked more
00:47:45
questions the fraction of children asking questions during the observation went from from 22% to 28% so a quarter increase in asking questions so that
00:47:58
there's again some evidence that this is happening at the classroom level it's not just something that's somehow in the air it's be behavior is changing now the
00:48:10
teacher interesting uh how do the teachers we I I suggested that if you if you you think you're the students are more more likely to be a source of
00:48:24
knowledge if you can talk to them as you know knowledgeable beings rather than vessels for filling or bank accounts for pouring money into uh then you might
00:48:37
actually also get to them know know them better um so we ask students a bunch of things about themselves basically three questions did
00:48:49
you did you attend school yesterday did you attend school day before and who do you live with in Uganda in particular a lot of these upper primary children
00:49:01
don't live with their parents because their parents live in remote places and therefore they have to go live somewhere where there's a school an upper primary school so they often live with their
00:49:13
uncle or their you know some other relative so so these were the three questions we asked them and then we ask the teachers whether they knew the answers
00:49:25
to this questions so do you know who your students are and particular did they come to class did you notice them did they did they uh did you entirely um
00:49:38
you know did you notice who was there to days ago the other set of questions are about teachers and talking to other teachers you you know you again if it is
00:49:52
a more of an OP if you are more confident about the fact that you're not supposed to know the answer that you're supposed to you know find out by talking to people then you might actually be
00:50:03
more willing to exchange knowledge with others because you are uh you're I mean there could be other things happening maybe they made friends when they went to the went to the uh to the uh learning
00:50:18
you know when when they went to the program that was uh you know getting them to have this attitude so I can't I'm not going to say that this is proof
00:50:29
that they were talking more because of a different attitude to knowledge but they they it's it's consistent with it so we find that
00:50:43
teachers talk to other teachers more and teachers know more about students uh and teachers know actually a fair amount interestingly 60% of the
00:50:55
questions I get right so that's quite a lot they no a fair amount it still goes up by um another 10 percentage points and uh again they they do talk to other
00:51:07
teachers but and that goes up so reassuringly you see all the symptoms of a better functioning school we have other results uh we can
00:51:28
we we look at whether teachers are using more precise language for example that's one of the remember that was one of the starting points of the of the of the uh
00:51:40
kind of the pedagogical exercise was to get them to use more precise language and they do um that it's not because they know um more more it's because they
00:51:52
use so you ask them content questions they are about equally good so the teachers don't learn things and that's of course intentional because they were not taught anything they were just
00:52:04
taught uh they were just encouraged to discuss so it's not that their knowledge goes up what goes up substantially is their use of language of asking more
00:52:16
questions um and they are in general more sympathetic to the children we have measures of these things so here here's
00:52:33
um uh especially you can see that in in treed schools in general [Music]
00:52:43
um teachers are um are you know they they are they are more involved with the science shows that I told you about um [Music]
00:52:56
um they are and that's true at different points in the distribution it's generally true they are they use more precise language
00:53:08
um they're more gentle in the language they use not doesn't stop them from beating the children um finally uh what happens in the long
00:53:21
run so we go back remember this all of this ended in 2019 you go back in 2022 to look at what happened are these so what's happening
00:53:33
to these children so uh the children are more likely to be between 8 and 12 percentage points more likely to be in secondary school they're in better schools also they are in secondary
00:53:47
school they're in better schools um and then uh the other thing we can do is we can ask is is it the case that uh the
00:54:02
um in these schools where training there were three rounds of training the last round of training was in 2022 um do we still see treatment effects so is onetime excitement was it
00:54:16
onetime excitement but no it's you get very similar uh treatment effects and now you start to see it on math the culture seems to have changed you see
00:54:27
effects on math this is why I I suggested that there might be an effect before you start to see an effect on math um on in in all of the grades
00:54:47
actually so um so the and this this this is the effect on on on uh being treated in 2018
00:55:00
so uh so the teachers were treated in 2018 this is learning outcomes for children who were treated in 2018 in 20122 and you can see that they are
00:55:14
still doing uh significantly better in math and reading and in actually everything else but they're not all significant so so they 4 years later
00:55:28
after the their teachers four years after their teachers were trained they retain this is a they retain uh the learning gains they stick for
00:55:41
um it's very cost effective because in a sense uh it you the person who does the the animator for this these programs is not expensive because he's not a he's
00:55:55
not a specialist he's just someone who's trained by the NGO to to be you know good at getting people to talk it's not but he's not actually doesn't need to be
00:56:07
a specialist by himself so he he this is one of the most one of the cheapest ways to improve test scores according to
00:56:19
this so overall um so this this this this program is um was a success it was a success I think largely because the teachers
00:56:34
were were you know they were they seem to have imbued the uh the lessons that the that the the NGO wanted to get which
00:56:47
are not is what we almost always say but rarely unfortunately do which is teachers should be mostly in the business of you know asking questions
00:57:00
and helping people formulate questions in in helping them think through the answers rather than delivering answers to people so that's something that is often said it's say all and at least in
00:57:14
this context of uh in many of the contexts that we have studied in developing countries rarely done the I think the good news is that it's not so hard to do is doable the
00:57:26
teachers are open to it there there was we we didn't have any particular uh political problem with this because it was not never told as you are going to
00:57:39
be uh you know you're going to be uh given all this additional knowledge you just told this is this is for you to you know discuss with your colleagues and
00:57:52
you know get better at discussing and and and there was a it was politically it was not uh there was no resistance to
00:58:03
it and we we uh we will what we so we this program continues we what we don't know I told you that there could be effects at the
00:58:16
community level we're hoping that we go back and find those we see if these um you know this scientific mindset after all these now they are 20 many of them
00:58:28
or 18 and they can they are obviously you know themselves uh able earning earning and so are the labor market effects are the effects on the community
00:58:40
are they is are they organizing using the scientific knowledge to make agriculture run better for example all of these questions we are uh interested
00:58:52
in and will'll maybe in a year or so have some answers so I'll stop there thank
00:59:01
[Applause] you f [Music] at right
01:00:53
l [Music] C [Music] no d [Music] [Applause]
01:02:53
energy [Music]
End of transcript