Select text to annotate, Click play in YouTube to begin
00:00:00
[Music] foreign [Music]
00:00:18
[Music] [Music] and so I have the pleasure to be one of the co-chairs of easy with Schuler and to introduce this session about our work
00:00:43
in in Europe so actually let me start by introducing the two speakers and then I will tell you a few words of Izzy and then you will have some results of studies that have been conducted
00:00:57
as easy funded studies are as precursors of independent studies so let me start um with the Eliana so Indiana is a professor of Economics at Buckner University and chairing in development
00:01:11
economics she directs the laboratory for Effective anti-poverty policies she has conducted research on development economics political economics with a specific focus on on social norms
00:01:23
aspirations which will illustrate today she has also worked on political constraints to development in Africa conflicts Etc she has been the president
00:01:36
of bread which is a great organization in development economics uh she's been also the president of the European economic Association and last but not least she is an active Japan affiliate
00:01:48
and a contributor to to easy so we'll have the pleasure to to hear you in a minute and is a professor of Economics at the European University Institute in Florence
00:02:01
he also agent professor of Economics at Bill Kent University in Turkey and so your leading uh the humanitarian and the displaced livelihood initiatives at Japan so this is a teacher because in
00:02:15
a minute I will tell you more about about these initiatives as I have a chance to co-chair a GLI with you um I'm sure has worked a lot uh on the role in particular of schools of
00:02:28
Education in the social inclusion process and this is work that has been very influential in the economics profession among policy makers as well so I'm sure we will learn and a lot from
00:02:43
your presentation but before that let me set the stage a little bit and talk to you about Izzy so hopefully at that stage of the
00:02:55
curriculum you've realized no one in the room anymore believes that research against poverty can be done by a researcher alone in a lonely way in the Ivory Tower if you have if you still
00:03:08
believe that I hope we have a chance to definitely change your mind and so it's definitely a collective Endeavor each project as you saw with
00:03:21
this morning involves so many stakeholders um starting of course with those in poverty themselves but there is something even more correctly vegetable
00:03:33
which is a research initiative so what is this thing uh well let me tell you um concretely a few words about one of them is he so is he
00:03:46
is this broad Endeavor that Josh already mentioned in particular triggered by the refugee and migrant waves and that that Europe has known uh recently and so what a research initiative does is to Foster
00:04:00
research and Outreach on a specific Topic in that case social inclusion and basically led by dedicated jpal Associates who do
00:04:13
most of the work of the initiative already to to Foster the activity what we do in initiative is a frame and agenda try to
00:04:25
relate it to the broader literature think about where we have a comparative advantage where studies are badly needed selecting and then funding research project sharing the results with policy
00:04:36
makers and with the broader audience and so is he is still a a little girl in the you know in the galaxy of initiatives
00:04:47
as it was founded only uh four years ago after let's say two years of gestation more or less and so it started uh like other initiatives by thinking about its
00:05:00
agenda what social inclusion exclusion meant and what social inclusion could be there was a long literature review process there was also maybe I should insist on that a scoping exercise where really you meet with potential of
00:05:12
implementing Partners you discuss with policy makers you try to see what the main issues are Etc and then basically you you call to the broader Japan Network and say okay we would like good
00:05:23
proposals and if possible especially on these topics of course if you have other questions of Interest don't hesitate but yeah we try to to foster a kind of collective effort
00:05:36
so most importantly this is this is something that we do only based on the literature it's something that we do by talking basically we talk a lot to people during these two years that come before the initiative and that basically
00:05:48
leads us to say okay maybe as a first priority we're going to start with the education for migrant children that was kind of the outcome of this of this process then a key step is and that's really
00:06:00
initiative at work you know compared to what a single project would do is to put lots of people in the same room lots of potential implementing partners and so that happened at Pac I think in 2020 and we call it the incubator you know we are
00:06:13
in the startup Nation so uh that's the incubator and what's that it's a three-day Workshop Where We Gather potential implementing partners and we brainstorm and out of these contacts between researcher potential
00:06:27
implementing Partners uh some projects uh start and and then apply to call for proposal Etc and Europe and some of the projects that will come on the next slide are actually the results of this
00:06:41
incubator so it's really a privileged moment for researchers and implementing partners and policymakers to to discuss and to go from the big picture to very specific projects that in turn will help
00:06:53
us understand the big picture better so in the interest of time I want to tell you about all the projects that Azia has funded anyway as it is a baby initiative it's still a bit too early to give you many results but as you can see
00:07:06
it has spread already it has funded quite a few projects and it has a little bit uh change the focus the concentration of project which is very to be very much in France so thanks to it it's gone a little bit further away
00:07:20
now and so there are pilots projects there are large-scale rcts Etc and of course at the 30th birthday 30th anniversary of Japan we will show you
00:07:31
all the results of this project and many others so I'm aware of this presentation is probably way too short but also a bit you know I need to stick to the time and I haven't had the stop sign yet but I I
00:07:44
will stop now to leave the floor to Schuler and to Eliana if you're a bit frustrated because we didn't enter into the list of topics of this
00:07:58
supported and the study supported by the initiative then you can just scan this QR code and you will be led to the initiative website where you can learn a lot more but don't do that now no you
00:08:10
should first listen and to uh Schuler and Eliana and so that's me have sugar first thank you very much [Applause]
00:08:24
so the stop sign comes from there okay thank you thank you very much for inviting me uh to present my work to such distinguished guests today I'm
00:08:38
going to talk about an agenda and what we learned from that agenda that I initiated in 2018 with the very surprising support by Jay Paul we weren't expecting and then it was a
00:08:51
wonderful wonderfully timed um support and later on uh later by Jay palacey so this this this project started in 2018 and I'm going to give you some lessons what we learned this is about the the
00:09:05
project the aim of the project was at the time and still is to understand how we can build social capital and have how that Social Capital rebuild especially
00:09:16
in schools can shape the way we live and can shape the the prosperity Prosperity we actually aim for for our lives so um just to step back and tell you
00:09:29
what I mean by social capital and prosperity so just I don't think I need to convince you at all that we human beings we all pursue are or our own
00:09:41
interests to thrive in life to be successful but we all know that we can't do that alone so we need our fellow human beings to be successful to to to to
00:09:54
um to live happily we just we cannot exist without other human beings so and we are species that are um um that are um cooperative and we are uh
00:10:07
very great cooperating if we want to uh if we have uh if we have an aim and we have an objective that requires cooperation we are pretty good at doing that because our survival often depends
00:10:20
on this cooperation so economic prosperity and social Prosperity requires Social Capital so cohesive and non-toxic environments when
00:10:33
we talk when I talk about this this could be in the country context where the the environment should be non-cohesive you should not be in a war you should not have Intergroup conflicts you should not have different religious
00:10:45
factions or different ethnic factions are fighting and it could be in a a big large corporation context we also work in and departments should not be fighting with each other within departments should not people should be
00:10:58
nice to each other and cooperating with each other and in a school context which I'm going to talk about is is the same the kids should treat each other well and the teachers should treat kids well so the environment should be nice and
00:11:11
cohesive that's what we mean by Social Capital so in an environment where we think social capital is high is the environment where we see a lot of social connections a lot of friendship a lot of
00:11:24
bombing and overall a collective will to live for for the case of a country and a community we also see a lot of pro-sociality prosocial behavior like we
00:11:35
see trust we see reciprocity we see cooperation and tolerance for differences whatever those differences mean could be and we also see law relatively low intra
00:11:47
and inter-group violence and conflict it could be inter ethnic interracial we see less of those in cohesive and communities where social capital is high
00:11:58
and of course we see less of segregation that could be ethnic segregation or related to social socio-economic segregation we see less of those yet
00:12:13
yet growth exclusive environments the social capital is excellent for economic growth it makes you rich it makes you happy yet we do see non-cohesive environments we see violence we see conflict everywhere in the world
00:12:27
in many cases this has the historical Roots you see a country you look at a country you see actually the very tender fault lines ethnic fault lines racial fault lines and or sometimes on top of
00:12:41
everything as something it's socio-economically turbulent thing just comes and hits the country like Mass migratory flows or human
00:12:53
humanitarian disaster human displacements which I'm going to focus on today so that kind of that kind of movement can also threaten Social Capital social cohesion you already built however good however uh week it
00:13:06
was but it can extra it can shock what you have built already in those cases we might need economic and social policies to actually
00:13:17
prevent uh stuff going south further or or policies that actually see this coming and mitigate what's coming so what I'm going to show you is we can do this in by by using by
00:13:32
leveraging the power of public education we have been doing for centuries now and I'm going to show you this can be done and I'm going to show you in a specific laboratory called what I call Turkey
00:13:45
okay turkey is a country by the way it's it's name changed but I still use turkey so I'd be called we call it turkey now so turkey is an ideal laboratory to study inter-ethnic cohesion and so how
00:13:57
to build and how to lose Social Capital how you could rebuild in schools uh for for very basic reasons turkey itself is an ethnically diverse country it already
00:14:11
has a lot of some tender religious and ethnic fault lines on top of it since the Civ the start of the Civil War the Syrian Civil War in 2011
00:14:25
turkey experienced a large influx of Syrian refugees many of them remain in the Border regions of turkey but a lot of them actually moved into Istanbul and Ankara
00:14:37
kind of cities where they could actually find economic opportunities what happened was with the with the with the agreement between turkey and EU
00:14:48
starting 2016 Ministry of Turkish uh Ministry of Education in Turkey started placing these Refugee children into classrooms and it kind of organized nice organizing manner in the in 2016 at the time the
00:15:02
policy looked quite successful that in a very short period of time changed the ethnic composition of public schools and these children at the time are those
00:15:15
who don't speak any Turkish obviously they go into Turkish schools they don't speak any Turkish and and a lot of them at the time had some some kind of War trauma so that
00:15:28
um that overwhelmed the teachers and Turkish teachers usually well educated and very well trainable teachers actually but they were overwhelmed they were over
00:15:40
overwhelmed by the sheer number of them basically so and they nobody actually it was an orderly orderly movement but no one gave them a real clear road map how to do this if you all of a sudden you
00:15:53
have you have 25 kids all of a sudden you have five extra kids who cannot speak the language and what can you do what you what you should do and the culture is is not approximate not that
00:16:04
not that close so we needed to do something about this so just to give you at the Baseline in 2018 when I actually started getting into the classrooms what I have seen before doing anything this
00:16:17
is what we have seen we have seen great heterogeneity in some schools in some classrooms great integration in some classrooms great segregation so what I'm going to
00:16:29
show you is is it you see two classrooms here from the same school same school two third grade classrooms what you see here every every dot every head is a kit
00:16:41
a a third or fourth grade kid I can't remember now but the thing is what we have here is the code to classroom and we say in secrecy tell me who your friends are in this classroom so if I say okay look
00:16:55
it's my friend in secrecy my the an arrow goes from me to look and of course the same question is asked to look who's your friend in secrecy he may not nominate me obviously but he may
00:17:07
nominate someone else so what you see here is in the first classroom in the same school you see the host children which is represented green they hang out each other and you see a
00:17:21
bunch of refugee children who nominate each other as their friends this is what we call a social segregation within a classroom yeah so in the other classroom literally next door you see a great
00:17:32
integration so everybody is friends with everybody of course you see some isolated kids over there we call a social isolation the kid says I have no friends in this classroom so but we do have that but overall you see this
00:17:45
difference before we do anything so we have some work showing that teacher has got a lot to do with this teacher's attitude toward migrants teachers attitude toward
00:17:59
Refugee kids have could create a classroom like this or a classroom like that so knowing this we decided and and although a lot of the
00:18:13
things when you show this to the teachers teachers talked actually because a lot of issues don't do what they do by just but with the deliberate act they don't want to segregate those kids or anything like
00:18:25
they just their practice is actually lead to that so what we did in the end is like okay let's let's give them something let's give them some tools to to do this integration better to make the classroom more integrated not
00:18:38
segregated so we created an educational program and then this program uh just it was a curriculum program but it was an extracurricular program that was uh
00:18:51
given to the kids through teacher training and it it emphasized basically what we call what we call it perspective taking or what the psychologists call the cognitive empathy cognitive empathy
00:19:02
is your ability your cognitive ability to put yourself into other people's shoes as simple as that so and that is teachable and that is malleable especially for kids so we actually gave the teacher tools to teach these to the
00:19:16
kids so we evaluated this program after a year we used 80 primary schools in southeast turkey where a lot of refugee students were already placed for the treatment
00:19:27
for the control schools Baseline 2018 and line 2019 April for the short term and then we were able to talk collect thanks to SC again collect long-term
00:19:40
data after the schools open three years later the implementation the schools opened after covid-19 pandemic just to give you an example of one classroom after the intervention this is
00:19:53
a treated classroom very segregated at the beginning this is the same classroom this time you're looking at the same classroom before and after now okay I know before and after doesn't mean much A lot of the times but you see here an
00:20:07
extreme case of extremely segregated classroom the refugees is our host is age refugees are all friends with each other host friends with each other the same classroom one year later becomes a
00:20:19
very very integrated classroom after the program moreover we see a significant reduction in um social isolation what was social isolation
00:20:32
declaring that you have no friends so your isolated kid you have no friends we see a great decline in overall but very much so for the refugee children so we have a lot of refugees children saying I
00:20:45
have friends I mean we look at those the structure of those friendships we see more reciprocity so I say here is my friend Luke is my friend he's Syrian and he nominates me I'm a friend who's Turkish so we see these inter-ethnic
00:20:59
friendships a lot more what we also see is significant increase in trust reciprocity and cooperation among the kids so they trust each other more not only themselves but also some
00:21:12
some unknown The Stranger which is very important there's a huge literature in economics that shows trust is highly correlated with economic Prosperity you have to trust each other to actually get
00:21:24
into any economic transactions so this is very important we we place a lot of weight into these um these results what was surprising for us the program
00:21:36
did nothing to teach Turkish or did nothing academic we did not aim to do so because these kids were coming and going to school as long as they come and go in school they learned the local language anyways
00:21:50
here is a treatment had a huge impact on the refugee children's Turkish ability without teaching them any Turkish but which makes a lot of sense because you you introduce a program that puts the
00:22:04
kids together they play together they hang out together of course they learn the local language you don't need to teach them separately this is this is the result we use for the ministry to show because they were trying to
00:22:17
separate the kids to teach them Turkish first I said this is the this is not really gonna work for the little kids you have to put them together you have to put them together and let them play football for them to learn the local
00:22:28
language nothing else so so so this is hopefully going to work in the in the future the long term in the long term we see still some results persistent despite
00:22:41
the fact that we had a devastating pandemic in between and we lost a lot of refugee children they just disappeared from the system some say they some some went back to Syria some went to other
00:22:53
cities we don't know but we we lost a lot of them so they are not in the education system much so but we see with the remainders the probability of
00:23:05
forming inter-ethnic friendship is still very high among the kids who got this program the effect on trust dissipates a little bit and we have good reasons for this and the effect of language training
00:23:18
teach language Turkish language also dissipates that's normal because everybody eventually learned the local language so that's that's not surprising so I want to end with this this slide I
00:23:30
wasn't sure should I put the slide should I not put this line I was in the field in in may just recently yes recently is launching a new study that involves all sorts of refugees
00:23:42
uh things are not looking good not not looking good at all because in 2018 when we were in the field we weren't seeing much of that so we see now residential segregation appearing we
00:23:55
see now School segregation we've seen our exclusively Syrian or Refugee hosting schools unfortunately so do we see also huge absenteeism among uh Syria
00:24:07
the refugee children they just come to school and they don't come to school for another month and they come back again so the language local language is not developing so um
00:24:19
that's why they don't learn much unfortunately so on top of it I don't know if you're following so we have now a lot of hate speeches and violence is on the rise and recent elections have
00:24:31
been using this as a rhetoric and it's very sad but I'm optimistic I'm very very optimistic because I work with children when you work with children you can do lots of things with children so in
00:24:44
schools so I'm quite optimistic that we can actually push this agenda forward with the help of Jay Paul and and and then and and create create social capital in an environment like this thank you
00:24:58
[Applause] thank you sure and now let's have Indiana thank you very much and it's really a
00:25:18
great pleasure and an honor to be here so I want to thank Esther Abhijit and jaipal Europe for inviting me to this wonderful event and what I'll do today
00:25:30
is talk a bit about supporting social inclusion through education in Italy so education is as we all know a key determinant of Labor Market outcomes and
00:25:45
social mobility and it's unfortunately often the case that marginalized groups in society do not get access to adequate educational opportunities so what I'd
00:25:59
like to do in my 15 minutes today is to discuss two policy interventions that I've been working on in Italy that were aimed at reducing gaps in learning and
00:26:12
in opportunities for disadvantaged students the first one is going to be called EOP equality of opportunity and it's a career counseling and tutoring program
00:26:25
that we implemented about 10 years ago with the goal of improving high school track choice for middle school students immigrant students in particular the second
00:26:39
is called top and it's a tutoring online program this one is younger it was born in 2020 during the pandemic and it's a remote tutoring intervention
00:26:50
that we launched with the goal of coping with school closures but then eventually scaled and maintained also when schools reopened and both programs were
00:27:04
evaluated rigorously using randomized control trials and in the case of the tutoring or line program actually j-pal helped us fund one of the editions of this program
00:27:18
so thanks so let's start with the equality of opportunity which was a project I worked on together with Michaela Carolina from Harvard Kennedy School and
00:27:31
Paolo pinotti from bocconi so the Italian schooling system like many systems in Europe actually stratify students into tracks when they go in
00:27:42
high school so in Italy this happens after the eighth grade when the kids are around 14 years old and they have to opt into one of two three trucks the most
00:27:55
demanding is called lycheo and it's a academic oriented typically students after the niche will go on to college then there's a middle track called technical education and here you should
00:28:09
think of accountants or people you know doing becoming a chemist for employment in the private sector and this can all open the road to college but you can also get
00:28:22
white collar jobs straight out of technical education and then the lower tier is vocational schools which typically leads to Blue Collar jobs and you cannot go to college directly you
00:28:35
would need one extra year to make up for some of the studies you didn't do and different from France in Italy there's no restriction you can really enroll where you want okay the selection
00:28:47
happens later if you fail to pass the grade but the school somehow tries to influence your choice there is an official recommendation that teachers provide to the student and to the family
00:29:01
it's not binding so you can disregard it but if you imagine families that don't have a high socioeconomic background and that may not know very much about the system like immigrant families very
00:29:14
often follow this recommendation and I want to show you what the track Choice looks like for Native Italian students and immigrant students when we do our
00:29:26
best to take students that are comparable in terms of ability so what you see in this graph is the probability of enrolling either in lychell or technical education which we're gonna
00:29:39
call the high track for Native Italians and that's the green line or for immigrant kids the red line and you see five bins because those are
00:29:51
the five Queen tiles of ability as measured through a standardized test that they took three years earlier okay so the people on the left are going to be the lowest quintiles the least
00:30:03
academically uh performing kit and of course as you move to the right you have the better kids so these lines are positively slope it means that as you as you're better in school you are more
00:30:16
likely to enroll into these demanding tracks but you see there is the wedge a pretty consistent difference between native Italians and immigrants and the immigrants are systematically less
00:30:28
likely to go into this good high schools even though the system is not preventing them interestingly this is the graph for boys if you look at girls you still see
00:30:40
a gap in the lower Queen tiles so when the girls are not very proficient they're still making different choices but they High performing Girls in the
00:30:51
right part of the graph in this case whether they are immigrant or native Italians they make the same choices and this is per se an interesting fact but let me not spend time on this let me
00:31:05
tell you what teachers might have done to contribute this in this graph you see the probability that the teachers have recommended the high track when they sent this message back to the kids
00:31:19
family and again the blue line is for Italian students the red one for immigrants this time around both for boys and for girls if you take two kids who are performing in a similar way when
00:31:33
you when they take tests and that's the horizontal axis you see that the teachers are more likely to say oh if it's a native Italian they should go to the better schools okay
00:31:45
so what did we do with the Ministry of Education we implemented a program that was targeting High performing immigrant students in middle schools and the goal was to help them make choices that were
00:31:59
consistent with their academic potential in particular the program was providing two types of activities one was career choice consultancy basically meeting with counselors and getting some
00:32:12
psychological support to reinforce your perception that you can achieve the goals you set for yourself and this was done during grade seven and eight for a total of 14 meetings and then for those
00:32:27
who were not so good in the Italian language there were some tutoring activity on the side because the concern was that if we push the students to aim
00:32:38
at demanding schools but then they was they were less prepared we might actually not be doing their best interest and the total samples of schools that we
00:32:51
worked with was 145 in five different provinces in Italy and in each of these schools we took the 10 immigrant children who were best in that school
00:33:04
okay so different from some of the work that's been done here in France with immigrant children where the target were the lowest performing immigrants this program was a bit more of an upward
00:33:15
Mobility type of intervention so we wanted High performing kids and in the treatment schools these 10 kids received this counseling and tutoring and in the
00:33:27
control schools we simply collected data on them so here you see the results the blue bar on the left tells you what fraction of these immigrant kids
00:33:40
chose the high track so delicious or the technical in the schools where we did nothing and you see it's something around 67 so it's not too low because these are the 10 best immigrant students
00:33:53
in the school okay the white bar to the right is the choice of native Italians that were comparable when they started okay so this is not the entire population of Native Italians
00:34:06
otherwise the bar would be higher but comparing the test scores this is a group that was similar when they started middle school and so if we had done nothing the Red Bar in the middle would
00:34:20
have moved like the blue bar because you know that's the counterfacture we're using and instead two years after our intervention their choice had basically become the same as that of the Native
00:34:33
Italian so what we did was we closed the Gap in you know aspirations educational aspirations of these immigrant boys when we look at girls
00:34:45
nothing happened why because you we saw earlier that the girls were not choosing any different from the Native Italians already this is something that we didn't know before starting the program because
00:34:57
the data had not yet been matched so we could have saved some money if we do and just targeted boys but we learned through the program that this was one of the fact and we tried to investigate why
00:35:10
was it the case that they changed their choices and we had a few potential mechanisms it turns out that students motivation and teachers recommendations
00:35:22
played the biggest role these students also did better in school we find effects on academic performance but in this mediation analysis the performance was not really responsible for the
00:35:36
choice so let me move to the second Intervention which is the tutoring online program and this is also joined with Michaela so top I'm gonna call it
00:35:47
uh is a program for disadvantaged middle school students were disadvantaged in this case is not equivalent to Immigrant is more General socioeconomic status
00:36:01
language barriers learning disorders and it has two key features the first is that the tutoring is entirely dying online there's a lot of research in education on tutoring programs most of
00:36:14
that until recently was in-person tutoring the second characteristic of top is that the tutors are not professionals they are not teachers they're actually volunteer University
00:36:25
students who are trained and supervised but the fact that their volunteers has two advantages first it costs very little but second there's a lot of you
00:36:38
know commitment that these students have and this intrinsic motivation helps in the relationship with the student so top started in the spring of 2020 in
00:36:49
May you may not remember I remember very well when the first covet case was announced in February and two weeks later schools shut down and they did not reopen until the fall okay and uh
00:37:03
Michaela and I uh immediately started having these perceptions that this was going to be a disaster for students because Italian teachers and the Italian system was not really prepared for remote learning but also for families
00:37:18
that were where the parents couldn't help the kids and at the same time being professors in universities we started seeing students who felt very lost they were themselves in lockdown they felt
00:37:30
you know what can I do to help with this situation and we decided why don't we ask them to help kids who are in school and tutor them for free so between April
00:37:42
and June we implemented the first version of top you will understand why we were able to evaluate it but the results seemed really positive so we offered a second
00:37:56
edition and now there's been a third and the fourth and these last two are now in a mode where we are no longer running it as researchers there's an NGO that manages it and so it's more scalable if
00:38:10
you wish so we recruited volunteers and schools by sending emails we said we are a group of researchers we want to do this and we got an amazing turnout and in the email
00:38:23
we were asking the schools to indicate maximum three students per class who could benefit from this program and also to tell us which subjects they needed
00:38:33
help with and once the parents consented and kids and parents filled in this Baseline questionnaire we entered them into a lottery to receive the program
00:38:46
why a lottery actually you know we could have pretty much served everybody if we had the money back in Spring 2020 because we got almost 2 000 volunteer tutors in the space of a couple of weeks
00:39:00
the problem was that we were running it out of our offices with student research assistant and the maximum that we could pay for in terms of training and pedagogical support was for 530 tutors
00:39:13
so we basically had to leave half of the families without a tutor they had been warned in advance that this could be an outcome but the fact that we ran these lotteries allowed us to evaluate the
00:39:26
impact and these were the results measured about two months after the intervention so you must think of this as a three hour per week program that runs for about six to seven weeks and
00:39:40
these are the results the left bar shows you the impact on performance in units of standard deviation so 0.27 of a standard deviation is actually a big effect okay so the kids who did not get
00:39:55
the tutor were not going to school remember this was locked down so possibly this is actually you know reflecting that we also saw improvements in the kids interest in continuing with
00:40:08
education and Ambitions in education so the index of aspirations some although not quite you know significant effect on great locus of control and
00:40:20
a significantly positive effect on well-being happiness and less depression among the kids who got the tutor so if we look exactly at who benefited the
00:40:32
most we see that it's basically kids whose parents do not have a college degree so these are the estimates that I circled on the left and
00:40:43
uh kids whose parents have a blue collar job and whose parents do not work from home so these were precisely the families that we thought may not be there to help the students through
00:40:55
school closures and when we look at well-being and depression or happiness the converse the by far the ones that drive this impact were the Immigrant
00:41:07
students why because maybe they had less of a network to start with that could keep functioning through social media or something else so this was the first edition of top as I told you there were
00:41:20
several here I'm comparing top 2020 with 2022 2022 is the first edition that we implemented in a more systematic way with the NGO and everything and the
00:41:33
results are pretty clear okay what we found in terms of academic performance is there even in a time where there's no lockdown School dysfunction if school is functioning normally it's also
00:41:46
comparable in terms of magnitude everything else the aspirations the well-being and so on once the kids are back in their normal social life the
00:41:57
tutor is no longer as necessary so I'm gonna skip the additional results because I got the threatening stop sign but I want to show you what top looks like today so there's a funding
00:42:11
Foundation leap the small brother of jpa at bocconi University conducts the the impact evaluation for every round chai is the NGO that
00:42:25
implements it and then the University of bicoka is the team of pedagogists who help us train and support the tutors and now we have distribution of digital devices and an online platform where the
00:42:38
where we both track the activity of the tutors and we provide materials for them to use in the tutoring so there has been interest for this program we have implemented it for example in the
00:42:50
Dominican Republic and I really think this remote characteristic makes it uh you know amenable to reaching realities that might be problematic to stuff with
00:43:04
highly qualified teachers but let me stop here and thank you for your attention [Applause]
00:43:17
thank you Indiana thank you so we have 10 minutes for discussion correct remember this okay uh so so actually I would start by uh saying that I liked
00:43:29
your last slide Schuler uh with you know this daunting picture of what's Happening currently with the recent election in Turkey uh I I liked it because it tells us that changes are still there her head while listening to
00:43:43
your three stories these are three success stories in a way and I liked your style because you commented it with the optimism you still said okay I'm optimistic and we can be optimistic because we are doing these studies and they show that we can have large impacts
00:43:55
so what I suggest for the discussion is keeping the changes in mind try to unravel a little bit what makes such successful um policy evaluations possible in
00:44:06
context that are difficult context because how to change behaviors and and beliefs on these topics on which people have very strong beliefs are politically loaded how also to face the emergency
00:44:18
because it is a good example and so I I suggest we take a little bit of time to think about what made these successful evaluations possible and what lessons we can we can derive from that to be
00:44:29
realistic Optimist people optimistic people and perhaps the first question is um as these are politically loaded questions people have strong prayers usually on these issues what makes
00:44:41
integration how immigrant students should integrate in schools Etc and the first question to the two of you is how when working with the implementing Partners you could
00:44:54
Foster or whether there was already a kind of learning mindset because we know as evaluators that when people are with you and thinking okay I want to learn something then you already have way in implementing a project so welcome the
00:45:08
Willing from where did the willingness to evaluate and to work with you come in you know these studies I mean when we first started there was a
00:45:19
lot of will to come up like to see programs and and several things were being distributed and done but it was just not very systematic
00:45:32
so evaluation was in nobody's mind in fact when we started doing the perspective taking study as soon as we before even we got any results it's like okay can we do it to the other classrooms can we do it do it to other
00:45:45
schools that was that that was the uh the will so but we held them to to to to to do the evaluation but later on things changed that's what I was I had my last life because of that it's like in in we
00:45:59
are now in the field we see completely different um um different scene actually we have like we in 2018 we did not have residential segregation we did not have schools
00:46:12
exclusively um Refugee we and we did not have completely overwhelmed rundown and I don't want to think about it anymore kind of teachers so I think at this
00:46:24
point uh people still open the programs uh trying different things and finding out what's how what is the best way of handling this problem um but we have to be a lot more creative
00:46:36
to to make the the people who are actually dealing with this in the field happier so anything that is coming from outside or at the top do this do that that was the at the
00:46:50
beginning that was a problem like UNICEF would come and it's like this is the way you do this is the how this is how you integrate and then that didn't work for the Turkish context and that that was a lot of problem but later on like what we
00:47:04
know now is like we need very creative maybe targeted in different regions different programs for Refugee integration and in dealing with the kids is easy I mean they are so happy to pick
00:47:18
up anything you give but dealing with now adults is becoming harder and harder in these regions so the two programs were actually quite different in terms of how I would answer your
00:47:36
question the first one was handed to us basically by the Ministry of Education saying there's this Foundation they want to fund this I have the schools on board I need evaluation so that's easy we
00:47:49
didn't have to persuade them they were interested in figuring out whether they could do something the sour side of this is that even after seeing the results eventually it was so
00:48:02
expensive that the the ministry couldn't scale it so they learned that it worked but they should have done a version for example without the academic tutoring which was a component that we found had
00:48:15
very little to do with the effects and that would have made it much cheaper and at that point maybe the enthusiasm had gone down so I see that as maybe you
00:48:27
know not such a great message after all because you find the impact you say okay maybe we can try a little bit of a an alternate you know to cut down a bit on cost so that it becomes feasible and at
00:48:42
that point nobody follows up the second one started with a lot less ambition basically us emailing the principles directly it was very clear that the
00:48:54
principle was the key some principles were super responsive others were so responsive but also helpless because they said look things are
00:49:07
falling apart to such extent that if I have to ask the kids to log in for three more hours they're just gonna stop following and others didn't get back to us so but once the principles were on
00:49:21
board um there was no need to persuade anybody about the evaluation they saw the potential usefulness and and even now you know we report the findings when we
00:49:33
approach them but they have a pretty clear mind that if they can get tutors for certain students not much harm can come one qualification I would maybe give is
00:49:46
that these evaluations in Italy are with the schools are very disadvantaged especially in the last two rounds but this University students are committed
00:49:58
well prepared it's been a lot harder in the Dominican Republic to find tutors who you know can adjust to the task you need to kind of improvise help with this
00:50:10
homework do that and so the supply side of uh volunteers who have their skills might not be so easy to create as in the Italian example so I would say we should
00:50:25
qualify that success story but that I see more of a success story because I see you know following up the funder came afterwards because of what we did and now we you know we are experimenting
00:50:37
we tried group tutoring then we figure out it's better to go back to individuals so there's quite a process there but yeah thank you thank you maybe stage two in in this
00:50:52
project so you've already anticipated on stage three but I'm still a little bit curious about stage two which is uh you're changing the behaviors of some players so teachers in particular
00:51:05
um when we do economics application we know it's not always easy to change behaviors of teachers and again I would insist you know you're doing that in emergency situations that may help but that may
00:51:18
also not help because they have many things to do and and perhaps uh it's hard for them to to take as a priority a new intervention or to learn about
00:51:29
potential intervention Etc so I guess you know and having in mind all the difficulties and the the political debates on on integration anything any hints from this study on
00:51:44
you know what are uh productive way of changing behaviors of changing beliefs and and how we as researchers can help I guess [Music] I just want to say something here is it
00:51:57
that uh I think the one of the things I I'm learning is is how you frame the program so when you when you frame the program like what is this good for how is this going to help the host kids and
00:52:10
the refugee kids and the teacher and the the country in general it was super important so you don't go and say okay we are here to help the refugee children because this this you're already in a
00:52:24
very very disadvantaged setting in Turkey all the children need all the help they can get right so and and that really backfired for a lot of people right so it's like what about the other
00:52:36
kids they are not looking good either right so so the idea of a program that the cohesion building let's do this together like let's
00:52:47
hold hands right so without actually making any ethnicity Salient so all the areas that I was working there were Kurds there were Turks there were Arabs they were on it on top of it they were
00:53:00
Syrian refugees so if you don't make this Salient kids never make that Salient anyways so it's it was it to me it was very clear at the the beginning that that sort of program would be
00:53:12
successful and that's why I I advise the program to be like that not targeting a particular group but targeting the community the school community in that in that I think that is important
00:53:26
how we frame how we designed the programs and to listen the listen the field the the implementers actual implementers in my case was teachers
00:53:40
so um our experience with teachers especially in the first program I showed you that graph that showed that you know they are much more likely to tell a native Italian go to a good school
00:53:54
that triggered something in us we we felt oh these teachers are so biased okay and Italy is a country of relatively recent immigration so there's less familiarity and not necessarily
00:54:07
these teachers were biased because they were bad you know they might have thought oh the family won't be able to follow through with the support that you know a certain type of high school would
00:54:19
need and so on but we weren't sure about whether they knew uh about their own attitudes or not so in a subsequent project we actually tried to measure
00:54:30
those using measures from psychology of implicit bias and we decided to tell some of the teachers what their score was on these measures and it turns out that after learning their scores that
00:54:44
were pretty bad across the board those teachers that we gave the information to started changing their grading and becoming you know less biased towards immigrant and more fair in their
00:54:58
assessment of immigrant and Native students and Michaela is now trying to launch a new program where she instead of taking these psychological measures
00:55:10
she just uses administrative data computes the share of kids that a given teacher has recommended for one track another track based on their ability and just gives them information says you
00:55:24
know do you know that you sent this fraction of immigrant student to that school and then matching it with the success in terms of retention rates and so on later on tells them here is how
00:55:37
they did in these schools so I do think that one way of dealing with the bias that's built in the system maybe from you know lack of experience if not for
00:55:50
from explicit intentions is to give people information uh about how they behaved in the past or you know what their what their cognitive mechanisms seem to suggest about
00:56:04
implicit bias it might not solve every problem but it seems something that's not particularly costly to do and we haven't faced resistance when when we did that actually people are curious
00:56:18
they want to know yeah one minute so but there's still a one question I was thinking about you know um what we want is to have impacted as
00:56:29
larger scale as we can and for that actually I could see two ways from your studies one which is the usual way we think about it well let's scale up these studies and let's develop these programs at a larger scale but also thinking
00:56:41
about all the difficulties we have in the public debate to go beyond some very strong prejudices against migrants and the possibility of integration one way your study could be very useful it could be simply to be in the public debate
00:56:54
that people realize that simple things make big difference so is it something that that came out you know through the media coverage or to so direct reaction from the public
00:57:06
opinion to your studies are not necessarily through policy makers so I don't know if that's something that happened or if just wishful idea that came to my mind at the last minute so I can start with this it's another
00:57:20
one of the maybe a small disappointments but small so there was a little bit of coverage especially the the more recent one the after the pandemic when we saw
00:57:32
those results were like wow we need to have it out but not so much because oh we did a great study because at that time it seemed such a hopeful message in terms of mobilizing Volunteers in the
00:57:46
space of less than four weeks running it and so to me I was proud of being Italian I thought in this country people still have you know the willingness to help when it matters and then I gave it
00:57:58
to the Press office of my universities they tried to send it to newspapers there was a little bit of coverage but nothing major and then a few weeks later I think one of the tutors I don't know
00:58:13
if they posted something or basically they ran a huge uh a story but more on the tutor experience no mention of the evaluation of the results it was kind of
00:58:27
a you know less scientific and I was still happy that they were talking about the program but part of me thought you know the incentives of the media are not really driven by what you can tell them
00:58:41
about the success of a program they try to see what's the best story if I talk about some bocconi Professor people won't read if I talk about this student
00:58:53
with these interesting story and maybe it's right that way but so I think one of the just to tie up with the maybe the goal of these two days one of the great
00:59:05
services that jpa is doing in my view to the profession is that they're also trying to make this research and these stories a little closer to what the General Public
00:59:18
can understand and likes to hear and sometimes as researchers we fail in that Dimension or at least I still have a lot to learn in that respect
00:59:29
uh yeah also the situations that we work in and J-pop totally recognizes that this is this is also wonderful that continuously changing if especially if you're working in a humanitarian setting uh one program that that worked in that
00:59:44
context all of a sudden the context changes right so then you communicate this this evidence and then you say okay scale up is not costly less skilled up but the context already changed for example the segregation started or the
00:59:57
public opinion completely turned because there's now economic hardship you have to to think about something else and then launch something else very fast which is what we like to do in the in the new j-pal initiative right so if
01:00:11
you're working in settings like this situation is very Dynamic problems change very quickly and you have to be quick in program design and evaluation
01:00:23
thank you so much so we've exhausted our time so let me stop here to thank you and to thank uh all uh who made it possible this initiative including Community Jamila and particles and and
01:00:36
those who followed and and now all the premises with the uh this place Library initiative that will you know the the twin initiative complementing uh this one so thanks a lot to everyone for your
01:00:49
patience and time for coffee break and we are back at ten to four correct ten two three sorry oh at four okay so we're meeting at four [Music]
End of transcript