Video: ianno21 Panel: Social Annotation Research (DocDrop)

00:00:08

i am frannie from hypothesis i'm on the marketing team here and i am so happy to be here and love this conference and love open annotation and just the

00:00:19

amazing people who are taking part in it and i'm going to turn it over in a minute to one of our amazing people our moderator bodong um but i want to introduce our guests today

00:00:32

as well we have jin ranju and yanji jung and chris andrews and like i said i'm super excited about this

00:00:44

and um we are recording it so if anybody ends up coming in late they'll they can watch the beginning after the fact so with that i'm gonna be quiet and turn it over to bodom

00:00:58

thank you so much for the introduction franny hi everybody i hope you are doing well and i know this is a international event attracting colleagues and friends from

00:01:11

around the globe so i'm really happy to be here and this is my second um i annotate conference um my first attendance was in

00:01:22

three years ago actually in 2018 uh in san francisco so really happy to be back to this community and say to see the growth of of work related to hypothesis

00:01:35

and annotation um my name is bo dong chem i am an associate professor from the university of minnesota and i see myself as a learning scientist who

00:01:49

does who does research on learning processes and also design technologies and pedagogical practices for for learning so i'm happy to really see three interesting projects represented

00:02:03

in this panel and i will let our panelists introduce themselves of course and their project as well a bit later um thank you for putting up the slides um

00:02:14

so our panelists as friendly mentioned um yunji jin from nyu xin and drew from minnesota university of minnesota and chris andrews from university indiana and i see there's um

00:02:28

and by the way of them are phd students actively working on cutting-edge research in the learning sciences or and also in the uh field of learning analytics and i'm actually very excited to listen

00:02:41

to their talks and and hear from them about what they're doing and what are they designing and what are the research in um and so on so and how um next slide please here

00:02:57

and this is a really a tentative agenda for this panel and the structure of this panel will be for each panelist to spend 50 minutes to present their work and then have a really quick

00:03:12

5-minute q a after their talk and and and after every panel let's present their work we will have a panel discussion we will invite everybody to

00:03:22

post uh questions along the way so that you know those questions will be either addressed after their talk or add it to the final discussion um so i hope that's um

00:03:36

i believe we'll have plenty of time to really listen to each other and also to have a conversation during the panel so without further ado um our first presentation will be

00:03:48

um from here take this from here hello everyone um thank you vodon for the great introduction so my name is sin andrew

00:04:01

a phd student from university of minnesota and it's really a great honor to be here to talk about collaborative annotations um in general it's really a fun and great conference i heard

00:04:14

lots of inspiring voices from people from all different areas and so for my interest what do i do i see myself as a learning science

00:04:27

researcher and learning experience designer i'm interested in learning analytics and design based research to develop educational innovations and implement learning theories into the real world

00:04:39

so those also have been my guide to the study i like to share in a minute so about a study as we all know hypothesis can support note-taking in many different areas so today i'd like to share how it

00:04:53

has been used in educational settings so this study is based on our collaboration with college-level instructors who are highlighting the hypothesis tool

00:05:04

so we started this design-based research to support their integration and study how collaboration how collaborative annotations work in their classrooms so here's the study designing support

00:05:16

for productive social interaction and knowledge co-construction in collaborative annotation so i also would like to introduce our team here hong shui who is also a phd student

00:05:30

at university of minnesota she might be here today as well and budo he's the moderator here as you already know and they all contributed a lot of wonderful ideas and efforts in this

00:05:42

project and yeah and shanna and boron just mentioned shanna in the chat has been supporting our study through at the beginning

00:05:56

so let's get started of the study as you may see a key element of the study is the design so what is the design as a preview we designed two scaffolding strategies to support teachers teaching and

00:06:10

students learning the first strategy is called the dynamic grouping strategy so this first strategy the purpose is to create or design a learning community as

00:06:22

the first step for example when a class has really big size like 100 students the instructors may want to divide students into smaller groups like 10 or 15 students per group

00:06:35

this could avoid a student be overwhelmed by a hundred annotations per rating and another example could be the instructors may want to assign

00:06:47

different ratings to different groups of students so they can later share their understanding of different readings and achieve a higher understanding of the topics and in general this strategy will help

00:07:00

the instructors to create a learning community at the first place and that can fit their teaching goal the context and the community itself and the second strategy is called the

00:07:13

participation role strategy with which is also today's focus so for this strategy we want to provide an opportunity for students to take more responsibility of their own learning

00:07:25

and support the process of their learning and to improve social interaction and knowledge co-construction i'll dive into more details later about this strategy

00:07:39

so why did we design this study it was initially conducted at the university of minnesota in fall 2020 when the campus shut down due to kobe 19 and then many instructors piloted to

00:07:53

online instruction and were looking for solutions so based on our observation there were generally two directions the first um given the limited time some instructor may want to

00:08:04

by using some technologies or tools to replicate their face-to-face instruction um and then another direction would be they want to transform the student-teacher relationship

00:08:17

by taking this opportunity so this is where our collaboration started because we share the same understanding that the effective usage of technology requires the consideration

00:08:29

of both technology and pedagogy so we think there is a need to design meaningful scaffoldings when using technologies sometimes even redesign of the curriculum instead of just throwing the

00:08:42

tool directly to students and by meaning pedagogy here the pedagogy is drawn from both theories and practice such as the computer supported

00:08:55

collaborative learning theory and some other learning series and based on that we want to um through the design let students take more responsibilities in learning and we want to transform the dynamics between

00:09:08

students and teachers embrace nanotechnology and then we want to facilitate a natural space for social interaction and also engage knowledgeable construction in online learning

00:09:20

so about the study design we want to support collaborative web annotation in college classrooms by designing sophisticated participation roles methods we use is a co-design between

00:09:33

researchers and instructors to design scaffolding roles and support their implementation with course-specific class customization and we use hypothesis as the

00:09:46

social annotation tool and this hypothesis in the current study has been integrated into the lms system of umn canvas so the participants were from three

00:09:59

fully online undergraduate classes in liberal arts and the focus of the current study is dance history class i had 13 students and one teaching assistant

00:10:11

and one instructor so what is the design what is the participation roles strategy in general is a generic scaffolding framework

00:10:22

comprising three scripted participation roles based on the computer-supported collaborative learning literature which includes a facilitator a synthesizer and a summarizer the facilitator

00:10:36

is responsible for stimulating conversations by finding connections seeking clarifications and encouraging their peers to consistently tag their annotations for an entire week

00:10:50

and for synthesizer they usually synthesize the initial ideas highlights agreement or disagreement and suggest directions for further discussion in the middle of the week for example in the dance history class

00:11:03

in the middle of the week there is a synchronous class discussion via zoom so the synthesizer are required to submit a paragraph or two or some nice bully points as to

00:11:17

synthesis before the class discussion and during the class discussion students can have a further discussion based on the sentences and other peers annotations and then for

00:11:29

summarizer they usually summarize group conversations at the end of the week based on both the class discussion and all the annotations for the whole class and also at the end

00:11:40

of week for this class every students were required to do individual reflective writing and all those roles and time frame can be adjusted by the instructor accordingly based on their

00:11:53

own teaching goals and context in order to study if the design worked or not work we we had two research questions

00:12:04

the first one is how did the activity design facilitate social annotation the second is how did the design facilitate knowledge co-construction so first we conducted a social network

00:12:17

analysis to study the social interaction such as the participation patterns in the collaborative annotation activity then we conducted a content analysis to study the knowledge co-construction

00:12:29

levels so for the coding scheme we use is a revised interaction analysis model of collaborative annotation we developed this coding scheme based on

00:12:41

guna vaderna's iem model and arubia and angles model of collaborative knowledge construction and we identified four levels of knowledge construction the first one is

00:12:54

called initiation where students started to share initial understandings and ask questions and share resources without too much elaboration and then it's called level two is

00:13:06

exploration when uh in this phase students started to elaborate on the text or contact personal experiences with some critical reasoning and then level three is negotiation

00:13:18

where students started to ask a question through critical reasoning or negotiate disagreement or connect readings with critical reasoning and the highest level level four

00:13:30

is called co-construction in this level students started to reach a consensus on the previous questions and they apply the knowledge or way of thinking and also they can make metacognitive

00:13:44

statement illustrating their learning outcome so did the design work in general the answer is yes for social interaction as you can see from the table the integrate our degree

00:13:56

means how many uh replies or annotations they received and were sent out so the facilitators in general sent out more replies and they reached out to more peers

00:14:08

and they also received more replies and from the table the betweenness constrained and dominance they're all centrality measures so the higher scores in betweenness and dominance

00:14:20

and lower scores in constrained meaning those students are in the center of this community so the facilitators they were always in they were always influential in the collaborative annotation

00:14:34

activities which means they are always in the center position and also the social interaction pattern varied across the facilitators in different weeks and for synthesizer they participated more than more than

00:14:48

non-role takers in terms of the numbers of posts they sent out but not as much as facilitators did since they tended to focus more on synthesizing the readings and

00:15:00

annotations on their own and for summarizers they participated as same as non-role takers which is also expected since the responsibility for them was to write the weekly summary on their own

00:15:14

and for knowledge construction the facilitator they generally ask questions or provided answers with elaboration examples critical reasoning to start and push the discussion

00:15:27

for example here's an example thread facilitated by one facilitator as you can see the facilitator tried to connect other two students annotations

00:15:39

when replying to student a and also the facilitator proposed some follow-up questions to engage thinking and engage with deeper thinking and um for knowledge construction level

00:15:55

also varied across the facilitators in different weeks and the synthesizer their posts were also mostly classified into levels two and level three in terms of the knowledge code

00:16:07

construction well for summarizer they on average contributed much less annotations in all levels most of their posts were in level two this result over in line with the scripts role in

00:16:21

the scaffolding framework for example because they focus on the class discussion during zoom meetings and composed summary that connected zoom discussion with annotations

00:16:33

so in general the result indicated that to a great extent the designed activity was enacted by students properly and then the role assignment was associated with

00:16:45

student social interaction pattern to some extent and different role takers may have different strategies when they're playing the roles and most importantly in weeks when role takers posted more

00:16:58

higher level posts the knowledge construction level from non-role takers tended to be high too so um implication of the design first we proposed a scaffolding

00:17:12

framework for collaborative annotation which is applicable to many college-level classes including the both the dynamic grouping strategy and the participation role strategy

00:17:24

and then we developed a revised interaction analysis model for collaborative annotation that is more appropriate for analysis of students discussion anchored in web documents

00:17:36

this can also support teaching as a reference for evaluation of the annotations finally the result of data analysis has shown promise of the designed scaffolding framework for

00:17:48

facilitating productive collaborative annotation in a study context in particular the first the facilitators and the synthesizers played roles in deepening collaborative

00:18:00

annotation so as for some final words um just to illiterate our purpose of the design we believe effective usage of technology

00:18:12

includes the consideration of both technology and pedagogy so for pedagogy students are not always natural collaborators and they need we need to make intentional efforts to help them become

00:18:26

better collaborators and for instructors they need to provide careful scaffolding and detailed guidelines for students to take various roles and the technology also

00:18:37

needs to connect students and teachers needs to provide a natural and effective environment for collaboration and there's a plus sign here i just want to say that this placement

00:18:50

doesn't mean the relationship between technology and pedagogy is linear actually they impact each other at every moment so the pedagogy can impact the technology design and development

00:19:02

and in return the technology can impact the pedagogy in terms of such as curriculum design and the class evaluation so um so we need to go back to the fundamental

00:19:16

questions to rethink the relationship between pedagogy and technology for example we call hypothesis as a note-taking tool when we discuss how discuss how to make

00:19:29

better notes or how to support note-taking in a better way we also want to go back to the questions that why students are taking those and how are they going to use it in the future

00:19:41

so then we can think how we as researchers designers and teachers can do a better job to support the process and as the fighter knows i also would like to

00:19:54

invite everyone to rethink the relationship between technology and pedagogy and also between students and teachers what can be done as researchers designers and teachers

00:20:07

so that's all for my presentation thank you very much for listening please let me know if you have any questions or suggestions thank you

00:20:22

uh thank you so much for the presentation shiran i already see uh or two questions in the q a section um so the question is from michael welker

00:20:34

and michael was asking on the assigned roles were they rotated across multiple assignments also how much orientation was needed to define those roles

00:20:46

for the students yeah that's really a good question so first yes the role will rotate each student will have opportunity to try each different roles and each week there's

00:20:59

one facilitator for one for each reading one synthesizer for and one summarizer for each reading and for orientation at the beginning of the semester uh we actually the instructors spend a lot of

00:21:13

time to explain how the different roles are different from each other and we kind of spend two or three weeks to try and to adjust the roles to

00:21:25

kind of make a co-design not even between researchers and instructors and also between the students because during the class meetings the students also shared their understanding of the roles um and

00:21:37

then the students and the teachers adjust the role together i i think started from week four and uh all the roads have been settled down and the activity have been

00:21:48

uh smoothie and um yeah that did not answer your questions all right michael says awesome okay great thank you any other questions quick

00:22:02

questions for you i see another question coming from shanna would you recommend using the framework level one to five for rubric for grading and or

00:22:14

for scaffolding for students i think uh by framework machina means the coding framework you used to analyze the code construction yeah i'll go back to that slice so

00:22:29

um this this coding thing has four levels um i think it can be uh work as a reference for um grading or evaluation of students annotations but sometimes i'm hesitant

00:22:43

about using some framework or coding scheme to grade students annotations quality because it's really hard to to see if

00:22:55

to identify if an annotation is is a high quality or low high quality um even when some students are in level one or level two it doesn't mean they're not learning uh maybe they're

00:23:07

in some phases of learning or just their learning styles but in general i think it could be used as a helpful reference for teachers to see how students are learning you know

00:23:20

in a specific week and how other scaffolding or support need to need to be provided to the students great thank thanks for the questions and thanks for your response this year and

00:23:36

i think we will have more opportunities later after the presentations to engage in those questions again and at this point i want to thank you here and for your presentation and then invite chris

00:23:48

to share your screen and and give us your presentation all right let's see if i can get this to work here okay so i have a somewhat weak connection uh for those that are uh

00:24:05

listening in i'm gonna try and uh turn on my video for a minute i had to i had to drive down the road to find a better uh spot for my um for my phone to be able to hot spot in so we'll see

00:24:17

uh that's why i'm in a car right now it's also raining um but uh i'm happy to be here i'm excited to uh present with uh these others

00:24:29

um shinran that was that was awesome i loved the um what you've been doing i've already seen your your scaffolding framework from uh that you had posted up before so i think that's awesome

00:24:40

so what i want to present on today um is this this is kind of part of what i'm trying to do for my dissertation which is thinking about kind of instructors and how they use it um how they use social annotation uh

00:24:54

particularly within these undergraduate reading and composition courses which i'll talk about in just a second um uh yeah go ahead and go to the next slide um so some of these this is these are some things that we kind of already know

00:25:08

about uh social annotation um and shinran talked about this uh that kind of with the pandemic many people moved online needed something to um

00:25:22

you know kind of mimic uh what they were already doing in their classes or just something to help them with their their online courses that move to online and many turn to social annotation um

00:25:36

because it's this flexible tool that we can use even in a hybrid situation or even a face to face um and i love this quote from raymie kalier and antara garcia their

00:25:48

new book on annotation that they mentioned that social annotation is seldom and end in and of itself rather it most frequently complements a repertoire of other educational practices

00:26:00

so we need to focus on you know not just social annotation and what's happening in social annotation but also kind of how it interacts with other things particularly within a course um and so that's part of where i'm kind of thinking about this in terms of

00:26:13

instructors um and uh and then also what happens to these annotations once they're once they're created um so uh this also comes

00:26:23

again here i'm i'm quoting uh bodong and and shinran here uh on their most recent article um in the information learning sciences journal that we need more in-depth qualitative

00:26:36

inquiries into how instructors are using social annotation and these this interaction of social annotation activities with other course activities go ahead and go to the next slide so these are some of the questions that i'm

00:26:48

thinking about um as as i'm you know trying to understand analyze research um you know how do instructors implement social annotation in their online undergraduate reading and composition courses again we'll

00:27:01

get to the context here in just a second what are actual what are instructors actually doing with the annotations and then how do these social annotations impact or align with other course activities and and course goals

00:27:13

next slide so some of you may have may already be familiar with this um but i'm i'm at indiana university and hypothesis and uh the department of english at indiana university have partnered

00:27:28

to uh to do this this project that's really uh so what i'm doing is kind of a small part of this long-term project at iu particularly with

00:27:39

the english w131 classes that's their freshman composition courses reading writing and inquiry but uh part of the data that i'm looking at has to do with um a couple other courses as well

00:27:52

this introduction to fiction and introduction to poetry courses and i've got the nice little uh covid graphic there that basically what happened is again as we all know forced online um because of the pandemic

00:28:05

and uh so they started to implement they've been using hypothesis in some of their courses and when they started making this this course shell for um you know all of the instructors to use for english w131 the freshman

00:28:19

composition course they decided that they wanted to embed hypothesis as part of this and social annotation activities um as part of this so across all of these uh there's there's more than 50

00:28:33

sections of english w131 they're going to be using or that have used over this this last academic year um social annotations using hypothesis um and so there's a lot of data that we're going to be

00:28:44

sifting through and i'm just kind of piecing off a little bit of this no that was actually perfect you like read my mind um so spring 2021 semester this is uh where the data that i'm looking at in

00:28:56

particular there's uh several um data sources so uh the the semester started and you know we're collecting kind of course activities that are happening

00:29:08

including their social annotation uh assignments that are occurring throughout the semester as well as other assignments that they're working on we had two instructor questionnaires one at the beginning of the semester or near

00:29:21

the beginning of the semester and one at the end to kind of get an idea of what instructors are doing with the annotations and then i also facilitated what we called an instructor inquiry team

00:29:33

it was really kind of like a mix between a focus group and like a professional learning group professional learning community where basically we just talked once a month ish about uh the decisions that they were

00:29:47

making about their social annotation use in the classroom what kinds of uh intentional decisions were they making why were they making those choices um in terms of how they were using social annotation or

00:30:01

um how they were uh you know what they were doing with the annotations and that sort of thing um and so you kind of see that we've got kind of diff multiple time points all across the semester of kind of

00:30:13

trying to understand what's going on you can go to the next slide um so i kind of mentioned this already i guess that that we have the two instructor questionnaires um uh the the instructor focus groups there

00:30:26

was actually there were six instructors that participated that we split into two uh two groups where we were doing these deeper dives and then uh those are part of the course artifacts we were collecting these artifacts that

00:30:37

um that the instructors created but some of these artifacts were artifacts that students contributed to even though it was a instructor created and i'll mention that in in just a minute so we go to the next

00:30:51

slide so uh this is i don't know how many of you are you know college uh football or basketball fans well right as soon as the season ends um they come out with rankings for

00:31:08

the next year uh even though nobody knows anything that's gonna happen and they typically typically call those their uh way too early rankings so um these are my because i have not

00:31:22

dived into the analysis yet um this is uh just kind of my thoughts as i've uh as i participated in some of this data collection so i haven't really done analysis yet so this is my way too early

00:31:36

thoughts and analysis uh so we can uh keep going i think i think i have these down one by one so i'll just tell you to go to the next piece so you can go to the next one actually perfect so uh instructor's design of

00:31:49

annotation activities as and and what i mean by this is that instructors how they were thinking about using um annotations uh and i i have an ore between these but i don't

00:32:00

necessarily mean them to be um either or uh it might be somewhere along probably a continuum of some kind um but some of the instructors were thinking about

00:32:12

uh the annotation space as a student-only space not really a space that they really participate in they might provide some private feedback for example within canvas the the lms

00:32:26

because the hypothesis we were using the hypothesis within the learning management system and so some of the instructors were thinking about is a student-only space a space where the students didn't have to worry about the

00:32:38

instructor uh uh making sorts of uh you know judgments or comments on uh in the public space um but there are other peers so really kind of more of a space for

00:32:51

peer-to-peer engagement um or some of the instructors were thinking about it as this kind of joint space where they could together um make meaning and so again there's some

00:33:04

and i love how uh shinron put this idea of you know technology plus pedagogy and these are some intentional decisions that these instructors are making these pedagogical decisions as they're thinking about how

00:33:17

they want to build this space and why they want to build that the space that way um and so that was just one of the interesting things that came out of it and then there's this idea of individual engagement with the text or threaded conversations what is what is the goal

00:33:29

of these social annotation activities is it in in these reading and composition courses for example is our focus really on uh analysis of the text um or and

00:33:43

really kind of having that conversation with the text or are we trying to uh promote this this threaded conversation um you know focus more on the interpersonal

00:33:57

or the um community building ideas uh within that and again not that these are mutually exclusive you can do both but just as the teachers are thinking about uh how they want to approach the the

00:34:11

pedagogical uh process and pedagogical decisions they're thinking about some of these things and sometimes they're focused more on i need them to understand the text and sometimes they're focused more on i want them to build community

00:34:24

or uh share with each other understandings um and go ahead and go to the next one and then there's this idea of accountability and exposure to the text this is sort of similar to individual

00:34:36

engagement with the text but um some of the instructors are thinking about um this idea of accountability uh of you know is this just social annotation is maybe an easy way to find

00:34:49

out if students read the text or not and maybe that's all you need it to do obviously i think many of us here in this session and many of us that have used social annotation see

00:35:00

other rich and and wonderful uses for it but also some instructors all they really need is they want to make sure that the students have have read the text and had been exposed to the text and so some of them aren't thinking

00:35:13

about um this deeper power of social annotation um or another uh you know part of this and i see uh jeremy in the chat has mentioned kind of these tensions

00:35:24

um that these could be portrayed that way this idea of tensions that or are they can we use these social annotations as resources for future activities

00:35:36

can we um and this was in particular i'm thinking one of the instructors who was having students use tags pretty heavily as their reading particularly relating it to

00:35:49

um conventions of a particular genre of of writing and then they would go into a google doc a shared google doc where

00:36:02

students could go back to the annotations and they could search for these tags um and then they could kind of add some information okay this or you know about a character or about jean

00:36:14

a particular convention of a genre and and start to build out this um i don't want to say a catalog i'm trying to think i need a better word for this but um they were kind of building out this

00:36:29

this new resource so they were using social annotations as a resource to build a new resource for when they went to their essays where they could kind of organize they kind of self-organized all of these

00:36:41

um all of their annotations in in kind of new ways um so again just kind of some of these interesting tensions that some of these instructors were thinking about how do we use social annotation

00:36:53

um and and how do we use them in different ways than maybe we haven't thought about before uh and then going to instructors use of annotations um you can go to go to the next one i think most of us are totally

00:37:06

we get this idea of preparing for class um you know many of them were using the social annotations they'd go read their students annotations to uh prepare for uh their their next class they would you

00:37:19

know the they'd be due 24 hours beforehand or something like that they would read through them um maybe to see misconceptions or ideas from the text that students missed a lot of times they could tell by what

00:37:31

was what was annotated and what wasn't annotated that you know if they didn't annotate a whole section of the text that uh the instructor felt like was really important then they would you know bring that up in their in their

00:37:43

conversations or some of them one of the students um you know mentioned they used it as a warm calling on students in the sense that they knew a student had already made an annotation had already said

00:37:55

something about this and they would reference their annotation and maybe either ask them to expound on that or um or maybe there wasn't uh there weren't

00:38:07

replies to that particular annotation in the social uh space and so they would in their synchronous session they would uh you know warm call on that student and use that annotation as kind

00:38:19

of a jump start for uh further discussion um obviously there's part of this uh there's another tension here where we don't want to uh

00:38:32

rehash a conversation that has already been had in the social annotation space um you know we want to be careful about not having a redundancy of a conversation and so kind of trying to think about

00:38:43

what what that might mean um but uh that we can still use the annotations in this you know in this other synchronous space as well um and then this idea of public versus private feedback i kind of

00:38:55

mentioned that already in terms of this student-only space versus the joint student instructor space but thinking about okay when do i make

00:39:07

public comments and when do i make private comments to students and and just thinking about that uh that dynamic and and trying to identify when

00:39:21

at what point uh or you know what is happening in the annotations that i'm going to make a public comment versus a private comment where i'm going to go into the the annotation space where all the other students can see it or i'm just going to do it within the

00:39:33

the canvas lms and um you know give them some some private feedback on the assignment for example so again i reiterate that these are way too early thoughts and analysis

00:39:46

that uh i'm looking forward to diving into it more this summer um and uh being able to have some some more specific uh and uh much more backed up by data

00:39:58

and quotes and and other things that i can link to to to uh promote the trustworthiness of what i'm saying so this is this is just from the top of chris's head based on his participation in uh collecting the data um

00:40:12

but i think some of these things are are interesting and and hopefully will bear some some really uh some more really interesting data um and i think that's i think that was my last slide there so

00:40:23

i think i'm i think i'm done all right this is fascinating chris i already saw some really interesting conversation going on in the chat all right i see uh something popping up

00:40:40

um so yumji asked the question do you want to just directly ask to chris yeah sure chris thanks for a wonderful presentation and i really enjoyed your presentation but i

00:40:53

have a question or your personal thought about the potential impact of different areas of content like your classes were in english but

00:41:04

what if like we are designing such as like stem courses such as physics or statistics like their reading materials are really different from english or like social sciences do you

00:41:17

have any thoughts on that yeah let me think um i i know i i have a um i have a brother-in-law who teaches

00:41:29

physics and he's used another social annotation platform that i won't mention because it's not hypothesis and this is a hypothesis event but no i'm i'm just kidding i know hypothesis is all for uh any kind of social annotation but

00:41:42

um he uh he he talked about and and we know from some other some other places like um uh oh well now i'm forgetting there's the the the website where you can see

00:41:55

experts annotations on um uh some science content science in the classroom is that what it's called i think um so i i think there's some really uh there's there's some really great uh

00:42:10

ways that we can uh i i see that annotation can be different in terms of you know there's there's the disciplinary side of it right that um so for example i was talking about the tags you know genre conventions within

00:42:24

english you know as a specific thing but thinking about how what kinds of tags you might use in a you know a science or a stem situation to help students organize some of the

00:42:36

knowledge that they're you know co-creating or recognizing um the other thing i was thinking about that you know english is is primarily text based right but when we get into

00:42:49

stem um and maybe that's an over generalization of english or there's uh maybe uh jeremy who i know has an english background might might take some issue with that but um that i'm also thinking about like graphs

00:43:01

and um and uh you know other other kinds of figures that um annotation and on those kinds of um those kinds of representations um i

00:43:16

could see being you know quite a bit different uh than you know thinking about a paragraph um and trying to make sense of a paragraph and and connecting to a paragraph but the this idea of data analysis um you know

00:43:29

of these these figures or things like that those are just some of the things that come to my mind um i feel like maybe i didn't did i did i address your question i feel like we kind of tend to i tend to go off just a little bit there

00:43:42

yeah i feel like you address the potential like considerations that instructors might consider in terms like uh like the annotations and interaction can be mediated but not only by the english

00:43:55

text but also by like ref or like formulas or the others in stem courses yeah addressing yeah and and again i think that one of the key things that i think

00:44:08

is just so important and i think this gets a little bit to that that quote that i shared earlier um of this idea that social annotation you know is seldom an end in and of

00:44:19

itself that's just getting students to socially annotate is uh you know might be fun and interesting for students but unless it's tied to some sort of course goal or

00:44:31

um it's helping us achieve some sort of uh you know other objective um you know you mention this idea of mediation right that we want to think about not only how does social annotation

00:44:43

you know mediate our accomplishment of some of these goals but then how do those social annotations then become mediators for other activities in the course um and so that's part of what i'm thinking about like if

00:44:56

we need to make sure we we start with that or consider those goals so what what goals are the instructors trying to accomplish and then how does you know do these social annotations

00:45:08

um how can they impact that and then now we can start to think about okay how am i going to get students to create the kinds of social annotations they're going to be most useful for them to accomplish this you know it could

00:45:21

just be an essay but it also might be some other really interesting projects that they're that they're working on so those are just some of the things i'm thinking about thank you thank you both for for the

00:45:34

conversation i see another question from shanna but i think at this point uh i will save that question for the later panel discussion because i think that's relevant um to a lot of the project we already

00:45:45

see and now i want to invite umg to share your screen and share your work to us okay uh hi everyone could you see the screen

00:45:57

okay hi everyone i'm yunji and i'm a phd student at nyu and i think it's a nice trans transition from uh singlang and chris's presentations to my one because

00:46:11

uh my one is more about in what ways we could support uh quality and better social annotation activities by using another intervention like

00:46:23

social annotation itself can be on learning in intervention but i want to add one more layer to that so i can introduce that idea so today's presentation is mostly about

00:46:36

suggest this idea and conceptualization of what i mean by data informed action or like what i mean by data informative feedback which can be used to support

00:46:53

better participating in social annotation activities okay so i think everyone here is very interested in social education and

00:47:04

we are very understanding why social annotation engagement is very critical but uh by limiting this into a higher education context especially for the course

00:47:17

activities i wanna say uh quality learning engagement matters a lot for student success in higher education and so especially in nyu there are several

00:47:31

courses that apply social annotation tools to course activities to help students a better success in high in their courses and also better understand the course

00:47:43

concepts then what kind of effective learning would be occur with social annotation we can think about three different dimensions about it so the first thing is by using social annotation students can

00:47:57

interact with peers which can be an additional ways for their learning activities and also get exposed to diverse views and increase their perspective beyond their beyond their

00:48:11

previous ones and another way is by participating social annotations students can increase their additional participation in course activities as well as

00:48:24

additional social annotation activities and also put more effort to complete their learning tasks which we can say by behavioral engagement

00:48:36

and of course by participating in social annotation and reading the materials and reading other students ideas about the same text students can deepen their understanding about the course contents and also

00:48:50

formulate new inquiry or argumentations so students can increase their conceptual and cognitive engagement in the course but uh if there if uh

00:49:04

if here uh there are instructors for the courses who are using social annotation uh you could see there was a lot of low quality annotations and also students share the very

00:49:18

diverse level of engagement commonly reported also in the literature so then my question is how can we design learning support where students can effectively

00:49:30

engage with social annotation which have been expected to help students so one way i want to introduce about it is providing learning providing kind of the

00:49:43

learning analytics dashboard but i don't want to use the term dashboard because dashboard is more like very sophisticated term but the main idea about here is what if we provide uh

00:49:56

providing kind of the feedback or weekly summary report that summarizing students engagement progress in social annotations so because when when students

00:50:10

when students participate in hypothesis students produce a lot of different data sources such as annotation contents themselves and click stream data about when and how

00:50:22

when when and who and who put annotation and other metadata so if we make use of these different data sources and create a

00:50:37

important feedback that can help students to take actions to improve their social their engagement in social annotation activities or also

00:50:49

change their mindset to better engaged in social annotations it might be great so my general idea here is how could we make use of the social annotation data and how could we

00:51:03

create a great feedback design so the expected uh expected outcome is by using this data in front of feedback

00:51:14

not only students but also instructors can get a better sense of what students are doing and how students are engaging in social annotations and like compared to a

00:51:25

reading material b reading material got get most attention so this kind of quick diagnosis can be available and also based on that instructors can provide intervention

00:51:39

to particular students who seemed in need or problematic and also instructors can change the order of the reading materials or take away a specific material or they can change

00:51:52

this kind of modification in instructions but going back to students so students are actually not only the primary source of

00:52:04

this learning data but also they can be the main target for this feedback so the information generate generated by students themselves can be fed back into students themselves

00:52:17

to help them to take actions for better social education engagement or also can help them change their attitudinal or motivational changes in social annotation and

00:52:30

the main reason about we are focusing on this action taking and motivational change is that uh i have been uh participating in several graduate level courses in niu we which uses

00:52:43

a hypothesis and other social annotation tools such as perusal but students mainly sometimes dislike just the use of social annotation they sometimes

00:52:56

perceive these activities as additional things that burden their coursework so we want to see but we already but the instructors see see the high value

00:53:09

of using social annotations for students to better conceptually engage in course topics and also increase the interaction between students not only as in offline but also

00:53:24

in the online spaces so that what kind of action takings can be possible based on this kind of feedback so based on the literature so there can be three different kinds of

00:53:38

actions so awareness can be part of action as or as a prerequisite for action so by looking at what is showing about their progress in

00:53:49

in social annotations students can can be better aware of their status of learning and also they can reflect on what they are doing good or what they are having

00:54:03

problems in their previous weeks of social education activities or if we could provide the peer references or the classes average or classes

00:54:14

um data then students can also monitor there and their classes progress together and they can be better motivated to keep engaging in social annotation

00:54:26

and also by looking at the feedback students can also increase their intention attention to the course and materials and also they can be more responsible for social annotation activities

00:54:39

later and also they can actively eat listening to and speaking in annotations and they can also change their argument argue their their way of

00:54:53

participating in social annotations when they actually look at what they are doing in the social annotations such as they can develop an argument or they can change their positioning in the threads

00:55:06

or they can develop the new questions or change the questions and the most most cited actions were actually the help and resource seeking behaviors

00:55:19

so by looking at the feedback students can further go to instructors by asking what kind of additional resources i can see or what kind of or just

00:55:33

asking the help to better understand the reading materials that they were not good at in social annotation at that week or they can also start they can also manage or plan their

00:55:45

their way of participating social annotation in a more strategic way however we all know that just simple exposure to data is does not always lead to meaningful

00:55:58

awareness and actionable insights and going back to that students always when they got the data and from feedback they have they can be challenged in figuring out

00:56:10

what to do with this and also students are usually reluctant to make decisions or take actions to improve social engagement based on this because they don't know what to do

00:56:20

or they were just afraid of using that and by the literature this kind of misconnection it came from students mistrusting data even though the data itself shows their

00:56:35

progress some students say it's not my data it's out of think or it's not showing my true progress and also students are usually having less experiences in

00:56:47

looking at their own learning data so this kind of deficiency deficit experiences help them to reduce their confidence in making sense of what the data tells

00:57:01

about them or what to do with them so and also usually those feedback tools especially the dashboard usually focused on awareness or monitoring

00:57:13

rather than helping students to burden prompting students to do something after the after looking at the feedback so then my other uh

00:57:26

question is then okay so we all know this kind of uh potential exciting opportunities but also we know the challenges so how do we design actionable data and firm

00:57:38

feedback so my positioning about that is the most problems of challenges in making use of the previously developed feedback

00:57:49

is usually from either the researchers side or just the developer side or not including students or not including teachers for designing process

00:58:01

so i understand the co-design conceptualization for developing this kind of feedback tools and also involving teachers for that so what i mean by co-design with

00:58:15

students is to design and characterize the values or analytic metrics and tool features that can help students to take actions in a way to improve their

00:58:27

social annotation activities so we think this kind of way can be helpful because this can address several ethical per aspect of learning analytics what i

00:58:39

hear say la from students perspective by involving students in designing on this kind of co-design process students can develop their awareness of what kind of data from social notations can be collected

00:58:54

and they can also increase their agency in their privacy control and they can increase their trust in of data use and also we want to get some idea about how to

00:59:06

make a balance between um letting them know about we will gonna collect your data and then gaming behaviors happening and how could we make a balance about it

00:59:18

so we're going to have some broad idea about iterative co-design sessions so first we can start from needs and problem analysis that students have

00:59:30

especially about social annotation engagement so we need some the real experiences from students and by listening then we

00:59:41

we can make students to develop and formulate a set of values which they think very critical for effective engagement with social annotation so what i mean by values here

00:59:53

is for example oh i think uh providing um question is very important for social annotation activities then this kind of questioning can be one value

01:00:05

and like constant interaction with peers is very important then that can be another value so by setting up the values we can think about later than what kind of analytic metrics students

01:00:18

really want to see so for example it can be a like the graph showing uh the progress of their great quality of this

01:00:30

this questioning development or it can be just a it can be written by a text just one text of oh you are doing good in this week so we want to get some rare uh

01:00:43

perspective from students about it and then as a final stage we want to help help students to develop the and prototype the finer tool which we expected to

01:00:56

purchase as kind of the weekly report or as unlike html version that visualize a set of metrics and with explanations and in addition to that we also think it's very

01:01:09

very important to involve instructors indeed in designing the course activities of using this kind of data and firm feedback as part of the social annotation activities

01:01:21

um so we think this is another step for designing so the purpose of doing that is to we we get more we want to integrate students use of data informed

01:01:36

feedback activities into core course activities and therefore students can increase their awareness and action taking in a way to help them better participate

01:01:48

in social annotation so integrating strategies of feedback is very critical for them not only to revisit the tools but also make use of the feedback tool for their annotation

01:02:02

engagement so we kind of suggest four different principles from the literature so the first one is we can start from needs and problem analysis for

01:02:14

implementation and then we need to think about integration piece which is how to incorporate this feedback use into core course activities tied to course goals

01:02:26

how can we introduce this feedback into students and so and by and another piece is how to implement this feedback to increase student agency in their own

01:02:39

learning not only just introducing the tool and just letting them use and just observe what they are doing we really want to see how students make use of

01:02:50

this feedback into their own learning and um the next phase is about dialogue so based on the literature it has been very critically argued that it is very

01:03:02

important to being transparent especially in implementing this kind of analytic feedback or learning analytics dashboard to students so in what ways to inform students of the availability

01:03:16

of their data collection and how to get agreement about it and how to inform students of the existence of this feedback so being a transparency

01:03:30

way is really important and also in a way to facilitate dialogue for their social annotation engagement between students and instructors so what i expect is also to see

01:03:41

how instructors and students make a dialogue and conversation mediated by this feedback so then the next thing we can think about is

01:03:55

okay so all of the great imaginary ideas then how could we imagine how effective this design would be or how could we analyze its impact or how could they think about in what way students make

01:04:08

use of it so i have several questions here so the fundamental questions about daytime feedback is during social annotation activities do students really benefit

01:04:20

from looking at their own data and how do students feel about it so maybe as an audience you might think about oh okay this is a good idea but i'm not fully buying this idea because

01:04:34

it can be an additional burden for students so i think it's important to check and confirm that students can benefit from looking at their own data

01:04:47

and in what ways we could improve this tool in a way a better benefiting student and another point to think about is from learning awareness to actions for

01:04:59

improvement improvement in social annotation activities so how do they how do students make sense of what the feedback tells about their learning is a very critical question to see and

01:05:12

whether and how students translate what they find in the feedback into actions they want to take for further social annotation engagement

01:05:23

and then in what ways and of course as i introduced several sets of specific actions i also really want to see specific convergences about how does this experience help them

01:05:37

take actions for facilitating their engagement annotations like awareness um like adaptation or attention like what kind of actions do they take and how does this experience finally

01:05:50

impact their learning attitude in the course and also motivation in social annotation further so uh because uh the reason why i present uh these kind of uh conceptual

01:06:05

contents is i'm not conducting any study yet but we are working on uh implementing this interesting design to the classes in the upcoming bar we into the courses

01:06:17

in two courses which use hypothesis or other social annotation tools for their courses so we really want to implement this design and see what happens and why something did not happen will

01:06:30

not happen so our my general idea about it is first uh implementing social annotation activities to students like by

01:06:41

having them equally participating in the reading materials using hypothesis and then see in what ways they can get the data and firm feedback and also

01:06:52

how could they make use of it by interacting with them and then after that i wanna see what kind of impact happening during the term and also at the end of the term so yeah this is kind of the general

01:07:06

model that i have in mind about how to designing and supporting the actionable social annotation engagement in in courses so

01:07:18

yeah uh this is the end of the presentation so feel free to ask me questions or or so if you have any questions later or feel free to reach out to me using my

01:07:31

emails here thanks thank you so much yamg i see a round of virtual applause and heart for your for your talk um this is great and invite

01:07:46

um colleagues who are attending this to post questions in the q a or raise your hand if you want to ask ung a question directly i see another already a question coming from jeremy

01:07:58

which is uh is implementation whether it's a typo or not i don't know or information or data informed feedback personalized report of your study going to be produced by you or by social annotation tool so

01:08:14

yep i think that's a good question for clarification so yeah it's actually both so first what i'm trying to do is summarize a set of the data sources that

01:08:27

can be measured by or collected by the hypothesis tool or other annotation tools and provide students a set of these kind of potential data sources

01:08:38

and then make them produce the design or ideas for what would be interesting data sources they want to see to improve their social annotations and what kind of analytic metrics they want to see in the

01:08:52

feedback report and after doing that i want to make them develop some idea about what the report would look like and then as a final step

01:09:04

i want to uh the report with instructors and me together so definitely it's not just made by the social annotation tool

01:09:15

but uh i have some idea about it because the the other social annotation tool such as perusal has some automatically produced analytics but it's not for

01:09:30

students it's usually for the teachers and it's for grading so they allow automatic rating which uh which those instructors can use but students cannot see at all

01:09:43

so yeah do i answer your question okay i think so i think yeah that's a great response any other question quick clarification question for yonji

01:09:57

before we move on to the panel discussion and um i don't see more questions in the q and a or in chat um

01:10:12

at this point i think um i want to thank all three panelists for your wonderful presentations and especially um thanking you for sharing your ongoing dissertation

01:10:25

research i think that's a very very generous of you to share your ongoing thinking which is really inspirational to me and i see a lot of connections across

01:10:38

different projects and and shared investment in a few principles such as co-design or centering what we know about learning

01:10:50

and teaching and also the researcher instructor partnership i see that across all three projects as well and so i have some questions i want to invite you

01:11:03

to think with us and i also want to encourage the audience to post your question to the panel you could direct your question to the everybody on the panel or you could direct your question to a specific panelist

01:11:15

as we launch this panel discussion so my first question as i was really thinking about the design element that is cutting across different project the question i want to ask is because

01:11:28

you do work directly with instructors or with students i want to learn from you a little bit more in terms of how do you see the instructors contributing to the

01:11:41

design or the design product in your project which could be a framework or a student dashboard or tool and any tips you have that you can share

01:11:53

with the audience in terms of ways to really engage the instructors to be active co-designers or contributors to your design so feel free to jump in we have a we

01:12:08

have a thought to share with the audience i can go first if that's okay um so i would say i think instructors they're really a key element during our

01:12:23

collaboration because they're experts of teaching and the curriculum design so first as we collaborated i would say we all have shifted our identities a little bit

01:12:34

we're we're not just researchers and they're not just educators we all became the research informed practitioners as we're all trying to turn research into practice

01:12:46

by co-designing the social annotation so during our design we have co-design meetings and then they shared their course objectives their insights and their teaching

01:12:59

strategies then based on that we introduced some scaffolding activities and collaborative learning strategies from the research literature and i also really

01:13:12

appreciate the strong wrapper we have established during our collaboration i think that's really important so throughout this semester we kept our routine design meetings as a chance

01:13:24

to share both positive or negative updates and solve problems together so our co-design has always been an ongoing process it's not just we designed the activity at the beginning

01:13:38

and never talked again so there are always great ideas coming from the design meetings throughout the semester with the instructors and they have inspired us a lot as well

01:13:50

so i think the for the tip i think the first step is to build the partnership um and respect and listen to their needs insights and confusions as we are here to solve a problem

01:14:04

together yeah that that's my thoughts thanks shunyan anything to add um yum g and chris yep uh i cannot so uh zinglan shared a

01:14:23

great uh insight about how to like the importance of involving instructors in the whole process of designing and implementing

01:14:35

the specific intervention for social annotation activities and i want to add some another layer about the importance of involving instructors

01:14:46

or teachers in not only designing the intervention itself but also another work for designing the integration work to the course

01:14:57

so in my case i want to um not only not only make students participate in social annotation activities i also want to do another experiment of could

01:15:09

introducing the the data feedback to students so in that case usually instructors have been not involved in the implementation project but rather

01:15:22

rather instructors are usually involved in designing the tool of this kind of report or feedback but they were not leading the main role or reading role of

01:15:36

introducing this kind of data to the class or students so from my uh pilot study not involving social annotation but in overall discussion activities uh

01:15:49

students were really wanted to see see that instructors really emphasize the importance of why we are using uh social annotations or why we are using the discussion post

01:16:03

and how my activities really really impacts my grade or my future career etc so i think it's really important for

01:16:14

us to help instructors continuously participate and make interventions during the semester during the course of the semester by mentioning the

01:16:27

the purpose or importance of why using the using social social annotation tools or in my case also why using data feedback for social annotations yeah

01:16:44

yeah and i loved that yonji and shinron both you know emphasized in their presentations it's like this co-design idea with instructors because um i mean i i'm thinking about a couple of

01:16:57

things like one uh coming from you know what we know about learning and and uh you know learning sciences is that context matters and uh the instructor is

01:17:09

typically the one who understands the context the best and so when we're trying to design or understand what's going on i mean obviously students or whoever's

01:17:21

in that in that context is going to understand it better than the outside researcher typically and there are other approaches that we have such as research practice partnerships or um

01:17:33

implementation research or there's other kinds of approaches that we can take um that kind of involve their voice more in the research but um i i think it is so important to make sure that we're getting instructors voices

01:17:46

in uh in in this because uh some a lot of times they're left out of this you know we focus so much on students because uh we we want to make sure that students are learning but instructors can give us so much

01:17:59

insight on on what is going on in the classroom and little little things that we might not think about that are really important for that particular context i'm also probably quite biased i come from a practitioner background i taught high

01:18:14

school for seven years before starting my phd and i really love teaching and so there's there's a background part of that as well the other thing i want to mention with um with mine that i kind of i kind of

01:18:27

said this that the instructor inquiry team these focus groups that we had that um i tried to i intentionally designed it as a sort of a professional to mimic sort

01:18:39

of a professional learning community or professional learning group with the instructors that i didn't want it to be i'm coming in and i'm going to ask you some questions and you're just going to answer but we started each session with

01:18:52

what like what's going on with annotations in your class like what are some challenges that you're having um with annotations that you want to talk through with the group because we had other instructors who had been using

01:19:04

annotations and so i wanted that space to also be kind of for them to just uh yeah have this learning community to be able to say you know what um and you know one of there were two specific instances that made me feel

01:19:17

like okay i i don't know if i accomplished it 100 but at least uh some of the instructors saw this as a great opportunity because one of them said you know a lot of my students are just doing these one-off annotations there's not a

01:19:30

lot of threaded conversations happening what are other people doing to have threaded conversations now that was also you know one of you know a potential research question or something i wanted them to talk about but instead of me coming up with that

01:19:43

question they already had questions and i kind of let them lead that discussion and then another one where i said okay here's the general topic that we're going to talk about in this session and uh one of the instructors the first

01:19:56

thing he said uh in in the session was um i'm so glad that you posed this question because i've never thought about this before and i want to talk to everybody about it

01:20:06

um and so i i thought those were um kind of just a couple of instances that made me realize yeah like this is like for the research part that's great like i want to make sure

01:20:19

that i'm getting um you know good data for my dissertation but also i want to make sure that this is useful to the instructors and getting their perspective and having them uh you know talk through some of this can really bring some rich

01:20:31

data um that because they're invested in it now that they're uh you know they want to have some of these conversations and so because they're invested uh it makes for much more interesting

01:20:44

uh data and it also benefits them i wanted to make sure that you know part of this is it's benefiting these instructors thank you thank you that's a really good strategy to think about the community uh chris i really love the strategy and

01:21:00

we'll try to find a way to incorporate that in our future work as well and um i'm looking at the time and i want to really invite audience to if you have any questions for the panel

01:21:12

to uh ask the question um in the chat or directly raise your hand you'll see uh some um some really interesting conversation going on in the chat thread

01:21:24

which is great and and for me since there's no more question i'm really going to take that advantage of being as a speaker on the panel uh because i've been really thinking about this

01:21:37

question myself and want to really listen to the panelists as well is we use the term social annotation quite often in different contexts right and as a researcher myself from the learning

01:21:50

sciences we emphasize social cultural and political dimensions of learning besides cognitive and so on right there's multi-dimensional phenomena learning is a multi-dimensional phenomenon so i want to really engage

01:22:02

the panelists and ask the question what do you think about when you listen or use the word the term social annotation what types of um what do you mean by social

01:22:15

or how would you interpret the word social when you when you think about social notation because i'm really curious about your thoughts on this when i think about this term social and

01:22:34

when i use the term social annotation i was also thinking what social i mean i mean by because i think like this social annotation can be also widely used as like online collaborative

01:22:48

annotation or online um like collaborative discussion text or something like other there are so many different things so my answer is

01:22:59

i think it can be divided into three different factors so one is by the in what ways um the collaboration is and

01:23:12

is mediated by so the first thing is learner and learner like of course being social means making interaction between peers or between the learners but i think it's

01:23:25

like a social annotation is also individual activity at some level as the other audience said before so i think it's also more about learner to context interaction so like

01:23:39

learners can interact with the concepts and the materials themselves or text or topics in a more conceptual way so i think that's the second thing and the third thing is also

01:23:52

a learner to instructor learner to teachers so i think it depends on uh how much instructors want to involve in the annotation because usually some instructors do not want to

01:24:06

participate in annotation at that much but if yes then i think also getting scaffold from instructors in a better way to fully understand the concepts in the

01:24:18

materials or uh get exposed to new ideas from instructors is also another thing i can see as a social thing thanks a lot yunji xi you can go next

01:24:34

okay so yeah i i really like how umd um kind of emphasized the connections between the students and teachers and the context and i have a really similar ideas and first i think uh social of course does not mean

01:24:47

students are just talking to each other we're discussing things all the time but in the social annotations context or the learning context the social needs to be uh needs to support learning so it's not just two people

01:25:00

work socially but their mind or cognition associated to the learning context are communicating socially so sometimes i like to call it social cognitive learning

01:25:12

which means which means we expect there will be a cognitive achievement like understanding of the topics or application of the knowledge facilitate facilitated by the social

01:25:25

activity and also for my understanding of social learning or social annotation students don't just learn for themselves but also they take the responsibility

01:25:38

to contribute to the community by sharing their voices and helping others to understand the topics but as i said students are not natural collaborators um they're not natural social cognitive

01:25:51

beings as well so when we talk about social annotation we need to keep in mind that there are there needs to be a sophisticated design or support to

01:26:04

help them achieve the goal so in conclusion i think social is something like an ecosystem um in social annotations all students our teachers will take the their shared

01:26:16

responsibilities to contribute to the community's knowledge co-construction and that's my understanding of the social yeah and i don't i don't know that i

01:26:30

would add much more i thought those were both really lovely that i really like this idea of you know that we can't we don't want to lose the the learning aspect of it um in in the

01:26:42

work that we're doing in particular and so um you know i like that idea of uh you know thinking about that the co-construction of of knowledge in in this space and that comes from these

01:26:54

intertextual inter-subjective uh connections that we're making um as we're annotating uh so i i don't i don't think i have anything else to yeah i conquered those are really good

01:27:09

great points um i think really adding a lot of richness to what we think about social annotation and i think about the future work that's uh in front of us as a as a community if i could use this term

01:27:21

to describe our social annotation research community i think there's a lot work to be done and really appreciate of you sharing your work and i hope to learn as your projects move forward and

01:27:35

with that uh we are five minutes over and i want to thank all the panelists for your uh contribution to the panel and also thank you uh thanks to the attendees and for

01:27:46

participating in this panel for any uh anything from you you want to let us know or kind of housekeeping things to um to share with the audience

01:28:02

um hold on i'm gonna make it so that i'm on the screen there we go i have the power uh no i just that was great thank you so much um i just want to

01:28:14

encourage people you know to follow this incredible work that you all are doing and again thanks to everyone who's here and um we should probably wrap this up so that

01:28:28

people can get to some of the next sessions um check the schedule hang out in the lounge meet people talk to people and don't forget about annotate this party later today it's the penultimate one

01:28:41

uh it's going to be great i've got something kind of big plan for that so anyway that's another topic thank you again and uh yeah we'll wrap this up all right

01:28:55

sounds good take care everyone it's so great to see you