my lost words
words for nerds
Wednesday, 17 April 2013
Monday, 8 April 2013
Thursday, 28 March 2013
if i were to write
I'd write of lying, wrapped up in her soft body.
I'd write of the power of those golden eyes, and how they make me want to speak. To speak words of love, of those words harder to draw from me than blood from a stone.
I'd write of sunny Sundays, late to rise, simple breakfasts and walking through the lazy city. Of returning sun-drunk and at peace with the fresh air of an early Spring. Making love with the blue blue sky and the mountains just outside the windows, then wrapping ourselves in ourselves and falling asleep.
I'd write of this long road I've wandered these last years. Meandering, full of dead ends and glittering distractions. Winding across continents and through lives, passing through the years with only a notion of a destination.
I'd write of the struggle, upon seeing that imagined destination approach, of accepting it. Of seeing the off-ramps and the safety they promise, of tasting the worry of all that the destination offers. Knowing that to arrive, I'll need to do what I've been so afraid of.
Such little words that take so much to say.
I'd write of the power of those golden eyes, and how they make me want to speak. To speak words of love, of those words harder to draw from me than blood from a stone.
I'd write of sunny Sundays, late to rise, simple breakfasts and walking through the lazy city. Of returning sun-drunk and at peace with the fresh air of an early Spring. Making love with the blue blue sky and the mountains just outside the windows, then wrapping ourselves in ourselves and falling asleep.
I'd write of this long road I've wandered these last years. Meandering, full of dead ends and glittering distractions. Winding across continents and through lives, passing through the years with only a notion of a destination.
I'd write of the struggle, upon seeing that imagined destination approach, of accepting it. Of seeing the off-ramps and the safety they promise, of tasting the worry of all that the destination offers. Knowing that to arrive, I'll need to do what I've been so afraid of.
Such little words that take so much to say.
Wednesday, 20 February 2013
Monday, 18 February 2013
Monday, 4 February 2013
Monday, 28 January 2013
peer assessment of oral language production: a resource for educators
Much of what I do in my master's program is create comprehensive guides for teachers to help improve their teaching practices. These guides usually address educational research and make a case for why and how to implement strategies in the classroom. Last year I wrote on Critical Pedagogy, as well as an extended piece on how to address homophobia and heteronormativity in the ESL classroom. The former I posted here, while the latter is awaiting publication in an anthology on international language learning.
I wrote the following piece in the Fall and am posting it here to help educators struggling with the idea of incorporating peer assessment in their practice.
-----------------------------
Introduction
The value of peer-based
assessment
Holistic Learning and
Higher-Level Reasoning
As a Socializing Force and
Self-Regulated Learning
Frequency and Quality of
Feedback
Learner Perceptions of Peer
Assessment
Effects on Competency
Concerns about peer assessment
Do peer-assessment grades correspond
to those from the teacher?
A framework for productive
and valid peer assessment
How crucial is a strong
correlation between teacher/peer assessments?
Training & Practice
Advanced Learners
Familiarity among Peers
Peer Feedback
The Challenges of Group
Peer Assessment
Student Familiarity with
Assessment Criteria
Concluding Remarks
References
I wrote the following piece in the Fall and am posting it here to help educators struggling with the idea of incorporating peer assessment in their practice.
-----------------------------
Peer
Assessment of Oral Language Production
|
A
Resource for Educators
|
December 2012
Abstract
This paper addresses the potential benefits and challenges of utilizing peer assessment of oral language in the second language classroom. By reviewing the literature within and beyond the second language education community, it looks at arguments for and against utilizing peer assessment in education, and draws on a number of studies to show that opinion within the educational community is heavily divided regarding the utility of peer assessment. This paper argues that despite strong and valid misgivings about the role peer assessment should play in the classroom, the potential benefits are too great to forsake. The paper then reviews the literature to provide a series of strategies to maximize the value of peer assessment, such as focusing on more advanced learners, properly training peer assessors, ensuring that students have input into the assessment criteria and are well trained in applying it. The paper concludes with a statement in support of peer assessment of oral language in the language learning classroom, assuming that the strategies laid out here are followed and the limitations of the practice are recognized.
Contents
This paper addresses the potential benefits and challenges of utilizing peer assessment of oral language in the second language classroom. By reviewing the literature within and beyond the second language education community, it looks at arguments for and against utilizing peer assessment in education, and draws on a number of studies to show that opinion within the educational community is heavily divided regarding the utility of peer assessment. This paper argues that despite strong and valid misgivings about the role peer assessment should play in the classroom, the potential benefits are too great to forsake. The paper then reviews the literature to provide a series of strategies to maximize the value of peer assessment, such as focusing on more advanced learners, properly training peer assessors, ensuring that students have input into the assessment criteria and are well trained in applying it. The paper concludes with a statement in support of peer assessment of oral language in the language learning classroom, assuming that the strategies laid out here are followed and the limitations of the practice are recognized.
Contents
- Introduction
- The value of peer-based assessment
- Holistic Learning and Higher-Level Reasoning
- As a Socializing Force and Self-Regulated Learning
- Frequency and Quality of Feedback
- Learner Perceptions of Peer Assessment
- Effects on Competency
- Concerns about peer assessment
- Do peer-assessment grades correspond to those from the teacher?
- A framework for productive and valid peer assessment
- How crucial is a strong correlation between teacher/peer assessments?
- Training & Practice
- Advanced Learners
- Familiarity among Peers
- Peer Feedback
- The Challenges of Group Peer Assessment
- Student Familiarity with Assessment Criteria
- Concluding Remarks
- References
Introduction
Oral assessment
of second language acquisition is a difficult question in the world of
education. While most educators would agree that oral competency is fundamental
to feelings of success in second language learning, it is often treated as less
important than grammar and aural comprehension. From my own experience, I was
able to complete 21 credits of UBC’s Spanish undergraduate program without ever
having to do an oral examination. While unfortunate, the situation is
unsurprising given the difficulties of oral language assessment. Not only is
oral assessment more time consuming for educators, it is also trickier to
assess on global scales while maintaining strong validity and reliability in
the assessment. Given the challenges educators face in performing oral
assessment, it is not surprising that few educators incorporate peer-based
assessment of oral proficiency in their classrooms. However, there is a strong
case to be made for the value of peer-based assessment, both for pedagogical
and economic reasons. In the following paper, I will discuss the benefits that
come from peer-based assessment of oral language proficiency, concerns
regarding the validity of peer ratings compared to teacher ratings, and offer
strategies for how educators can use peer-based feedback without sacrificing
validity and reliability in their assessment practice. I believe that this
issue is of such significant importance that it merits a strong investigation
because of the potential benefits identified in the literature.
In the following
paper, I draw on research from a number of contexts and disciplines. Many of
the authors mentioned below work outside of the second language classroom, but
this paper operates under the assumption that the pedagogical effects addressed
here function in a cross-disciplinary manner and are not constrained to the
particular contexts in which they were studied.
The value of peer-based
assessment
As education
culture in the West develops, we turn further away from the traditional rote
learning of the past and more toward modes which take into account issues of
socialization, critical thinking, metacognitive skills and learner autonomy. In
particular, we have seen growth in activities that encourage students to
reflect on their own learning. For example, “during recent decades,” Segers et.
al (2003) note, “there has been an increase in the implementation of self- and
peer assessment“ in the classroom (p. 130). This is understandable, as practicing
peer assessment teaches our students so many skills we desire for them: they learn
how to analyze texts, monitor the language production of others, and assess
their performance. We want our students to learn these skills because if they
learn to practice them on others, they will intuitively begin to use them on
themselves. This development of metacognitive skills is fundamental and invaluable
for higher-level learning, something desirable for all students.
Holistic Learning and
Higher-Level Reasoning
Research shows
that peer assessment inspires in learners holistic approaches to learning as
opposed to instrumental approaches that lead students to study for the purpose
of passing a test (Entwhistle, 1987; Gibbs, 1992). When our students learn for
the sake of learning, they are more likely to be active participants in
classrooms, take responsibility for their own learning, and maintain strong
levels of motivation for language learning even after the course has finished (Ozogul
& Sullivan, 2009). And not only are they learning for better reasons, but
they are learning to think better. Birdsong and Sharplin (1986) study peer
assessment and how it can improve higher-level reasoning in learners as well as
higher level cognitive thought. “Peer assessment,” Saito (2008) notes, “is a
meta-cognitive activity in which student attention is drawn to the features of the
language learning task” (p. 554; Falchikov, 2005). Peer assessment promotes
strong meta-cognitive skills by “encourag[ing] reflective learning through
observing others’ performances and becoming aware of the performance criteria”
(Saito, 2008, p. 554). This connection to student-centred learning is also
noted in Oldfield and MacAlpine (1995), whose study of undergraduate learners
speaks to the potency of peer assessment in promoting learning that focuses on
developing well-rounded, intellectually mature students. The development and
promotion of meta-cognitive skills helps learners progress and supports
self-regulated learning, encouraging students to practice productive learning
habits. If we incorporate peer assessment into our classrooms, we will promote
an engaged, intrinsically motivated, critical classroom which will support student
success.
As a Socializing Force and
Self-Regulated Learning
Peer assessment can
also bring our students together. Earl (1986) discusses how this mode of
assessment acts as a socializing force by promoting interpersonal relationships
between students, as it encourages students to view themselves as working as a
team, instead of as individuals, toward their learning goals. This is a hugely
underestimated benefit of peer-based assessment. By promoting social cohesion
and a cooperative, as opposed to competitive, learning environment, educators support
self-regulated learning, powerful feelings of autonomy, and strong perceptions
of self-efficacy in students (Bandura, 1993). Dornyei (1994a) also speaks to
the value of cooperative learning environments, noting that they support
students better than competitive environments, as students have less anxiety
and more positive feelings about their learning. Anybody who has been in a
second language classroom can relate to that sense of anxiety when asked to
demonstrate oral language ability. Students who perceive their classroom to be
supportive and cooperative will feel far less anxiety than those who feel they
are in a competitive, judgemental environmental. And learner satisfaction and
success will be bred far more easily in such a positive space.
Zimmerman’s
(1990) theory of self-regulated learning addresses how students take ownership
of their learning process and guide themselves to success in their learning
environments. As Zimmerman (1990) explains, it is a study of “how students
metacognitively, motivationally, and behaviorally initiate and direct their
learning processes” (p. 185). Luc de Grez and Roozen (2012) connect
self-regulated learning to peer assessment by noting that it “results in a more
active involvement of students in their own learning process” (p. 130). When we
encourage our students to take ownership over their own learning process, we promote
self-regulated learning, while Luc de Grez and Roozen (2012) point out that
“both internal and external sources of feedback are helpful to foster the
calibration process to attain higher performance levels in the context of
self-regulated learning” (p. 130).
Frequency and Quality of
Feedback
One
limitation of relying on teacher-provided feedback on oral language skills is
the problem of economy. For a teacher to provide substantial, quality feedback
to a student, he or so will need to devote significant time. For example, it is
challenging for a high school teacher to provide feedback to 30 students in
each of his or her classes, which means that feedback for each student is
challenging to come by. This works against fostering strong perceptions of
self-efficacy, especially in language classrooms where proficiency cannot be
checked by reading an answer key or marking en-masse, as it is in other subjects.
“Positive feedback,” Rigby et. al (1992) argue, “tends to strengthen perceived
competence and enhance intrinsic motivation” (p. 175). Oxford and Shearin
(1994) also note that regular feedback supports productive goal setting
techniques, an integral function of self-regulated learning.
These authors subscribe to
Bandura’s (1993) theory of Perceived Self-Efficacy, which posits that
motivation must exist for a learner to engage in learning behaviour. “Conception
of ability as an acquirable skill,” Bandura (1993) writes, “foster[s] a highly
resilient sense of personal efficacy” (p. 121). Through his work, Bandura
identified a relationship between how people perceive their potential to
acquire a new skill and their eventual success in doing so. While strong
perceptions of self-efficacy do not lead directly to skill acquisition, they
set the learner up for success by facilitating other highly effective learning
strategies. Students with strong perceptions of self-efficacy “remained
steadfast in their perceived efficacy, despite difficult standards to fulfill
[and] they continued to set challenging goals [and] used analytic strategies in
effective ways” (Bandura, 1993, p. 121). Students who believe that they can
achieve their overall goal of language acquisition are empowered by their
perceptions of self-efficacy to enact the triadic reciprocal determinism that
Zimmerman (1990) discusses. One of the most effective ways to support strong
perceptions of self-efficacy in students is to provide them with regular,
constructive feedback, and peer assessment is an ideal tool for us to realize
this goal.
It is
irresponsible not to take advantage of the wealth of feedback opportunities
available to us in our classrooms, as fellow learners are capable of providing
“immediate, individualized, and [rich] feedback” to their fellow students which
is “formative in nature, [and] has a clear potential of fostering the
subsequent learning process” (Luc de Grez & Roozen, 2012, p. 130). The
pedagogical and economic benefits of incorporating peer assessment into the
language learning classroom are impressive; as funding for education becomes
more restricted, educators will need to either cut back on their services (and
thus reduce the quality of education) or find ways to be more efficient. Peer
assessment is one of these ways. Through peer assessment, educators are also
able to provide ample feedback to support students’ learning process and foster
strong senses of self-efficacy and self-regulated learning.
Learner Perceptions of Peer
Assessment
As most educators
recognize, our best laid plans come to naught if we cannot get student buy-in. “Student’s
perceptions of peer assessment will influence their willingness to take into
account the feedback generated by peer assessment and to actually do something
with that feedback” (Luc de Grez & Roozen, 2012, p. 138). If our students
do not appreciate or value the process of peer assessment, they will dismiss
the feedback, formal or informal, and not modify their learning process
accordingly. However, numerous studies have shown that learners appreciate and
value quality feedback from their peers, reflecting “a very positive attitude
towards the value of peer assessment” (Birdsong & Sharplin, 1986; Luc de
Grez & Roozen, 2012, p. 138; Zhu, 1994, 1995).
Effects on Competency
Thus far, we
have reviewed how peer based assessment can contribute to the development of
meta-cognitive skills, how it supports self-regulated learning, perceptions of
self-efficacy and learner autonomy, and how it can enable to educators to
supply learners with the substantial, quality, formative feedback that will
enable them to learn effectively and efficiently. These are all facets of the
language learning process and many of them support learning, but none are
evidence of second language acquisition. Where is the direct evidence connecting
peer assessment to improved performance in the classroom? A number of
researchers have studied this.
In the context
of developing presentation skills, Mitchell and Bakewell (1995) tested the
efficacy of peer feedback with undergraduate students. They found that students
benefited even more from peer feedback than from instructor feedback and that
significant improvements were noted in performance after having participated in
peer assessment. This study demonstrates a direct link between receiving
feedback through peer assessment and actual improvements in performances for
specific tasks such as oral presentations, a common practice in language
learning classrooms.
Stanley (1992) studied
the quality and reception of feedback in peer assessment. She found that
students trained in providing peer assessment gave specific, constructive
feedback and were more likely than their peers to revise their own work in
response to peer feedback. Not only did peer assessment involve giving a global
grade, peer assessors were able to address “concrete, specific issues” and
provide their peers with “a blueprint for revision” (Stanley, 1992, p. 229). The
same results are noted by McGroarty and Zhu (1997), who found that peer
assessment led to students spending more time revising their own work. What
these studies show us is that if we introduce peer assessment into our
classrooms, the opportunities for academic development are huge. But what are
the challenges?
Concerns about peer assessment
Given that there
are so many potential benefits associated with peer assessment of oral language
learning, why isn’t it widespread? One of the most commonly voiced concerns
from educators is that peer assessment is a threat to the reliability and validity
of the assessment process. I will address the question of consistency between
peer- and teacher-based assessment below, offer strategies on how to help
improve consistency between them, and challenge the idea that consistency
between grades is of such importance.
Do peer-assessment grades correspond
to those from the teacher?
Two
fundamental qualities of good assessment practice are reliability and validity.
Reliability demands that the same assessment record (in this case, the oral
performance) be given a reasonably similar mark regardless of which student
assesses the performance. If different peers give wildly different grades to
the same performance, the reliability of the assessment is questionable.
Similarly, if the same assessor gives highly different marks to the same performance
based on the setting of the performance, the reliability of the assessment
suffers. Validity, while similar to reliability, is different in that it
addresses whether the assessment measures what it purports to. Assessment practices
should be free of bias and address the performance, not other factors such as
the assessor’s relationship to the assessed or bias against certain groups.
With this in mind, we can consider the debate regarding consistency between
teacher marks and peer marks to be a matter of validity (Falchikov &
Goldfinch, 2000).
The
question of consistency in assessment between peers and teachers is fiercely
debated with conflicting evidence. A number of studies have shown that peer
assessment is correlated with that of the instructor (Forde, 1996; Jafarpur,
1991; Patri, 2002; Saito & Fujita, 2004), though some of these studies come
with caveats noting the importance of clear criteria (Patri, 2002) and the
differences between low- and high-proficiency students’ abilities to perform
assessment (Falchikov & Boud, 1989; Miller & Ng, 1994; Patri, 2002).
Forde’s (1996) study of 50 Cantonese-speaking students shows that
peer-assessors were able to assess the oral competency of their peers in a
manner similar to that of the IELTS-training examiner. While Cheng and Warren
(2005) are supportive of peer assessment of oral language skills, they also note
that the students in their research were able to give “marks for the language
criteria [which] lay within one standard deviation of the class teacher’s” (p.
103). With all of these studies evidencing peer assessment competency, it’s
surprising to see a host of research against peer/instructor consistency also
reported in the literature.
Many
of the same studies which speak to the correlation between peer/instructor
assessments also note that inflation may exist in the peer assessments (Cheng
& Warren, 1999; Patri, 2002; Saito & Fujita, 2004). This is
unsurprising, as it’s easy to imagine that sympathetic tendencies would compel
many peers to err on the side of giving their fellow students passing grades.
The literature supports this theory and shows that across many contexts
students give fewer failing marks to their peers, though they also give fewer
high marks. Cheng and Warren (2005) note this and comment that “there is a very
clear difference in the marking behaviour across all classes whereby the class
teachers awarded marks in a range 1.5 to 3 times that of their students” (p.
306). While students do not like to give low marks, it seems they also avoid
giving high marks. This disinclination is also noted in by Patri (2002), who
comments that peer assessors “were overestimating the performance of low
achievers” (p. 123; Mowl & Pain, 1995; Sullivan & Hall, 1997). The
overall effect of this narrowing of the mark spectrum in peer assessment is
“found to be … to the benefit of the weaker students and the detriment of the
more able ones” (Cheng & Warren, 2005, p. 105). Woolhouse (1999) believes
that peer assessors have a difficult time making honest judgements that would
negatively affect their peers, and Oldfield and MacAlpine (1995) support this
theory and believe that peer assessors are emotionally prejudiced.
This
inconsistency results is a serious challenge to the validity of peer assessment
or oral language. How can educators defend their assessments if so many valid
critiques are made of them? Some researchers believe educators cannot defend
peer assessment in the face of these critiques. Jafarpur (1991) conducted a
study looking for correlations between peer/teacher assessments in an Iranian
EFL classroom, concludeding that untrained assessors were not able to provide
comparable ratings to those of the teacher. Cheng and Warren (2005) similarly
decide that the costs of peer assessment outweigh the benefits, and that their
results are not “consistent enough for us to be able to argue that peer
assessment can be used reliably alongside teacher assessment” in language
learning programs. Clearly, to convince language educators to incorporate peer
assessment into their programs, we need to create a framework that responds to
these criticisms.
A framework for productive
and valid peer assessment
How crucial is a strong
correlation between teacher/peer assessments?
The first and
most contentious question that should be asked here is: are teacher marks the ideal result of assessment? Can we argue that
teacher-based assessments are perfectly valid and reliable, incorruptible by issues
of bias, previous experience with the testee, or general mood changes? Of
course not, yet many educators unquestioningly assume that their assessments
are the ideal. As noted by Luc de Grez and Roozen (2012), “the a priori
assumption that assessment by a teacher is more reliable and valid can be
doubted in some contexts” (p. 131). Falchikov and Goldfinch (2000) note that
“work in the area of marking or grading is fraught with difficulty, as teacher
marking has, itself, been found to be problematic (p. 288; also, Falchikov
& Magin, 1997; Newstead & Dennis, 1994). In fact, all of our experience
as educators and assessors may work against us, since “teachers add criteria
and/or indicators when judging the quality of an oral presentation … that add
to the unreliable, but often neglected, nature of teacher assessments” (Luc de
Grez & Roozen, 2012, p. 138). We need to question our presuppositions
regarding the validity and reliability of teacher-based assessments and
recognize that they are subject to the same challenges as peer assessments. With
that said, I will now address a number of strategies to improve peer assessment
validity.
Training & Practice
The most
effective strategy for improving the consistency between teacher/peer
assessments is to train peers in assessment and allow them ample opportunities
for practice. Cheng and Warren’s (1999) study analyzes the correlations between
teacher/peer marks of oral seminar and presentations. They found that
correlations between marks improved significantly between the initial seminars
and the later presentations, lending to the argument that practice makes
perfect. Luc de Grez and Roozen (2012) affirm that “the training of assessors
is very important” and Saito (2008) notes that “rater training seems to help
students provide more relevant comments” (p. 139; p. 574). Recognizing this,
educators who wish to use peer assessment in their classrooms should devote
ample time to training the peer assessors on how to perform assessment. They
should also revisit peer assessments to show how it measures up against the
teacher’s assessment, which will help close the divide between teacher/peer
assessments.
Advanced Learners
One consistent
finding in the literature is that intermediate- to advanced-level learners are
much more adept at assessment than beginners (Falchikov & Boud, 1989; Miller
& Ng, 1994; Patri, 2002). Novice students simply do not realize how much
they don’t know and are less likely to notice mistakes, instead focusing on
understanding the gist of the message. However, intermediate and advanced students
are able to focus on individual parts of speech to monitor appropriate usage
and overall communicative competence. In their meta-analysis of 48
“quantitative peer assessment studies comparing peer and teacher marks,”
Falchikov and Goldfinch (2000) found the correlation coefficient between
teacher/peer assessments to be .67 for novice students, .77 for intermediate
students, and .87 for advanced students. As “students in advanced courses
appear to be the more ‘accurate’ assessors than those in introductory courses,”
it makes sense to focus our energies on students at least at intermediate
levels (Falchikov & Goldfinch, 2000, p. 288).
Familiarity among Peers
An important
suggestion shared by Miller & Ng (1994) is that the students should be
familiar with each other. “Our students were better prepared for the task of
judging their peers” because they were “familiar with the testees’ oral
ability” (Miller & Ng, 1994, p. 53). At first this suggestion sounds like
it may introduce assessment bias, but the authors point out that “testers and
testees were familiar with each other and therefore the tests were conducted
under friendly circumstances” (Miller & Ng, 1994, p. 53). Jones (1985) also
points out that the “ease of elicitation in an oral interview is often affected
by the social relationship between the examiners and the examinee” (p. 81). The
first time two people meet, there is an immense amount of noticing that occurs
between them; they may notice particular physical attributes, or the distinct
speaking style of their interlocutor, or be too focused on making a good
impression that they fail to remember the person’s name. Familiarity between
the tester and testee directs the energy put into noticing aspects of speech,
which contributes to the validity of the assessment. And in the same way that
cooperative learning environments lessen anxiety and facilitate language
learning, a certain level of comfort with the assessor will help students
perform better.
Peer Feedback
Patri
(2002) researched the potential for peer feedback to improve the correlations
between teacher/peer assessments. She took groups of students who had received
training in giving peer feedback and divided them into control and experimental
groups. The experimental group discussed oral language performances before
providing individual peer assessments, while the control group provided peer
assessments without incorporating peer feedback. She found that the control
group’s correlation coefficient with the teacher’s ratings to range between
.46-.50, which is too low to be considered consistent with the teacher’s
marking. However, the correlation coefficient of the experimental group was
.85, which is strong evidence for the value of incorporating peer feedback in
an effort to bring teacher/peer ratings closer together. Another notable result
is that the incorporation of peer feedback prior to peer assessments was that
the tendency to overrate low ability students was reduced in the experimental
group. Patri (2002) concludes that “peers can assess in a manner comparable to
the teacher provided that they have the benefit of peer feedback” (p. 124).
Educators should consider implementing peer feedback in small groups prior to
having their students do peer assessments in order to take advantage of these
benefits and improve the validity of their assessments. However, educators
should resist the urge to have large groups of students provide peer assessment
for the same oral performance, as there are some challenges involved in this
that I will now discuss.
The Challenges of Group
Peer Assessment
A natural
inclination of many teachers faced with the challenge of increasing the
validity of peer assessments is to have many students rate the same
performance. This makes sense, as we would expect the average score to mitigate
the effects of assessors who mark too strictly or too generously. However, the
research tells us the opposite is true. “There is no evidence,” Falchikov and
Goldfinch (2000) report, “to support the superiority of multiple peer ratings
over ratings by singletons. Ratings by very large numbers of peers (20+) appear
to lead to poorer agreement” with teacher ratings (p. 315). Falchikov and
Goldfinch (2000) found that correlation coefficients plummeted as the number of
peers involved in each assessment grew. While the average correlation
coefficient for single assessors was .72, it dropped to .59 for groups of 20 or
more assessors. But how does this make sense? Researchers have studied this
question in recent decades to explain why validity drops as the number of
raters reaches higher numbers.
Latane,
Williams, and Harkins (1979) coined the term ‘social loafing’ to refer to the
phenomenon of the inverse relationship found between group size and individual
performance. They found that individual responsibility, and thus desire to work
hard, diminished as the group size increased and accountability and individual
contributions became harder to identify. Kerr and Bruun (1983) followed up on
this work and found that individual motivation decreased as group size
increased, presumably because individuals felt that their contributions were
becoming less critical for group success. Falchikov and Goldfinch (2000) relate
this to their meta-analysis of peer assessment studies and propose that “it may
be that when students work together in very large groups, some diffusion of
responsibility occurs and marking becomes less thoughtful or careless” (p.
316). Educators should involve peer feedback in the oral assessment process,
but they should refrain from involving too many students in marking the same
assessment.
Student Familiarity with
Assessment Criteria
An obvious
strategy for improving teacher/peer consistency in assessment is to have
clearly defined assessment criteria. If assessors are making with the same
criteria in mind, they are more likely to find agreement between than assessors
who are marking based off their gut feelings or their own interpretation of
what oral competence entails (Falchikov & Goldfinch, 2000). But what role
should students play in the construction of the criteria, and how specific
should their focus be when marking?
Falchikov (2005)
reviewed the value of involving students in the definition of assessment
criteria and finds that the more ownership students have over their learning
process and objectives, the more likely they are to provide consistent, valid
assessment. “Student familiarity with, and ownership of, criteria tends to
enhance peer assessment validity” because it draws assessors into the process
and supports learner autonomy (Falchikov & Goldfinch, 2000, p. 315). Patri
(2002) affirms this and suggests that “more guidance on the marking criteria
should be given to ensure that all markers can apply previously agreed criteria
in a consistent fashion” (p. 110). Falchikov and Goldfinch’s (2000)
meta-analysis shows stronger correlations (r=.85) when students are familiar
with the assessment framework than when students are asked to make a global
judgment without knowledge of the framework (r=.72). Therefore, we have a
compelling case to involve our students in the development of, and make them
familiar with, the marking criteria to support assessment validity. But how
detailed should this criteria be?
Detailed, but
not too detailed, the research tells us. In the same meta-analysis that showed
the value of having assessors be familiar with the marking criteria, Falchikov
and Goldfinch (2000) studied the effects on students marking on a general
criteria versus a checklist of multiple factors. The results were unequivocal: peers
marking on a general criteria had a correlation coefficient of .85 with their
teachers, whereas peers marking several factors individually had a correlation
coefficient of only .53. This dramatic difference tells us that “where
judgements are made in the knowledge of criteria or guidelines … may give rise
to slightly better peer-faculty agreement” than in other dynamics (Falchikov
& Goldfinch, 2000, p. 310). Miller (2003) also researched this effect and
agreed that more individual items in a marking criteria leads to greater
variance, though it can be beneficial in that it can result in students
receiving more detailed feedback. With this in mind, educators should have peer
assessors provide a global score for the performance with a few criteria in
mind, while also providing written feedback on those areas which need attention
and those which are notably strong.
Concluding Remarks
Despite strong
arguments for and against the incorporation of peer assessment in oral language
assessment, the majority of authors argue that in the long run, it has more
beneficial than detrimental effects (Cheng & Warren, 2005; Luc de Grez
& Roozen, 2012). Cheng & Warren remind us that “peer assessment can
work effectively if the teacher is more concerned with the long-term,
cumulative educational benefits rather than simply the immediate success or
failure of students’ attempts to imitate … their teacher” (p. 112). Any
decision regarding the incorporation of peer assessment needs to consider more
than just the immediate question of validity; it needs to look at the enormous
pedagogical value that comes with it.
Students who
practice peer assessment think better and learn for better reasons. They become
less focused on passing a test and more interested in attaining oral
competency. They develop stronger bonds with their classmates and practice
self-regulated learning, becoming more invested in their learning. These
students develop stronger perceptions of self-efficacy by receiving regular,
positive feedback and become more intrinsically motivated. Students develop
more positive attitudes toward their learning situation. They will simply do
better, learn more, and be happier about doing so.
Yes, the
integration of peer assessment into the language classroom is fraught with
challenges and concerns; however, the wealth of strategies provided in this
paper, coupled with the bounty of pedagogical benefits derived from peer
assessment, makes a compelling case for introducing peer assessment of oral
language into our classrooms.
References
Bandura, A. (1993).
Perceived self-efficacy in cognitive development and functioning. Educational
Psychologist, 28, 117-148.
Birdsong,
T. & Sharplin, W. (1986). Peer evaluation enhances students’ critical
judgement. Highway One, 9(1), 23-28.
Cheng, W.
& Warren, M. (2005). Peer assessment of language proficiency. Language Testing, 22(1), 93-121.
Cheng, W.
& Warren, M. (1999). Peer and teacher assessment of the oral and written
tasks of a group project. Assessment
& Evaluation in Higher Education, 24(3), 301-314.
Dornyei, Z. (1994a). Motivation and motivating
in the foreign language classroom. The Modern Language Journal, 78(3),
273-284.
Earl, S.E. (1986). Staff and peer assessment: measuring an
individual’s contribution group performance. Assessment and Evaluation in Higher Education, 11, 60-69.
Entwhistle, N. J. (1987). A model of the teaching
process. In J. Richardson, M. Eysenck & D. Piper (Eds.), Student learning: research in education and
cognitive psychology. Milton Keynes, UK: Open University Press.
Falchikov, N. (2005). Improving assessment through student involvement: practical solutions
for aiding learning in higher and further education. New York:
RoutledgeFalmer.
Falchikov, N. (1995a). Peer feedback marking:
development peer assessment. Innovations
in Education and Training International, 32, 175-187.
Falchikov, N. (1995b). Improving feedback to
and from students. In Knight, P. (Ed.), Assessment
for Learning in Higher Education. London: Kogan Page.
Falchikov, N, & Boud, D. (1989). Student
self-assessment in higher education: a meta-analysis. Review of Educational Research, 59(4), 395-430.
Falchikov, N. & Goldfinch, J. (2000).
Student peer assessment in higher education: a meta-analysis comparing peer and
teacher marks. Review of Educational
Research, 70(3), 287-322.
Falchikov, N, & Magin, D. (1997). Detecting
gender bias in peer marking of students’ group process work. Assessment and Evaluation in Higher
Education, 22(4), 393-404.
Forde, K. 1996: The effects of gender and
proficiency on oral self- and peer- assessments. English Language Studies Working Papers, City University of Hong Kong,
1, 34-47.
Gibbs, G. (1992). Improving the quality of student learning. Bristol: Technical and
Educational Services Ltd.
Jafarpur,
A. (1991). Can naive EFL learners estimate their own proficiency? Evaluation and Research in Education, 5,
145-157.
Jones,
R.L. (1985). Some basic considerations in testing oral proficiency. In Y.P.
Lee, C.Y.Y. Fox, R. Lord & G. Low (Eds.), New Directions in Language Testing. Oxford: Pergamon.
Kerr, N.L. & Bruun, S.E. (1983).
Dispensability of member effort and group motivation losses: free-rider
effects. Journal of Personality and
Social Psychology, 44(1), 78-94.
Latane, B., Williams, K., & Harkins, S.
(1979). Many hands make light work: the causes and consequences of social
loafing. Journal of Personality and
Social Psychology, 37(6), 822-832.
Luc De Grez, M.V. & Roozen, I. (2012). How
effective are self- and peer assessment of oral presentation skills compared
with teachers’ assessments? Active
Learning in Higher Education, 13(2), 129-142.
McGroarty, M.E. & Zhu, W. (1997).
Triangulation in classroom research: a study of peer revision. Language Learning, 47, 1-43.
Miller, P. (2003). The effect of scoring
criteria specificity on peer and self-assessment. Assessment and Evaluation in Higher Education, 28(4), 383-394.
Miller, V.W. & Ng, R. (1994). Peer
assessment of oral language proficiency. Perspectives:
working papers of the department of English, City Polytechnic of Hong Kong,
6, 41-56.
Mitchell, V.W., & Bakewell, C. (1995).
Learning without doing: enhancing oral presentation skills through peer-review.
Management Learning, 26, 353-366.
Mowl, G. & Pain, R. 1995. Using self and
peer assessment to improve students’ essay writing: a case study from
Geography. Innovation in Education and Training International, 32, 324–35.
Newstead, S. & Dennis, I. (1994).
Examiners examined: the reliability of exam marking in psychology. The Psychologist, 7(5), 216-219.
Oldfield, K. & MacAlpine, M. (1995). Peer
and self-assessment at tertiary level – an experiential report. Assessment and evaluation in higher
education, 20, 125-132.
Oxford, R., & Shearin, J. (1994). Language
learning motivation: Expanding the theoretical framework. The Modern
Language Journal, 78(1), 12-28.
Ozogul, G.
& Sullivan, H. (2009). Student performance and attitudes under formative
evaluation by teacher, self and peer evaluations. Educational Technology Research and Development, 57(3), 393-410.
Patri, M.
(2002). The influence of peer feedback on self- and peer-assessment of oral
skills. Language Testing, 19(2), 109-31.
Rigby, C. S., Deci, E. L., Patrick, B. C., & & Ryan, R. M.
(1992). Beyond the intrinsic-extrinsic dichotomy: self-determination in motivation
and learning. Motivation and Emotion, 16(3), 165-185.
Saito, H.
(2008). EFL classroom peer assessment: training effects on rating and
commenting. Language Testing, 25(4),
553-581.
Saito, H.
& Fujita, T. (2004). Characteristics and user acceptance of peer rating in
EFL writing classrooms. Language Teaching
Research, 8, 31-54.
Segers,
M., Dochy, F., & Cascallar, E. (2003). The era of assessment engineering:
changing perspectives on teaching and learning and the role of new modes of
assessment. In M. Segers, F. Dochy & E. Cascallar (Eds.), Optimising new modes of assessment: in
search of qualities and standards. Dordrecht: Kluwer Academic.
Stanley,
J. (1992). Coaching student writers to be effective peer evaluators. Journal of Second Language Writing, 1,
217-233.
Sullivan, K.
&
Hall, C. 1997. Introducing
students to self-assessment. Assessment and Evaluation in Higher Education, 22, 289–305.
Woolhouse, M.
1999.
Peer assessment: the participants’ perception of two activities on a further
education teacher education course. Journal of Further and Higher
Education, 23, 211–19.
Zhu, W.
(1994). Effects of training for peer revision in college freshman composition
classes. Dissertation Abstracts
International, 55, 951.
Zhu, W.
(1995). Effects of training for peer response on students’ comments and
interaction. Written Communication, 12,
492-528.
Subscribe to:
Posts (Atom)





