This article was downloaded by: [Moskow State Univ Bibliote]

On: 03 February 2014, At: 13:21

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41
Mortimer Street, London W1T 3JH, UK

The Journal of Educational Research

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/vjer20

Role of Rubric-Referenced Self-Assessment in Learning to

Write
a b
Heidi Goodrich Andrade & Beth A. Boulay
a
University at Albany
b
Abt Associates, Inc. , Cambridge, Massachusetts
Published online: 02 Apr 2010.

To cite this article: Heidi Goodrich Andrade & Beth A. Boulay (2003) Role of Rubric-Referenced Self-Assessment in Learning to Write,
The Journal of Educational Research, 97:1, 21-30, DOI: 10.1080/00220670309596625

To link to this article: http://dx.doi.org/10.1080/00220670309596625

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) contained in the
publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or
warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed
by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with
primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings,
demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly
in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is
expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 Role of Rubric-Referenced
Self-Assessment in Learning to Write
HEXDX GOODRICH ANDRADE
University at Albany
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
ABSTRACT The authors examined the impact of self-
assessment on 7th- and 8th-grade students written essays.
Students wrote 2 essays: historical fiction essay and response
to literature essay. All students received instructional rubrics
that articulated the criteria and gradations of quality for the
given essay. Students in the treatment group participated in 2
formal self-assessment lessons, during which they used the
rubric to assess the quality of their drafts. Authors used mul-
tiple linear regression to examine the relationship between
essay scores, treatment, and a set of control predictors. The
results from the historical fiction essay suggested a positive
relationship between the treatment and girls scores, but no
statistically significant relationship between the treatment and
boys scores. The results from the response to literature essay
showed no effect of treatment for either boys or girls. The
results are explained in terms of the insufficiency of the inter-
vention, as well as the possible effects of rubrics, school condi-
tions, and gender differences in response to self-generated
feedback.
Key words: assessment of writing, rubric-referenced, self-
regulated learning
Most students do not revise because they have not learned
how to evaluate what they write; they have not internalized
any consistent set of criteria or standards to which they can
hold themselves. (White, 1994, p. 10)
S tudent self-assessment is the Cinderella of the authen-
tic or alternative assessment movement. Whereas
teachers and researchers alike celebrate the marriage of
instruction and evaluation and the family of assessments
that result, student self-assessment stands apart as the poor
stepchild-poorly understood and assumed to be difficult,
impractical, or unnecessary. Yet, an examination of assump-
tions yields rich potentials. Rather than thinking of self-
assessment as simply a matter of asking students to give
themselves a grade, we hypothesized that self-assessment
could support learning and skill development via a process
of careful reflection on the quality of studentsown work. In
the Student Self-Assessment Study, we tested that hypothe-
sis by examining the effects of rubric-referenced self-
21
BETH A. BOULAY
Abt Associates, Inc., Cambridge, Massachusetts
assessment on seventh- and eighth-grade students writing.
Theoretical Framework
This study draws on several areas of educational and cog-
nitive research, including authentic assessment, self-regu-
lated learning, and the assessment of writing. Current per-
spectives on authentic or alternative assessment, as well as
on the assessment of writing, distinguish between evalua-
tion and assessment and promote a conception of assess-
ment as ongoing feedback that supports learning (Gardner,
1991; Goodrich, 1997; White, 1994; Wiggins, 1989a.
1989b; Wolf & Pistone, 1991). Cooper and Odell (1999),
for example, stressed the need to provide students with time
to reflect on their writing so that some sort of assessment
of strengths and weaknesses in a piece of writing occurs
before a final draft is written (p. x). The literature on
authentic assessment also provides guidance on the charac-
teristics of effective assessment (see Goodrich, 1996, for a
review). Those characteristics influenced the design of this
study, which,
articulated clear criteria for assessing writing,
supported students in assessing their own works in
progress,
identified techniques and methods for improving writing,
provided opportunities for improvement through revi-
sion, and
honored students developmental stages by referring to
appropriate grade-level standards.
The design of this study also reflects the literature on
self-monitoring and writing, which notes that novice writers
have difficulty identifying the problems with their writing
as well as techniques for improving it (Scardamalia & Bere-
iter, 1985) and that the use of on-line or hard-copy revision
Address correspondence to Heidi Andrade, University at Al-
bany, 1400 WashingtonAvenue, ED233A, Albany, NY 12222. (E-
mail: handrade @ uamail.albany.edu)
 22
checklists can improve students writing (Daiute, 1986;
Scardamalia & Bereiter, 1983). Similarly, research on self-
regulated learning and feedback suggests that learning
improves when feedback (a) directs students to monitor
their learning and (b) shows them how to achieve learning
objectives (Bangert-Drowns, Kulik, Kulik, & Morgan,
1991; Butler & Winne, 1995).The Student Self-Assessment
Study was based on the hypothesis that students can be their
own source of feedback, given the appropriate conditions
and supports. For the purposes of this study, the uppropri-
are supporrs were instructional rubrics that described good
and poor writing, and the appropriate conditions were two
self-assessment lessons that we designed to help students
use a rubric to assess their draft essays.
Instructional rubrics refer to those rubrics designed to
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
support student learning and development and to act as
standards-referenced assessment tools (Goodrich Andrade,
2000). Instructional rubrics included the following features
that supported learning: They (a) were written in language
that students could understand, (b) defined and described
quality work in as concrete terms as possible, (c) referred to
common weaknesses in students work and indicated how
such weaknesses could be avoided, and (d) could be used by
students to evaluate their works in progress and thereby
guide revision and improvement.
The historical fiction rubric in Appendix A is an example
of an instructional rubric designed for use in this research.
Like both of the rubrics that we used, the historical fiction
rubric drew on district, state, and national standards, as well
as on feedback from colleagues and teachers. The historical
fiction rubric articulated the criteria for the assignment, de-
scribed gradations of quality from excellent to poor, and pro-
vided suggestions for avoiding common pitfalls in writing.
Taken together, the research on authentic assessment,
assessment of writing, and self-regulated learning indicated
the potential for rubric-referenced self-assessment to sup-
port learning and skill development. Accordingly, the
research question that drove this study was: Can a formal
process of rubric-referenced self-assessment have measur-
able effect on student writing?
Method
Sample
This project was supported by the Edna McConnell Clark
Foundation, which asked that we implement the work in
schools with which the foundation collaborates. As a result,
we conducted the research in two middle schools in a city
in southern California. One of the schools (School A) was
located in an upper-middle-class,largely professional, sub-
urban neighborhood with little ethnic diversity. Many of the
non-White students who attended School A were bused in
from adjacent communities and tended to be placed in
lower level classes. The language arts teachers with whom
we worked in School A designed their cumcula indepen-
The Journal of Educational Research
dently of each other. School B, in contrast, was located in
an ethnically and linguistically diverse working-class urban
community. The teachers with whom we worked at School B
collaborated on an integrated curriculum that combined his-
tory and language arts. Their shared humanities curriculum
drew explicitly on the districts standards and an experi-
mental new portfolio process. The study took place during
the 1997-98 school year and involved 13 seventh- and
eighth-grade classes in the two schools.
The sample included 397 students: 25 1 students (63.2%)
attended School A; 146 students (36.8%) attended School
B. There were 183 (46.1%) seventh graders and 214
(53.9%) eighth graders. One hundred ninety-one of the stu-
dents were boys (48.1%); 201 of the students were girls
(50.6%). Five students (.01%) from the sample were not
identified by gender. None of those students were included
in the subsample that we analyzed. We also were unable to
obtain ethnicity data for 103 of the students (26% of the
sample). However, of the remaining 294 students, 126
(42.9%) were White, 68 (23.2%) were Black, 5 (1.7%) were
American Indian, 33 (11.2%) were Filipino, 46 (15.6%)
were Hispanic, and 16 (5.4%) were students of other ethnic-
ities. The average Abbreviated Stanford Achievement Test
(ASAT) score for the sample was 2.19 (SD= 1.80), and the
average EnglisManguage arts grade for the term prior to
this study was 80.53 (SD= 11.43).
Procedure
Students were assigned by classroom to either the self-
assessment condition or the control condition. Equal num-
bers of seventh- and eighth-grade classrooms were in each
group, and the groups were counterbalanced according to
scores on the ASAT. In those cases in which we could not
match the classrooms exactly according to ASAT scores,
the control group was favored such that they would have
higher ASAT scores. That procedure allowed us to obtain a
more conservative estimate of the effect of the self-assess-
ment intervention.
Students in each class were asked to write two different
essays approximately 1 month apart: a historical fiction
essay, and a response to literature essay (see Appendix A for
the historical fiction essay assignment and rubric, and
Appendix B for the response to literature assignment and
rubric). Students in the treatment and control classes were
given identical instructional rubrics with the assignment of
each essay, and their teachers briefly reviewed the assign-
ment and the rubric.
After students wrote a first draft of their essays (at least
in theory-not all students wrote first drafts on schedule),
the first author conducted two 40-min self-assessment
lessons with the students in the treatment condition. The
lessons involved students in a formal process of guided self-
assessment designed in collaboration with a participating
teacher at School B. Students used markers to color code
the criteria on the rubric and the evidence in their essays
 SeptemberlOctober2003 [Vol. W(No. l)]
that demonstrated that their writing met each criterion. The
first lesson guided students use of the rubric to evaluate
their drafts in terms of the three most global criteria-ideas
and content, organization, and paragraphs. For example, the
historical fiction rubric included a criterion that required
students to bring the time and place in which the character
lived alive. During class, students were asked to underline
time and place in blue on their rubrics, then underline the
information that they provided about the time and place of
their story in blue on their essay. If they could not find the
information in their essay-and they were often surprised to
discover that they could not-they were instructed to write
(at the tops of their papers) a reminder to add the missing
information when they wrote a second draft. The treatment
students were then ask.ed to write a new draft and bring it to
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
class for the second self-assessment lesson.
The second lesson followed the same process of guiding
students use of the second half of the rubric to look at the
four finer grained criteria-voice and tone, word choice,
sentence fluency, and conventions. The students were then
asked to revise their essays without our further intervention.
Control classes received copies of the rubrics and were
asked to write at least two drafts, but they did not formally
assess their own work in class.
Dependen t Measures
Students revised drafts were collected and scored. Each
essay was scored by at least one research assistant, accord-
ing to an adapted version of the rubric used in the class-
room. The scoring rubrics consisted of a 6-point ordinal
scale in which 0 was the lowest score, and 5 the highest
score. A research assistant averaged the scores on each cri-
terion to create an overall score for each essay.
The second author and a research assistant were trained
by the first author to score the essays using the scoring
rubric. For each essay, the scorers underwent a training
period during which they scored several essays in conjunc-
tion with discussions with the first author. The scorers then
rated several more essays independently. The scores were
compared to assure that scoring commenced with a high
level of interrater reliability.
We used Pearson correlations and t tests in combination
to determine the extent to which the raters were in agree-
ment. That method provided a more accurate assessment of
interrater reliability than did correlation alone because it
tested the relationshrp as well as the mean difference
between the scores of the two raters.
The scorers rated 10 historical fiction essays independent-
ly at the beginning of scoring and 5 historical fiction essays
during scoring to guard against drift. The overall (N= 15)
Pearson correlation coefficient was extremely high ( r = .92),
and a t test showed no significant differences between scores
(p = .71). In addition to those measures of overall reliability,
we judged the extent LO which the raters agreed on each of
the criteria used to assess the essays. The Pearson correlation
23
coefficients for the 10 criteria ranged from .60-.95, and all
but one of the t tests showed no significant differences
between the ratings. For the one criterion for which the score
differed (Al, historical content), each time the scorers dis-
agreed (4 times out of 15), Scorer 2 rated the essay 1 point
higher than did Scorer 1. Because the disagreement was
always in the same direction, the somewhat small discrepan-
cy reached statistical significance.
The scorers rated 12 response to literature essays inde-
pendently; they also rated 4 additional response to literature
essays during scoring to guard against drift. The two sets of
scores were highly correlated ( r = .79), and a t test showed
no significant differences between scores (p = .83). We
again assessed interrater reliability on the 10 individual cri-
teria. Nine of the 10 correlations were high (S9-.91), but
one was only .3. Again, all but one of the t tests (word
choice) were nonsignificant, indicating that there was high
agreement between the scorers. However, the dependent
measure used in the analyses was the average score on each
essay. We assumed that that average was resilient to the
minor discrepancies between the raters on the two criteria
noted in the previous paragraph.
independent Measures
Treatment group. We used an indicator variable to denote
whether students participated in the self-assessment lesson
(treatment). That variable served as our primary question
variable, and allowed us to test whether membership in the
group that engaged in self-assessment was related to higher
essay scores.
Previous performance in English. We expected that stu-
dents who performed well in English in the past would
write better essays than would those who had not done as
well in English, regardless of whether they participated in
the self-assessment lessons. We included two measures of
past performance in English to be used as control variables.
Each students English grade from the previous semester
was obtained from the schools records (grade). Those
grades ranged from 95 (A, highest) to 55 (F, lowest). We
also obtained students scores on the ASAT from the school
records. The scores represented an average of the language
mechanics and language expression components of the test.
School- and grade-level descriptors. To test whether the
effect of the intervention was different across the two
schools, we included an indicator variable in our analysis to
index the school attended (school). We also indexed the
grade level of each student (Grade 7 or 8) to test for grade-
level effects (level).
Student descriptors. We obtained four descriptive vari-
ables for each student from school records to use as control
variables: gender, ethnicity, participation in special educa-
tion, and participation in ESL (English as a second lan-
guage) programs. The ethnicity measure contained eight
categories (White, Black, Asian or Pacific Islander, Native
American or Alaskan, Portuguese, Filipino, Hispanic, and
 24 The Journal of Educational Research

Other). Because the numbers of children were small in sev-
eral of the categories, we collapsed the variable into one
categorical variable, ethnicity, to represent membership in
any minority group.
Results
Descriptive Statistics
We completed scoring the historical fiction essay for a
subsample of 119 students (30% of the complete sample).
Chi-square tests and t tests revealed no significant differ-
ences between the students included in the analysis and
those not included in the analysis on any of the descriptive
measures, indicating that the subsample did not differ from
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
the overall sample of students. Fifty-one students in the sub-
sample (42.9%) attended School A; 68 students (57.1%)
attended School B. Across both schools, there were 59 stu-
dents (49.6%) in the treatment condition, and 60 students
(50.4%) in the control condition. Sixty-two students
(52.1%) were in eighth grade; 57 students (47.9%) were in
seventh grade.
Scores for the response to literature essay were complet-
ed for a subsample of 98 students (24.7% of the complete
sample). Because of implementation difficulties at School
B, all of the students in the subsample of those writing the
response to literature essay attended School A. There were
48 students (49.0%) in the treatment condition, and 50 stu-
dents (5 1.O%) in the control condition. Half of the students
were in eighth grade; the other half were in seventh grade.

TABLE 1. Frequencies and Means for Descriptive Variables, by Treatment Condition:

Historical Fiction Essay (N= 119)

Treatment Control
(n = 59) (n = 60)
Variable n % n %

School A 23 39.0 28 46.7

Gender (female) 33 55.9 32 53.3
Level (Grade 8) 36 61.O 26 43.3
Ethnicity 25 (of 39) 64.1 22 (of 34) 64.7
English as a second language 6 10.2 9 15.0
Special education 3 5.1 5 8.3

M SD M SD

Abbreviated Stanford
Achievement Test score 4.73 1.79 5.47 1.61
Grade 80.45 11.99 80.0 11.12

TABLE 2. Frequencies and Means for Descriptive Variables by Treatment Condition,

Response to Literature Essay, and School A (N= 98)
I
Treatment Control
( n = 48) ( n = 50)
Variable n % n %

Gender (female) 20 41.7 25 50

Level (Grade 8) 23 47.9 26 52
Ethnicity 19 (of 43) 44.2 10 (of 41) 24.4
English as a second language 0 0 2 4
Special education 1 2.1 5 10
I
I M SD M SD

Abbreviated Stanford
Achievement Test Score 5.95 1.94 5.81 1.65
Grade 85.0 9.56 84.0 11.04
 SeptemberlOctober 2003 [Vol. 97(No. l)] 25

We summarized the descriptive statistics for each sub-
sample by treatment condition (see Tables 1 and 2). We
assessed the equivalence of the treatment and control groups
on each of the descriptive variables by using chi-square
tests for categorical variables and t tests for continuous
variables.
In the subsample of students who completed the histori-
cal fiction essay, significant differences existed between the
groups with respect to level (more eighth graders than sev-
enth graders in the treatment group) and ASAT scores (con-
trol group earned higher ASAT scores than did treatment
group). The treatment and control groups of students who
completed the response to literature essay were significant-
ly different with respect to the number of minority students
(more minority students than nonminorities in the treatment
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
tionships between the predictors and the scores on the
response to literature essay.
The first column of Table 3 shows the bivariate correla-
tions between student scores on the historical fiction essay
(HF score) and the predictors. Our question predictor (treat-
ment) had essentially no correlation with HF scores, which
suggested that there was no effect of treatment on essay
scores without controlling for the effects of other variables.
ASAT scores, grade, and school attended (school) were all
moderately correlated with essay scores. ASAT also was
correlated with treatment, indicating that the relationship
between treatment condition and average essay score may
have changed after we controlled for ASAT. In addition,
ethnicity and gender were somewhat correlated with essay
scores and correlated with each other, suggesting that

group). Therefore, we included those variables as control
variables in all subsequent multivariate analyses.
Correlations
Table 3 shows the correlation matrices that we construct-
ed to explore the relationships between the predictors and
scores on the historical fiction essay, as well as among the
predictors. In Table 4, we summarized the analogous rela-
although both variables could be important control vari-
ables, after controlling for one, the other might not explain
much additional variation.
The first column in Table 4 shows the bivariate correlations
between student scores on the response to literature essay
(RtoL score) and the predictors. Treatment had little or no
correlation with RtoL score, whereas ASAT scores and
grades were moderately correlated with RtoL scores. As with
the historical fiction essay, those correlations suggest that

TARLE 3. Correlations Between Historical Fiction Essay Scores, Treatment, Previous Performance in
English, and Demographic Predictors

Variable HF score Treatment ASAT Grade School Level Ethnicity

HF score 1.o
Treatment -.02 1.o
ASAT .52*** .22* 1.o
Grade .54*** ,004 .45*** 1.0
Schtml .38*** -.08 .40*** .25** 1.o
Level .11 .I8 -.17 .26** -.I2 1 .o
Ethnicity -.40*** -.01 -.34** -.41*** -.70*** -.07 1 .o
Gender .18* .03 -. 10 ,003 -.20* .ll .30**

Nore. N = 119. ASAT =Abbreviated Stanford Achievement Test.

* p < .05. **p < .01. ***p < ,001.

TABLE 4. Correlations Between Scores on the Response to Literature Essay (RtoL), 'Ikatment,
Previous Performance in English, and Demographic Predictors
-
Variable RtoL score Treatment ASAT Grade Level Ethnicity
-
Rtol. score I.o
Treatment .02 1 .o
ASAT .31** .04 1 .o
Grade .42*** .05 .38*** 1 .o
Level -.01 -.04 -.I4 .40*** 1.o
Ethnicity -.I9 .21 -.30** -.19 -.09 1.o
Gentler .01 -.09 -.I0 .I3 -.03 .28**

Nore. N = 98. ASAT =Abbreviated Stanford Achievement Test.

* p < 05. **p<.Ol. ***p<.OOl.
 26 The Journal of Educational Research

there was no effect of treatment group on essay scores with-
out our having controlled for the effects of other variables.
Multiple Linear Regression Models
Tables 5 and 6 show separate taxonomies of multiple lin-
ear regression models fit to assess the relationship between
the scores on each of the essays and the question predictors,
and the high-priority control variables and their interactions.
Treatment. As expected, Model 1 in Table 5 shows that
participation in the self-assessment lessons was not related
to scores on the historical fiction essay. That finding indi-
cated that no treatment effect existed without controlling for
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
other variables in the model. Model 1 in Table 6 reveals the
same finding for the response to literature essay scores. We
expected that finding because of the low bivariate correla-
tion between treatment and RtoL score.
Previous pe$onnance in English. Model 2 in Table 5
shows that there was a significant main effect of ASAT
scores and grades on student scores on the historical fiction
essay. That is, students who had higher grades and those
who had higher scores on the ASAT than did lower achiev-
ing students tended to have higher essay scores, controlling
for whether they were in the treatment group. A similar
effect was seen on the response to literature essay (Model 2,
Table 6). Again, students with higher grades tended to score

TABLE 5. Multiple Linear Regression Models of the Relationship Between Historical Fiction Essay
Scores and k t m e n t

Model
Variable I 2 3 4 5 6

Intercept 3.16*** .61 .62 .70 .76 .83*

Treatment -.03 .I0 -.I6 .27 .04 -.I4
ASAT .15*** .11* .14*** .15*** .15***
Grade .02*** .02** .02*** .02*** .02***
TX*ASAT .06
TX*Grades -002
School .39* .33* .28
TX*school -.43 -.37 -.28
LRvel
TX*level
Gender -.22* -.a** -.47**
Tx* .36 .44*
Gender

Model R Z ,0596 39.37% 43.50% 45.57% 47.14% 45.29%

Note. N = 119. ASAT = Abbreviated Stanford Achievement Test. In the six models, previous performance was controlled for in
English, school. grade level, gender, and interactions with the question predictor.
*p < .05. **p < .01. ***p < .001.

TABLE 6. Multiple Linear Regression Models of the Relationship Between Response to Literature
Essay Scores and k t m e n t

Model
Variable 1 2 3 4 5'

Intercept 3.145"' 1.10' 1.35 1.41 1.98'

Treatment .027 .06 .06 .05 .004
ASAT .048 .044 .041 .03
Grade .02*** .02" .02" .02"'
Level -.04 -.05 -.11
Gender -.04 -.06
Ethnicity -.001
Model R2 .06% 22.73% 22.84% 22.98% 26.21%

Nore. N = 98. ASAT = Abbreviated Stanford Achievement Test. In the six models. previous performance was controlled for in
English, grade level, gender, ethnicity. and interactions.
T h i s model conlains 70 participants.
*p < .05. * p < .01. ***, < .001.
I
 SeptemberlOctober2003 [Vol. W(No. l)]
higher on their essays across treatment and control groups
than did students with lower grades. However, on the RtoL
essay, ASAT was not ii significant predictor of essay scores.
(Although not shown in Table 6, we ran models with each
of the measures of English performance separately. Without
grades in the model, there also was a significant main effect
of ASAT. As seen in Table 5 , ASAT and grades were corre-
lated and collinear; therefore, we kept them both in the
model throughout the model building process.) There were
no significant interactions between previous performance in
English and treatment in either analysis. That finding indi-
cates that the effect of the treatment does not differ as a
function of previous performance in English for either type
of essay.
School, level, gendel; and ethnicity. We investigated
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
whether there were any main effects of school or of grade
level on essay scores, or if the effect of the treatment dif-
fered by either of these variables (statistical interaction).
When we examined student scores on the historical fiction
essay, the treatment-by-school interaction approached sta-
tistical significance after we controlled for previous perfor-
mance in English (Model 3, Table 5). That is, when we con-
trolled for past performance in English, the effect of
participation in the self-assessment lesson on historical fic-
tion essay scores differed across the two schools. However,
as Model 5 reveals, that effect was no longer significant
when the interaction of treatment and gender was added to
the model. The interaction shown in Model 5 indicates that
the effect of the treatment condition differed for boys and
girls. No additional variation in essay scores was explained
by any of the remaining variables or interactions.Therefore,
4.51
21
2 3 4 5
ASAT Scores
FIGURE 1. Plot of multiple linear regression model using treatment, Abbrwiated
Stanford Achievement Test scores, English grades, and gender to predict scores on
historical fiction essay. English grade is set to the sample mean (N= 119).
-
27
we refit Model 5 to remove any predictors that did not meet
statistical significance (p < .05)-this appears as Model 6.
Analogous models were fit to investigate the effects of
the variables on response to literature essay scores (Models
3,4, and 5 in Table 6). The models indicated that after con-
trolling for treatment, ASAT scores, and grades, there were
no main effects of grade level, gender, or ethnicity (all stu-
dents who completed this essay attended School A). We
also tested for interaction effects between each of the pre-
dictors and treatment. None of the interaction terms were
significant, which indicated that the treatment effect did not
differ across any of the subgroups.
Overall Findings
The results from the historical fiction analyses revealed
some interesting relationshps. Figure 1 shows a series of
prototypical plots of the effects revealed by Model 6. Essay
scores (Y-axis) are plotted as a function of ASAT scores (X-
axis). The black lines refer to the treatment condition, and the
gray lines refer to the control condition. Solid lines refer to the
effects for boys, and dotted lines represent the effects for girls.
All four lines have an upward slope, which indicates that,
as expected, students with higher ASAT scores tended to
earn higher essay scores than did students with lower ASAT
scores. Also, there was a positive main effect of gender,
such that boys consistently scored higher than did girls on
the historical fiction essay-the solid lines are always
shown above the dotted lines. In addition, Model 6 reveals
that although there was an effect of treatment condition, it
differed for boys and girls (p < .0389). That is, we found a
-
--- treatmen-
treatmentJglrla
-
---
controlbys
COnWglrls
6 7 8 9

positive effect of treatment condition for girls; those in the
treatment group scored .31 points higher, on average, on the
historical fiction essay than did girls in the control group
(black dotted line is above the gray dotted line). That dif-
ference was statistically significant (p < .0395). However,
the effect was reversed for boys-the gray solid line is
above the black solid line. The boys in the control group
scored .14 points higher, on average, than did boys in the
treatment group. That effect, however, did not attain statis-
tical significance (p < .3739).
Boys and girls in the treatment groups did not differ from
each other-note the distance between the two black lines
(p < 3364). It appears that without treatment, on average,
girls tended to receive lower scores than did boys (on aver-
age, .47 points lower; p < .0018). With treatment, however,
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
girls tended to receive scores similar to those received by
the boys in the treatment group.
In contrast, the results from the response to literature
essay analyses suggest that there was no effect of treatment.
The only significant main effect was the predictable effect
of grades. We ran a power analysis to determine whether we
had a large enough sample to measure even small between-
group differences. The results showed that when the sample
size was 98, the multiple linear regression test of R2 for six
normally distributed covariates would have a 95% power to
detect an R2 of .1857. That effect size was comparable to
those that we observed in previous essays. Therefore, we
were confident that we had the statistical power to detect an
effect of the self-assessment treatment if one existed.
In summary, for the first essay (historical fiction), partici-
pation in the self-assessment lessons tended to have a posi-
tive relationship with girls scores, but no statistically signif-
icant relation to boys scores. For the second essay (response
to literature), the self-assessment lessons had no relationship
to either girls or boys scores.
Discussion
The results of this study indicate that the self-assessment
intervention was insufficient to obtain a consistent, measur-
able effect. Teachers may have to do more than provide a
rubric and facilitate two self-assessment sessions to deter-
mine meaningful improvements in students essays. Guid-
ance from a teacher in solving the problems identified by
students during self-assessment lessons could help the stu-
dents, given the well-established difficulties that they have
with revision (e.g., Cooper & Odell, 1999). Other promis-
ing additions to the intervention include teachers (a)
extending the treatment time; (b) clarifying students under-
standing of the rubric, perhaps by co-creating it with them
and by making it more central to each writing assignment;
and (c) providing feedback about the accuracy of students
self-assessments and the effectiveness of their revisions.
We also have considered other possible influences on our
results, including the Iikelihood of a positive effect of the
rubrics on the writing done by the students in the control
The Journal of Educational Research
group and the effects of school conditions, essay topics, and
gender differences in response to feedback.
Egects of Rubrics on Students Writing
Andrade (200 1) has suggested that instructional rubrics
alone are sometimes-but not always-associated with an
increase in the scores assigned to middle school students
writing. In the current study, we investigated the possibility
that focused self-assessment could enhance the effect of
rubrics by helping students use them to critically evaluate
their own work. It is possible that the process of self-assess-
ment (treatment condition) contributed too little to the final
essay scores, above and beyond the contribution of the
rubrics (control condition), to reliably measure. That is, the
rubric may have helped students in each group to write
more effectively by advising them of what to expect in the
essays, and the self-assessment lessons may have had little
additional effect. Further research is needed to determine
whether boosting the self-assessment by extending treat-
ment time, co-creating rubrics with students, and providing
feedback to students about their self-assessments and their
revisions makes a difference in students writing.
School Conditions
We discounted the results from the response to literature
essay treatment because of the difficulties that we experi-
enced when we implemented that portion of the study. The
response to literature essay assignment was given at the end
of April, presumably well before the usual end-of-the-year
chaos that is experienced by most schools. However, the
1997-98 school year in which this study took place was dif-
ferent. Administrators in the district in which we were work-
ing had issued new testing requirements in the schools that
made doing much else nearly impossible. We did not use the
data from School B because the study could not be carried
out appropriately in that school, despite the valiant efforts of
several teachers. The data from School A was scored, but it
was apparent that the essays had not received the same
amount of attention from students that they had in the past.
One teacher from School A summarized the situation best:
The last few months of the year brought the normal stan-
dardized testing, but there was a twist this past year. Litera-
cy portfolios turned out to be a HUGE deal with administra-
tion. Basically, we were pressured to make [School A] look
good as compared to the other middle schools in the city. I
calculated that we put 11 instructional days exclusively into
these folders over the course of the year. In addition, eighth
graders had to do an exit exhibition and we had to rehearse
them as well as sit on some panels. Teaching in some
respects took a back seat to satisfying the needs of the edu-
cational machine.
The irony that two assessment initiatives prevented us from
properly carrying out our research on self-assessment has
not escaped us. Regrettably, the data from the response to
literature essay was suspect. We included it here only to
 SeptembedOctober 2003 [Vol. W(No. l)]
make the obvious point that self-assessment, although
potentially beneficial. may not be sufficiently powerful to
counteract other negative influences on students writing.
Essay Topic
We speculated that some portion of the inconsistencies in
the findings for the two essays might be explained by the
simple fact that the students were asked to write two differ-
ent kinds of essays. Although the various demands of the
two essays almost certainly had an effect on students writ-
ing, one should remember that six of the seven criteria con-
tained in the rubrics for the historical fiction and response to
literature essays were either identical or nearly identical. We
believe that it is unlikely that the remaining criterion (ideas
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
and content) would have been enough to skew our results.
Gender Differences
The analysis of essay scores for the historical fiction
essay indicates that self-assessment has a positive relation-
ship with girls writing, but little or no relationship with
boys writing. We turned to the research on student respons-
es to feedback to understand the gender difference. In a
broad stroke, our finding is consistent with research on sex
differences in response to feedback and in achievement
motivation and learned helplessness. That body of work has
shown generally that girls and boys differ in their attribu-
tions of success and failure and in their response to evalua-
tive feedback (Dweck & Bush, 1976; Dweck, Davidson,
Nelson, & Enna, 1978). Briefly, research by Dweck and
others (Deci & Ryan, 1980; Hollander & Marcia, 1970) has
shown that girls are more likely than are boys to be extrin-
sically motivated and to attribute failure to ability rather
than to motivation, effort, or the agent of evaluation. As a
result of those attributions, girls performance following
negative adult feedback tends to deteriorate more than does
boys performance.
Furthermore, research by Dweck and her colleagues has
demonstrated that boys and girls tend to attribute failure to
ability when negative adult feedback is solution specific. A
hypothesis based on those findings would predict detri-
ments in boys and girls performance in the present study
because rubric-referenced self-assessment is solution spe-
cific and likely to uncover problems with students work.
That is, because the rubrics set high standards of quality
and addressed multiple components of the writing task,
self-assessment was likely to emphasize at least one weak-
ness in a students first draft of an essay. In addition, the
rubrics never referred to academically irrelevant criteria
such as neatness, timeliness, or effort. Although we did not
collect data on students actual assessments of their essays,
the first authors observations in classrooms during the
self-assessment lessons revealed that students typically
found one or more problems with their essays. On the basis
of those observations, we conjectured that the students
29
self-assessments were often, if not always, at least partial-
ly negative and completely solution specific, which would
predict a negative impact on boys and girls performance.
Such a prediction would have been incorrect. We further
speculated that students might tend to respond differently
to self-generated negative, solution-specific feedback than
to feedback from adults.
The findings from the Student Self-AssessmentStudy are
consistent with findings from an earlier study conducted by
the first author (Goodrich, 1996), which showed that rubric-
referenced self-assessment has a positive relationship with
girls metacognitive processing, but a negative relationship
with boys metacognitive processing. In combination, the
earlier study and the Student Self-Assessment Study sug-
gest that self-generatedfeedback may have a different effect
on girls performance than does negative adult feedback.
Some interesting contradictions in the research literature
indicate that that finding may not be peculiar to our
research. Roberts and Nolen-Hoeksema (1989) found no
evidence that evaluative feedback leads to performance
decrements in women, suggesting that womens maladap-
tive response to feedback is not absolute. Also, Bronfen-
brenner (1967, 1970) reported that when peers, instead of
adults, delivered deficient feedback, the pattern of attribu-
tion and response reversed: Boys attributed the failure to a
lack of ability and showed impaired problem solving,
whereas girls more often viewed the peer feedback as
indicative of effort and showed improved performance.
Noting that the more traditional finding of greater help-
lessness among girls was evident only when the evaluators
were adults, Dweck and colleagues (1978) reported that
boys and girls have not learned one meaning for failure and
one response to it. Rather, they have learned to interpret and
respond differently to feedback from different agents (p.
269). That conclusion seems to be reasonable and relevant
to the gender differences found in this study. Although our
research was not designed to permit us to examine students
attributions of success or failure, we believe that the results,
together with earlier research, raise the possibility that girls
responses to self-generated feedback tend to be construc-
tive, whereas boys responses tend to be either neutral or
possibly even negative. Clearly, the cognitive and emotion-
al mechanisms associated with self-assessment deserve a
closer look if they are to serve all students equally well.
Conclusion
Perhaps not surprisingly, we believe that there is reason
to hold out hope for self-assessment in spite of the less-
than-promising results of the Student Self-Assessment
Study. One reason is that the differences in scores between
the treatment group and the control group tended to favor
the treatment group, even when those differences were too
small to attain statistical significance. A more extensive
intervention is needed. We intend to create and test more
supportive conditions for self-assessment by extending the

treatment time, co-creating rubrics with students, and facil-
itating revision. By so doing, we believe that we can elevate
student self-assessment above its poor stepchild status and,
more important, help create effective writing programs and
reflective writers.
NOTES
We thank Stacey Koprince for her help in scoring hundreds of essays.
We also are grateful to the Edna McConnell Clark Foundation for the
financial support for this study. The opinions expressed in this article do
not necessarily reflect those of the foundation.
This study was conducted while the first author was a principle investi-
gator at Project Zero, Harvard Graduate School of Education.
REFERENCES
Downloaded by [Moskow State Univ Bibliote] at 13:21 03 February 2014
Andrade, H. G. (2001,April 18). The effects of instructional rubrics on
learning to write. Current Issues in Education [On-line}, 4(4). Available:
http://cie.ed.asu.edu/volume4/number4
Bangert-Drowns, R., Kulik, C., Kulik, J., & Morgan, M. (1991).The
instructional effect of feedback in test-like events. Review of Education-
al Research, 61,213-238.
Bronfenbrenner, U. (1967).Response to pressure from peers versus adults
among Soviet and American school children. International Journal of
PSychology, 2, 199-207.
Bronfenbrenner, U. (1970).Reactions to social pressure from adults versus
peers among Soviet day school and boarding school pupils in the per-
spective of an American sample. Journal of Personality and Social Psy-
chology, 15, 179-189.
Butler, D.. & Winne, P. (1995). Feedback and self-regulated learning: A
theoretical synthesis. Review of Educational Research. 65(3),245-281.
Cooper, C.. & Odell. L. (1999). Evaluating writing: The role of teachers
knowledge about text, learning and culture. National Council of Teach-
ers of English: Urbana, IL.
Daiute, C. (1986).Physical and cognitive factors in revising: Insights from
studies with computers.Research in the Teaching of English, 20. 141-159.
h i , E.,& Ryan, R. (1980).The empirical exploration of intrinsic moti-
The Journal of Educational Research
vational processes. In L. Berkowitz (Ed.), Advances in experimental
social psychology. New York: Academic Press.
Dweck, C., & Bush, E. (1976).Sex differences in learned helplessness: 1.
Differential debilitation with peer and adult evaluators. Developmental
P~ChOlOgy,12, 147-156.
Dweck C., Davidson, W., Nelson, S.,& Enna, B. (1978).Sex differences
in learned helplessness: 11. Contingencies of evaluative feedback in the
classroom and 111. An experimental analysis. Developmental Psycholo-
gy, f4(3), 268-276.
Gardner, H. (1991).Assessment in context: The alternative to standardized
testing. In B. R. Gifford & M. C. OConnor (Eds.), Changing assess-
ments: Alternative views of aptitude, achievement and instruction.
Boston: Kluwer.
Goodrich, H. (1996). Student self-assessment: At the intersection of
metacognition and authentic assessment. Doctoral dissertation. Cam-
bridge, MA: Harvard University.
Goodrich, H. (1997). Understanding rubrics. Educarional Leadership,
54(4), 14-17.
Goodrich Andrade, H. (2000).Using rubrics to promote thinking and
learning. Educarionul Leadership, 57(5), 13-18.
Hollander, E., & Marcia, J. (1970).Parental determinants of peer orienta-
tion and self-orientation among preadolescents. Developmentul Psy-
chology, 2,292-302.
Roberts, T., & Nolen-Hoeksema, S . (1989).Sex differences in reactions to
evaluative feedback. Sex Roles, 21(11/12),725-746.
Scardamalia, M., & Bereiter, C. (1983).The development of evaluative,
diagnostic and remedial capabilities in childrens composing. In M,
Martlew (Ed.), The psychology of written language: A developmental
approach. London: Wiley.
Scardamalia, M., & Bereiter, C. (1985).Fostering the development of self-
regulation in childrens knowledge processing. In S. F. Chipman, J. W.
Segal, & R. Glaser (Eds.), Thinking and learning skills, Vol. 2: Research
and open questions (pp. 563-577). Hillsdale, NJ: Erlbaum.
White, E. (1994). Teaching and assessing writing: Recent advances in
understanding, evaluating, and improving student performance (2nd
ed.).Portland, M E Calendar Islands.
Wiggins, G. (1989a).A true test: Toward more authentic and equitable
assessment. Phi Delta Kappan, 70(9), 703-713.
Wiggins, G. (1989b).Teaching to the (authentic) test. Educarional h a d -
ership, 46(7), 4147.
Wolf, D., & Pistone, N. (1991). Takingfull measure: Rethinking assess-
ment rhmugh the arts. New York: College Board Publications.