Portfolio - "...the function of reflection in an assessment context isn't entirely clear."

Kathleen Blake Yancey writes in Reflection in the Writing Classroom (1998), Chapter 7, Reflection  and Assessment, that "As we shall see, the function of reflection in an assessment context isn't entirely clear."  Reading this statement in the second paragraph of the first of three readings this week shaped my focus as I read.  All three of our readings this week, Yancey (1998), Wolf and Siu-Runyan (1996) and Mondock (1997) are 12 to 14 years old. So the question I asked while pondering the readings was how does reflection which is at the heart of portfolio self assessment intersect with the current climate of Federal legislation and initiatives? Among them:

1. formal multiple choice high stakes testing, 
2. 'scientifically based' programs, skills, and strategies, 
3. nationally formed standards,
4. and funding for public schools?

Teachers rightly look for research to support, inform and guide their practice.  In the current climate of politicized education districts are careful to use data and research in more refined ways.  Now 12 to 14 years after these readings were created would Yancey still make that statement?


I think she would.  Searching for clarity produced more thoughts to ponder.  Unless my searches were flawed, which I always wonder when navigating the great world of the web, the NCTE last issued a position paper on portfolios in assessment in 1990,  Resolution on the Development and Dissemination of Alternative Forms of Assessment.  In 2007 they created a position statement, Principles and Practices in Electronic Portfolios, for postsecondary institutions. 

A short hunt for more research on portfolios shows portfolios continue to be used, have entered the digital age, figure heavily in self-assessment and multiple purposes.  Regardless of the political nature of high stakes testing, educators are asking students to reflect and explain due mostly to Mondock's insight that "this evaluation scale provides multiple indicators of student performance and acknowledges differences in individual learning styles. By basing evaluation on student self reflection, the evidence does speak for itself." (Mondock, p. 64.) So the function of reflection may be clear but it's role as a a research based practice remains open.

A few more Abstracts on the topic if you would like to meander....

Andrade and Valtcheva (2009)Abstract:

Criteria-referenced self-assessment is a process during which students collect information about their own performance or progress; compare it to explicitly stated criteria, goals, or standards; and revise accordingly. The authors argue that self-assessment must be a formative type of assessment, done on drafts of works in progress: It should not be a matter of determining one's own grade. As such, the purposes of self-assessment are to identify areas of strength and weakness in one's work in order to make improvements and promote learning. Criteria-referenced self-assessment has been shown to promote achievement. This article introduces criteria-referenced self-assessment, describes how it is done, and reviews some of the research on its benefits to students. 

Andrade, H. & Valtcheva, A. (2009). Promoting learning and achievement through self-assessment.  Theory into Practice, 48(1), 12-19.

Barrett (2007) Abstract:
Theoretical background for researching student learning, engagement, and collaboration through the development of electronic portfolios is described in this article. After providing an overview of the limited research on portfolios in education, the author discusses the accepted definitions, multiple purposes, and conflicting theoretical paradigms of electronic portfolios. Principles of student motivation and engagement are covered, and the philosophical issues related to portfolio assessment and reflection are outlined--paying particular attention to the difference between assessment "for" learning and assessment "of" learning. 

Electronic Appeal: Writing Portfolios Go Digital (The Council Chronicle, March 04)

Elizabeth Beagle, an English teacher at Landstown High School and Technology Academy in Virginia Beach, has seen a lot of teacher interest in electronic portfolios. She attributes this to the many benefits, which she lists as: "storage, ease of reading, option of alternative assessment, variety, and integration of state technology standards."
She also notes some hesitancy among teachers who might not be as familiar with technology. But Beagle encourages everyone to try it, suggesting they start out slowly and build as their skills and confidence do.
"I think what the electronic portfolio does for a lot of students is that it unlocks their minds. I know when I first started doing this, I had students who would have been reluctant to turn in a standard, written portfolio, but because it was digital, it intrigued them. They were able to express their personality, using fonts, color, and graphics." Enticed, students turned in a portfolio, even if the quality wasn't always stellar.

A Quick Tour of Four Portfolio Programs (The Council Chronicle, March 04)

At St. Olaf College in Northfield, Minnesota, students who want to design their own major are asked to create a Web portfolio that demonstrates the connections they've made in their studies.
David Booth, associate professor of religion and director of the Center for Integrative Studies, says that the college has identified four "habits of mind" as central to students' development in liberal arts. Booth and his colleagues believe that Web portfolios are uniquely suited to developing these habits of integrative thinking, reflective thinking, thinking in a community, and thinking in context.
He says that the college is exploring further possible uses for electronic portfolios, such as in the first-year writing program, to increase the cohesiveness of the first-year experience. Some established interdisciplinary programs are also thinking about including Web portfolios, he adds.

The Real Purpose of Rubrics

Mabry and Wilson tell us that rubrics have failed to live up to their promises of fair, reliable, and consistent evaluations. The kinds of rubrics they describe—which stifle writing, force it to be contrived, or ignore the creativity of the writer—must be those “one size fits all” rubrics that accompany commercial curriculum materials. Perhaps their mistake was to equate rubrics and grading. The real purpose of rubrics is to stretch a writer to try something different and see what happens. It’s ultimately a way to create a more confident writer who will have received peer and teacher feedback on the effectiveness of his/her writing and will be more likely to reproduce what worked in the next piece of writing.

I agree that to be a useful tool, a rubric must clearly identify what each criteria is but also what it means. A rubric with excellent criteria but vague descriptors will only end up confusing students: it’s nothing more than a tool to play “Gotcha!” with kids. On the other end a 14-page fully explicit rubric is unrealistic and ridiculous. So how do we create rubrics that clarify expectations and demystify the complexity of writing? How do we use rubrics so they not only provide a language for talking about what good writing looks like but also help students understand and assess their own performance in order to improve their writing?

First, we need to be realistic. Which trait or criteria needs to be emphasized? Which traits/criteria are the “no excuse” criteria that everyone is expected to demonstrate? I liked the way Janice focused her rubric on no more that 3 or 4 traits, knowing which ones were most important for the purpose of the writing. Her rubric also left room for some self-evaluation and creativity, which Wilson would certainly approve.

Secondly, we need to teach students the meaning of the rubric, using student exemplars to show clear expectations. Students know what excellent work looks like. They can also tell you when a text lacks so much substance that it stinks. Those are the 4’s and the 1’s on our rubric’s scale: they’re the no-brainers. That’s why I also like Zach and Janice’s idea of not even showing the 4’s or the 1’s. Instead, we need to spend the most time and energy teaching about the 2’s and 3’s: the average stages of the rubric where most of our students sit, stuck in the rut of the limited writing skills they possess. This is the true key to unlocking the hidden talent in writers by nudging them to try something new, something slightly better. Through self-assessment they will see how they stretched themselves just a little further, which is key to increasing students’ writing skills. When students clearly understand not only what good writing looks like but how theirs compares to the criteria, AND how to get there baby step by baby step, they will be able to improve their writing.

So the answer to Wilson’s questions about rubrics comes down to the purpose and the process for using the rubric:
Purpose: Stretch yourself and practice what good writers do.

Process:

• Look at exemplars, concentrating on the 2’s and 3’s.
• Get student input in writing the rubric’s criteria.
• Use the rubric for self-assessment, peer-conferencing and teacher feedback.
• Reflect on what worked well in students’ writing so they can replicate it next time.

I'm not sure if this is fully reconciled with Wilson's stance on rubrics but it leaves me more confident that rubrics are indeed the right tool to help kids make progress in writing.

Speaking of figure skating...

Linda Mabry argues that "Rubrics incorrectly imply that good writing is the sum of the criteria on the rubric" (p. 678). When I read this, I thought of figure skating. When I was a kid, figure skating was judged differently; the scores were not point-based and judges had more discretion in awarding scores to skaters. Now, figure skaters are awarded points based on the difficulty of the elements of their program (jumps, spins, when in their program they perform their jumps, etc.). This was done to make the judging of skating fairer and less subjective. It strikes me that this new system of judging is a lot like a rubric. It breaks skating down into individual parts and then the final score is the sum of those individual parts of a program. There is also some room for artistry or performance points, and those are largely left to the judges' discretion.

The problem with the new system is that sometimes a skater can perform beautifully and still not get a score that reflects the artistry of her piece. Or, conversely, a skater can really flub up, but still get a good score because of his reputation or past performance. For example, I really thought that American Johnny Weir did a nice job on Thursday night during the men's free skate. He had no noticeable big mistakes and his program was interesting and evocative. I guess (according to Scott Hamilton and Peggy Fleming) his program was less difficult because it had fewer transitions. Still, Weir only ended up in sixth place. It felt unfair.

Overall, I do feel that this system has been a good change in skating. When I was a kid, it seemed like judging sometimes fell along the old Cold War lines. Eastern European countries gave higher scores to Russian skaters. American, Japanese, British and Australian judges gave higher scores to American skaters. I recall this happening with Nancy Kerrigan, an American, and Oksana Baiul, a Ukrainian skater, in the 1994 Olympics. But I do think that sometimes this system can, like a writing rubric, fail to account for the beauty in skating. I would much rather watch a clean, cohesive, beautiful program than a program with a bunch of triple jumps and very little artistry. To me, those programs feel like watching a bull in a china shop.

This isn't a perfect analogy--skating and writing are different. But, like in skating, a well-crafted piece of writing isn't always the sum of its individual parts. Good writing would probably score high in all of the different areas--strong voice, interesting word choice, cohesive organization, etc. But sometimes, as Wilson argues, there will be pieces that most would argue are wonderful, but that may not earn the highest point totals. What to do...what to do.... Go with your gut?

Still speaking of rubrics

While I found myself nodding in agreement with many of the arguments made by Mabry and Wilson, I still consider rubrics to be a real help to me as a teacher and assessor of student writing. At least, I don't think there's a better alternative. If I had time to sit down with every one of my students each week and conference with them, that would certainly be better, but I have 149 students.

When I was in high school and college, I had never heard of a rubric. The writing process as well as writing assessment -- how teachers assigned grades -- was a mystery to me. I had some good natural tendencies as a writer that were rewarded by good grades, but that was as much as I understood. I think I learned most from the instructors who wrote copious comments on my papers, but overall there wasn't much direct instruction about writing or help with revision.

Now I think writing teachers, including me, have worked hard to de-mystify what good writing is, how to do it, and how it's assessed. And the challenge is how to do that in a way that encourages creativity and voice rather than stifle it. In the past, I have taught AP Lang and Comp students who have formulaic essay writing down cold (and can score 5's easily on the AP test), who are "good writers" in a technical sense, but whose writing is flat and dull and stilted. With those students, I felt like I was trying to help them unlearn what well-meaning writing teachers have taught them so well (like the precise number of supporting examples needed in a paragraph).


I don't think that rubrics have to "strip writing assessment of the complexity that breathes life into good writing," as Wilson says. If a rubric is carefully written, it allows space for this complexity. I think that my best rubrics DO honor complexity. . . but the downside of these very rubrics is that they are not so student-friendly. The language that I try to use in describing student papers needs a whole lot of interpretation from me, and this, I admit, is not good. But the best rubrics give students something to work towards, I think.



I do disagree with one of Mabry's assumptions about writing. She writes that "The standardization of a skill that is fundamentally self-expressive and individualistic obstructs its assessment." I don't consider writing to be "fundamentally self-expressive and individualistic." Certainly some writing, like journal writing, is. But lots of the writing we do in and out of school is for the purpose of communicating clearly, for persuading, for entertaining. For this writing, it makes sense to have objective standards, I think, for what "good writing" is. Let's keep our idea of "good writing" flexible, but let's do have an idea of what we're working towards.

Why I Grade

As a secondary English/language arts teacher I am working to create students who are curious, who find writing meaningful, who are willing to invest themselves in their writing (Alfie Kohn's "support model"). At the same time, I want to teach my students how to write coherent paragraphs, how to create thesis statements, how to support their ideas with examples, and how to use language with accuracy and precision (Kohn's "demand model"). Kohn sets these two sets of goals in opposition to each other, but I don't want to think that I have to choose between the two. I want my students to learn to play with words, to construct meaning through words, but I also want my students to be able to write strong thesis-driven essays -- the kind of high-stakes writing that they will need to do over and over again in school. I don't think these goals have to exclude each other, though I think Kohn believes that they do. . .

After reading Lynn Bloom's "Why I (Used to) Hate to Give Grades" and Alfie Kohn's "Grading: The Issue is Not How but Why," I am left thinking: "So the kids are better off without me???" In both pieces, the authors suggest that teachers most often do more harm than good, with regard to the assessment of writing. Grading writing (mostly) discourages, stifles, or misleads our students, so better to resist quantifying anything -- instead just let them write? Hmmmm. . . . . .

I believe that students become better writers through revision. I like what Kelly Gallagher tells his students: You have to write the bad writing first to get to the good writing. I really believe this. And it is grades that motivate students to revise. I think this works. (I know Bloom and Kohn would disagree). 8th graders tend to want to view writing as a kind of race -- get it over with as quickly as possible. Revision is not appealing at all -- but because a grade is attached, students do revise and rework and redo, and they become better writers and thinkers as a result. In the last two years, I have begun using a policy in which students are allowed to revise and resubmit all of their major writing assignments if they choose. Most of the students who do choose to resubmit are motivated to improve their original grade. If nothing else, students are learning how to revise their work.

Questioning Rubrics

"The grade's validity--or ability to reflect the 'worth' of the paper--relies on the rubric's ability to predict how these factors work together to create good writing." (Wilson, 32)

How many of you remember all the work we put into creating the power standards for writing? I, for one, spent countless hours creating rubrics to go with each of the genres I was teaching my 5th graders. The rubric is based on my instruction of writing, meaning it changes every time I improve my own skillS as a writing teacher. Even though Wilson claims rubrics save time, I still spend many weekends and evenings taking the time to consider how each student applied the skills we learned in class. I certainly wouldn't say it's easier, but it definitely creates a standard (even if it's just a standard that applies to me alone).

Our last writing assignment was a personal narrative that we turned into a VoiceThread. I was troubled by one student in particular because she didn't include a couple of the main revisions we worked on in class (Barry Lane's "Exploding a Moment" and "Adding a Setting"). As a result, I couldn't score ideas any higher than a 2. However, this student used dialogue brilliantly to help me envision what took place in this special moment she shared with her best friend. I didn't have "dialogue" anywhere on my rubric, and the girl was understandably upset when she saw a 2 under the trait of "Ideas". She asked how she could make it better, and I told her. I have to admit that by adding her "Exploding a Moment", it did help me visualize the scene better. However, she added other parts that just took away from the simplicity of having a couple third graders excitedly exploring their newfound spelling oddities, "pugnacious" and "schwa". Meaning, she added these descriptions because I told her to, but it actually made her writing worse.

If rubrics aren't the best summative assessment, then what is? There has to be an end to writing at some point. With 107 students, I can't keep allowing them to revise their writing. There just isn't enough time for one person to assess multiple revisions of one assignment. I am looking for some sort of magical answer here. Someone please...pull that rabbit out of the hat! Given the resources and number of students we have, how can we assess writing in a meaningful way? And how can we motivate students to improve their writing, even if the teacher isn't giving them any more feedback?

Let's just get rid of grades...

Two great articles for this week!

"Grading: The Issue Is Not How but Why"
I am fairly confident that, like several other schools in the metro, our district will adopt the live grade book. Every time a teacher enters a grade a parent will receive some kind of notice, and he or she can check the status of a student. This step seems to be contrary to what Kohn and supporting research suggests: "Only by abandoning traditional grading and performance assessment practices can we achieve our ultimate educational objectives." In fact Kohn points out that grades can squash creativity and increase fear of failure. I thought we are part of the "creative age," where young people are expected to come up with creative solutions to problems they did not create but will ultimately have to pay for during their lives. Schools seem to be emphasizing grades more than progress, understanding, and student desire.

What alternative do we have to grades? I readily admit that I am cynical of colleges such as Evergreen that have chucked the traditional grading system. How on earth is there time to write comments for every student? How can student performance be measured by outside interests, such as potential graduate schools? The few schools that are abandoning grades are facing a gigantic monster that is grades. Does that make sense? Grades are traditional. Grades are safe.

I am curious to see what would happen if a teacher decided to not put a letter or number grade on anything for a quarter. Instead, assessment would be done during conferences with the students and by written response sans a grade only. At the end of the quarter some kind of grade would have to be given to satisfy... everyone. There would be the pass or fail option as well, but would that be detrimental to a student's transcript? I don't know. Would colleges take the time to read the teacher commentary? It seems a post-secondary school would learn much more about a student this way. Would the administration of a high school support this experiment? Would a teacher loose her mind trying to find time to do this? Grades are easier, it seems.

Maybe I should try this next fall...