Annotating as an Assessment

I use annotating with my students all the time.  I find it so useful when trying to get students to revisit the text and look more deeply at the content and structure.  However, I never framed it as a formative assessment tool.  After reading this section of Cris Tovani's So What Do They Really Know?, I plan to use it this way in the future. To read Tovani's explanation of annotation, click here.   

When determining which of the myriad of assessment tools to use, Tovani simplifies matters by ensuring that all of her assessments meet four basic criteria:

• Can all learners use this tool to show thinking?
• Will this tool immediately inform my instruction and provide a way to give real-time feedback to students?
• Will patterns of understanding or confusion emerge as a result of using this tool?
• Is this tool convenient to design, use and administer?

If used deliberately, annotation can do all four.  Tovani concedes that

Annotating as a viable assessment opportunity is easy to dismiss because it is so simple to use.  Yet I continue to use this instructional strategy as my number-one source of trustworthy data, mainly because it puts learners in the driver's seat when it come to showing me what they understand. 

Without having "one right answer" annotation allows for divergent thinking on the part of students and allows the teacher to see the confusion, inferences, and strategies employed by students.  By looking at these "tracks of thinking" (as they are sometimes called), teachers can plan whole class or small group mini-lessons to target specific student needs.  

One extremely helpful strategy I got out of this chapter is using annotation as a pretest. Tovani talks about a math class annotating the end of chapter test on the first day of the unit to see what students already know about the content and what holes they have.  This in turn informs instruction since the teacher will know what content can be skipped or reviewed rather quickly and how students' past knowledge shapes their understanding of this new content.  Even if students have no clue how to do a specific problem, they still have to start it and write an annotation when they get stuck.  I can definitely see applying this to grammar, especially since students tend to come with ten years (or more) of background knowledge on the subject.  Though I give pretests, I have not had them annotate their pretests before.

Debriefing Round-up: Links from across disciplines

While searching for resources about debriefing, I stumbled upon the links below.    

Carrie Caudle, who teaches Improv Theater, at Improv Education explains how she formats her debriefing sessions to include student input on their observations- "What did you notice?", reactions- "How did it feel to...?", connections- ""How is this like...?", and change- "What are you going to do differently next time?"

Support Real Teachers, a support website for PE teachers, provides debriefing sentence stems I found valuable.

I came across quite a few articles dealing with debriefing after simulation activities in nursing and other medical training. One such article aptly named  "The Essentials of Debriefing in Simulation Learning: a concept analysis" (originally published in Nursing Education Perspectives) elaborated on the idea that "By providing opportunities to review events and make visible their meaning, debriefing offers a way to draw out student thinking and help students develop their complex decision-making skills."

Performance Learning Systems asserts that debriefing activities can utilize one or more of three objectives: summarize curriculum information, evaluate interpersonal skills, and identify the thinking process.  For each of these objectives, a list of questions or topics follows.  In addition to encouraging metacognition, identifying the thinking process (included below) would encourage transfer as well.

• What types of thinking did you use while doing the activity?
• How did you come up with the generalizations or conclusions you derived from the activity?
• What path did your mind follow to get to those ends?
• Could your thinking have taken an easier path? A different path?
• Did you develop any new thinking patterns by doing the activity?
• What strategies did you use to complete the activity?
• Could your thinking processes be applied to any other learning situations?

Debriefing

From my reading of So What Do They Really Know? by Cris Tovani, I have been doing some thinking about debriefing. At the end of her chapter outlining her workshop model, Tovani cautions:

Make sure you don't skimp on time to debrief.  This crucial component of the workshop model provides an opportunity for students to reflect on their learning.  Regular opportunities to debrief not only hold students accountable for the way they use their work time, but also give teachers a second chance to assess what students need next.  Allowing students to collect their thoughts at the end of the period and share new learning makes others in the class smarter. The debriefing also gives the teacher insight into students' patterns of understanding and confusion.

Though debriefing is hardly a new component to lesson design, it it often overlooked as it occurs at the end of the lesson.  However, it is an important part of the lesson design for several reasons:

• it encourages metacognition
• it allows students to share their learning with others
• it encourages transfer and interaction with overarching ideas and patterns
• it is a type of formative assessment
• it guides upcoming learning experiences

From my own experience, debriefing can take a variety of forms:

• exit slips
• small group share out
• journaling
• whole class discussion

Tovani emphasizes the importance of monitoring students as they engage in the lesson and recording data to share with the class during debriefing.  These recordings could be notes about content or group process.  The problem with this, as even Tovani points out, is that the teacher can not sit at his/her desk and grade while students are working. Though Tovani does not mention in this chapter how she works around this, I am assuming (based my initial perusal of the table of contents) that she will touch on this topic in greater detail later in the book

Wellness Checks vs. Autopsies

Stenhouse recently released a new book by Cris Tovani called So What Do They Really Know?  The subject matter of this resource: formative assessment.  As are most professional development books put out by Stenhouse, this one is currently available via their website to be perused in its entirety. 

If I was unsure about this book, all fears were allayed as a result of the opening quotation taken from Dylan Wiliam and Paul Black, two researchers who have written on the subject of formative assessment in the classroom:

We use the general term assessment to refer to all those activities undertaken by teachers--and by their students in assessing themselves--that provide information to be used as feedback to modify teaching and learning activities.  Such assessment becomes formative assessment when the evidence is actually used to adapt the teaching to meet student needs.

From the initial quotation, a strong link between assessment and feedback is forged.  However, it is the second sentence which I believe encapsulates the essence of formative assessment: it is not formative assessment unless is is then used to adapt instruction in some way; without informing instruction, is ceases to be formative assessment. 

Later in the chapter, Tovani uses a metaphor from health care to further illustrate the difference between summative and formative assessment.  Summative assessements are compared to an autopsy, which

might inform the medical profession, comfort a family member, or provide useful information to a crime investigator, but it doesn't do anything for the person who has died.  Like the autopsies, summative assessments can rank and categorize learners, give colleges a way to standardize how they admit prospective students and allow parents to brag about their genius child.  Unfortunately, they don't help students get smarter in the tested area. 

One reason Tovani makes this assertion is that often this data is not available until after students have completed the school year.  It is then too late to reteach content or clear up misunderstandings.  Though it can be helpful for the individual teacher to see how the class as a whole scored on a particular concept, it is rather worthless for the individual students.  Many just see what their overall score is and then throw it in the garbage.

Contrarily, Tovani argues formative assessments are compared to wellness checkups where

doctors tell patients where they are healthy.  They also give patients feedback in areas where they are not so healthy.  Good doctors take it a step further and give suggestions for how to improve the conditions that are interfering with the patients' overall health.

Doctors do this is to provide tools the tools for patients to improve their health and prevent complications in the future.  The focus is on improvement, not just maintenance. These doctors understand that a person's overall health is not just their cholesterol level or their blood pressure.  Instead, it is a composite of multiple measures like activity level, sleep quality, nutrition, resiliency, social connections, results from blood tests, etc.

So far, I am grateful that I started to read this book, and I look forward to much insight as I work my way through the rest of it.  

The OK Plateau

I just finished Moonwalking with Einstein by Joshua Foer, a year-of memoir of the author's experiences preparing for and competing in the US and World Memory Championships.  I enjoyed the author's style and the way he juxtaposed personal anecdotes and research from a variety of experts and disciplines.

Though I did not read it with the intention of it intersecting with my summer professional development project (I read it because of the intriguing title and cover), one chapter in particular, The OK Plateau, really stood out.  The entire chapter focuses on moving past the plateaus that individuals encounter on their way to expertise.  Though an array of examples were given ranging from ice skaters to typists, the one that stood out to me was about mammographers.  According to Foer,

Even though you might be inclined to trust the advice of a silver-haired doctor over one fresh out of medical school, it’s been found that in a few fields of medicine, doctors’ skills don’t improve the longer they’ve been practicing.  The diagnoses of professional mammographers, for example, have a tendency to get less and less accurate over the years.  Why would that be?

For most mammographers, practicing medicine is not deliberate practice, according to Ericsson.  It’s more like putting into a tin cup than working with a coach.  That’s because mammographers usually only find out about the accuracy of their diagnoses weeks or months later, if at all, at which point they’ve probably forgotten about he details of the case and can no longer learn from their successes and mistakes.

One field of medicine which this is definitely not the case is surgery.  Unlike mammographers, surgeons tend to get better with time.  What makes surgeons different from mammographers, according to Ericsson, is that the outcome of most surgeries is usually immediately apparent—the patient either gets better or doesn’t—which means that surgeons are constantly receiving feedback on their performance.  They’re always learning what works and what doesn’t, always getting better.

The first thing I took away from this passage is the immediate feedback is essential to developing increased expertise. Because of the delayed nature of feedback, mammographers are unable to immediately reflect on their performance and make adjustments to their practice.  And to me, as an educator, I must admit that I am guilty of delaying feedback, perhaps not weeks or months, but long enough for it to lose much of its value. And the question I must ask myself is: if it is not valuable, what good is it?  If I am serious about providing my students with opportunities to grow and improve, I need to adjust my feedback to ensure that it is immediate and constant. 


Those who enjoy the structure of this book may also like Outliers by Malcom Gladwell, Year of Living Biblically by AJ Jacobs, or the Happiness Project by Gretchen Rubin.

Professional Development for 2011-12

Every summer I try to develop a professional development goal for the impending school year- just one aspect that I want to focus on, learn more about, practice, and improve.  After much thought, I decided to spend time on meaningful feedback and formative assessment.  Though these may seems like two different aspects, the more I think about them the more intertwined the become--and at this point, I do not believe that I can separate them.

I have begun reading about these topics and hope to post more info at a later date.

The Rise in Online Learning

Today I read an interesting article entitled "More Pupils Are Learning Online, Fueling Debate on Quality" published in the New York Times on April 5th.  Though the article seemed to provide several different viewpoints on the issue of quality in online learning environments, the conclusion seemed to be that the quality is sub-par compared to traditional means of instruction.  Since my school is currently using an online platform for credit recovery, I am especially interested in this issue to see if further research confirms  or questions the effectiveness of online platforms for high school students, especially after the opening anecdote:

"Mr. Hamilton, who had failed English 3 in a conventional classroom and was hoping to earn credit online to graduate, was asked a question about the meaning of social Darwinism. He pasted the question into Google and read a summary of a Wikipedia entry. He copied the language, spell-checked it and e-mailed it to his teacher."

If nothing else, it does serve as a reinforcement that I need to be sure to offer instruction that surpasses that which students can just get by listening to lectures online through reinforcing problem solving, critical thinking, and collaboration.