Skip to main content

Almost every adult in the United States has experienced at least one performance assessment: the driving test that places new drivers into an automobile with a DMV official for a spin around the block and a demonstration of a set of driving maneuvers, including, in some parts of the country, the dreaded parallel parking technique. Few of us would be comfortable handing out licenses to people who have only passed the multiple-choice written test also required by the DMV.  We understand the value of this performance assessment as a real-world test of whether a person can actually handle a car on the road.  Not only does the test tell us some important things about potential drivers’ skills, we also know that preparing for the test helps improve those skills as potential drivers practice to get better.  The test sets a standard toward which everyone must work.  Without it, we’d have little assurance about what people can actually do with what they know about cars and road rules, and little leverage to improve actual driving abilities.

This quote is from a briefing paper prepared for the Congress as it reconsiders NCLB.  Please keep reading and I will explain in more detail.

The title of the briefing paper is Refocusing Accountability:  Using Local Performance Assessments to Enhance Teaching and Learning for Higher Order Skills.  It was prepared by four people:

George H. Wood
Director, The Forum for Education and Democracy
Principal, Federal Hocking High School, Stewart, Ohio

Linda Darling-Hammond
Charles E. Ducommun Professor, Stanford University
Co-Director, School Redesign Network

Monty Neill
Co-Director, Fair Test (National Center for Fair & Open Testing)

Pat Roschewski
Director of Statewide Assessment
Nebraska Department of Education

Here I note that until a new bill was signed by the Governor, Nebraska had only one state-wide assessment, that was in writing and it was graded within the schools.  The state is now apparently adding a onetime application of high school tests in reading and math as well, although I am not yet clear on how that will affect the state's overall approach to assessment and accountability.

It won't take long to read the briefing paper, and I am not going to reproduce the entire document here.   Let me offer the first paragraph:

Performance based assessments, often locally controlled and involving multiple measures of achievement, offer a way to move beyond the limits and negative effects of standardized examinations currently in use for school accountability.  While federal legislation calls for "multiple up-to-date measures of student academic achievement, including measures that assess higher-order thinking skills and understanding" (NCLB, Sec. 1111, b, 2, I, vi), most assessment tools used for federal reporting focus on lower-level skill that can be measured on standardized mostly multiple-choice tests. High stakes attached to them have led schools to not engage in more challenging and engaging curriculum but to limit school experiences to those that focus on test preparation.

and the second:

Performance assessments that are locally managed and involve multiple sources of evidence assist students in learning and teachers in teaching for higher order skills.  These tools engage students in the demonstration of skills and knowledge through the performance of tasks that provide teachers with an understanding of student achievement and learning needs.  Large scale examples involving the use of such performance-based assessments come from states such as Nebraska, Wyoming, Connecticut and New York, as well as nations such as Australia and Singapore.  The evidence from research on these and other systems indicate that through using performance assessments schools can focus instruction on higher order skills, provide a more accurate measure of what students know and can do, engage students more deeply in learning, and provide for more timely feedback to teachers, parents, and students in order to monitor and alter instruction.

The key for me as a teaching is the final sentence of the 2nd paragraoh, so let me repeat that in bold:  

The evidence from research on these and other systems indicate that through using performance assessments schools can focus instruction on higher order skills, provide a more accurate measure of what students know and can do, engage students more deeply in learning, and provide for more timely feedback to teachers, parents, and students in order to monitor and alter instruction.

Many people have some experience in smaller levels of performance assessment within schools:  science fair and National History Day projects are both examples of performance assessment activities.  I have served as a judge for the latter at a County level as well as within our school for students I do not teach.  I have clearly seen evidence that students can show how to apply knowledge using the appropriate skills relevant to the student of history and related topics, and the level of knowledge and understanding demonstrated is far superior to any I would obtain merely by reading the results of their performance on a multiple choice test.

Let me offer most of the paragraph that immediately follows the one with which I began this diary.  The authors note the similarity of performance assessments in education to the ubiquitous driving test, and then comment:

They are tools that allow teachers to gather information about what students can actually do with what they are learning – science experiments that students design, carry out, analyze, and write up; computer programs that students create and test out; research inquiries that they pursue, seeking and assembling evidence about a question, and presenting in written and oral form.  Whether the skill or standard being measured is writing, speaking, scientific or mathematical literacy, or knowledge of history and social science research, students actually perform tasks involving these skills and the teacher observes, gathers information about, and scores the performance based upon a set of pre-determined criteria.  As in our driving test example, these assessments typically consist of three parts; a task, a scoring guide or rubric, and a set of administration guidelines. The development, administration, and scoring of these tasks requires teacher development to insure quality and consistency. The research suggests that such assessments are better tools for showing the extent to which students have developed higher order thinking skills, such as the abilities to analyze, synthesize, and evaluate information.  They lead to more student engagement in learning and stronger performance on the kinds of authentic tasks that better resemble what they will need to do in the world outside of school. They also provide richer feedback to teachers, leading to improved learning outcomes for students.

There is no doubt that the information obtained from a proper use of such an evaluation is far more meaningful and applicable than other methods of assessment.  What is also important is that it be evaluated as quickly and as locally as possible.   The authors discuss that in more detail.

Here I should note that George Wood is principal of a school that is a part of the Coaltion of Essential Schools, which, building on the work of Ted Sizer, places a strong emphasis on the use of performance assessments.  It is worth noting that among the common principles accepted by all CES member institutions are
   * Personalized instruction to address individual needs and interests;
   * Small schools and classrooms, where teachers and student know each other well and work in an atmosphere of trust and high expectations;
   * Multiple assessments based on performance of authentic tasks;

I don't want to overburden you in this diary, and I am also late leaving for school.  I am going to urge you to read the briefing paper.  You will see how the authors relate the material to issues of reauthorization of NCLB.  You will see how much of this can be done locally.  You will be pointed at examples of how performance assessment is already being done, sometimes on a relatively large scale basis.  

Some are likely to object to the cost of such an approach.  I would counter that any resources dedicated to this kind of assessment directly tie in with instruction, and the information obtained is much more applicable to improving instruction for the individual students.  Ultimately the ability to apply learning - knowledge and skills - in reall world application is the most meaningful measure of what students have really learned.  In testing, as in far too many other areas of American life, we do not fully measure the costs of what we are doing, thinking only of short-term out of pocket costs, and not fully valuing other things  -  I did address that in this recent diary which did not get all that much traffic, but that's okay.  

This diary may not get much attention either.  But I feel an obligation to do more than merely complain about our current approach to testing.  I have lobbied on the Hill about NCLB with one of the authors, and this was one of the materials we were offering to people.  I know that there are people involved with the NCLB rewrite who are interested.

I suggest that it is worth your while to become familiar with the material.  I also suggest that you pass on the link to anyone in authority with respect to schools:  that includes local and state school board members, and state and federal legislators, particularly if they sit on committee addressing issues of education.

I have to travel now to school.  I will have time during the day to respond to any comments you may choose to offer.

Peace.

Originally posted to teacherken on Wed Jun 13, 2007 at 03:26 AM PDT.

EMAIL TO A FRIEND X
Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags

?

More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  Of course I think this is important (23+ / 0-)

    that's why I took the time to do the diary.  You, the members of this community, will have to decide how important this issue is to you.  

    A heads up - there will be an opportunity to address this at Yearlykos - I will be posting more about this in the near future.

    Now, do with this what you will.  I have no further control over what may happen to this diary.

    Peace.

    Those who can, do. Those who can do more, TEACH!

    by teacherken on Wed Jun 13, 2007 at 03:26:15 AM PDT

  •  My wife thinks this is an important issue (2+ / 0-)
    Recommended by:
    annetteboardman, va dare

    which is one reason she has already recommended this diary!  Unusual for her to be active at this time of the morning  :-)

    I am now in transit.  Will catch you on the flip side of the Potomac in about 40 minutes.

    Peace.

    Those who can, do. Those who can do more, TEACH!

    by teacherken on Wed Jun 13, 2007 at 03:32:09 AM PDT

  •  I am reading AP exams this week (4+ / 0-)
    Recommended by:
    teacherken, plf515, va dare, evanindallas

    And it seems to me that in addition to evaluating the students, we are evaluating the teaching as well.  It is very interesting to talk with people here and discuss how their classroom practice has been shaped by their work as graders/evaluators.  
    I am not sure quite how to put things in words, as I am still working through this in my own mind, and of course I cannot talk about specifics on the tests.  But it is interesting -- it is a good readjustment to see what my incoming students will have and what abilities will have been developed (whether they took this particular exam or not, it is a whole bunch of high school students' writing, which is good for me to see on a regular basis).

    •  I'm not sure you are evaluating teaching (3+ / 0-)
      Recommended by:
      DFWmom, Wee Mama, annetteboardman

      since you have no control over what the students could do without the teaching.  After all, you do not know how many of the students whose papers you read did not sit in an AP class - last year we had a student who did not take AP US Gov and state for both the AP US Gov and the AP Comparative Gov and got 5s on both.  

      How would the students have performed without the AP instruction?  Only then are you in a position to be able to say you are evaluating the teaching.

      That is perhaps one reason the AP audit does not include student test scores as an indicator that is considered.  

      Sorry for the disagreement.   Unless you know what went on in my classroom, you have no way to evaluate my teaching merely by looking at a summary assessment on the AP exam.

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Wed Jun 13, 2007 at 04:31:47 AM PDT

      [ Parent ]

      •  My son took several AP exams including the one on (2+ / 0-)
        Recommended by:
        teacherken, plf515

        environmental science, the only one for which he did not have an AP class. He is very confident about his performance on that one (and he seems to have a good sense of  this). He mastered that material through an ecology summer class in junior high, his strength in chemistry, physics and economics, and our table conversations. We are all interested to see that score ---

      •  fair enough (0+ / 0-)

        I stand corrected.  
        What I have been seeing is paper after paper from the same school have the same answer with the same mistake.  I assumed it was the teaching.  But perhaps that is not the best explanation.

        •  that COULD be the teaching in the AP class (1+ / 0-)
          Recommended by:
          annetteboardman

          it could teaching in a prior common class

          it could be a defect in a textbook

          it could be a deficiency in the structure of the question

          and so on

          because you don't know WHICH it is you cannot assume that it is necessarily the teaching in the class, even though that COULD be the reason.

          Those who can, do. Those who can do more, TEACH!

          by teacherken on Wed Jun 13, 2007 at 02:34:52 PM PDT

          [ Parent ]

    •  I just revised and edited (3+ / 0-)
      Recommended by:
      teacherken, annetteboardman, cfk

      the Princeton Review's AP Statistics book.

      Fascinating to see how the grading is done.

  •  The structure of a test is important -- but... (4+ / 0-)

    Performance assessments are important in a number of ways you describe, and even Rick Hess acknowledges their value when "done right" (which I suspect is a Potter Stewart-like definition).

    But I've seen performance assessments treated as high-stakes tests, given quantitative rankings, and then taught to. The FCAT writing exam is the best in the bunch, because of the performance aspect to it, but it's no cure-all to the overemphasis on the FCAT.

    There is also some research to indicate that performance assessments are harder on students with disabilities - if I remember correctly, the speculation is that there is a higher threshold for performing well for some students, and so it is less sensitive to improvement.

    •  again, with multiple methods/measures (2+ / 0-)
      Recommended by:
      annetteboardman, va dare

      it removes some of the isssues you raise, which are issues of concern

      obviously, we cannot substitute a written test for the driving test.  Of course, we can as a preliminary method take some of the pressure off using simulators - soe we have some minimal amount of confidence before the rubber hjits the road

      I do owrry when I see rubric scores being analyzed statistically - it reminds me of some of the worst of the educationl literature I have seen - high degrees of statistical manipulation and analysis of Likert scale surveys, but I don't want to get too far off topic.

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Wed Jun 13, 2007 at 04:21:53 AM PDT

      [ Parent ]

  •  Please, Ken. You're Getting Ridiculous. (0+ / 0-)

    What is your aversion to tests and testing?  That you have to grade papers?  (Even those are machine graded these days.)

    MEMORIZATION and DRILL are ESSENTIAL for ANY ONE of the professions.  It is the ONLY way to KNOW that the person KNOWS what they have purportedly learned.

    If you don't believe that, then I suggest you have your next major surgery performed by someone who failed out of medical school but who has demonstrated that he or she is adept at handling a scalpel.

    Now, I'll wait for the UFT types to obliterate this comment out of existance, as they usually do.  (Honest debate is, apparently, not among the "soft" teaching methods that failed teachers use to cover up their incompetence by failing to give written tests.)  

    "The beginning of thought is in disagreement -- not only with others but also with ourselves." - Eric Hoffer www.InTheArena.bravehost.com

    by Thinking Republican on Wed Jun 13, 2007 at 04:19:57 AM PDT

    •  I don't have an aversion to tests (11+ / 0-)

      I have a strong aversion to misuse of tests.  And btw, I am NEA  -

      oh, and by the way, someone in medical school is not just doing book work -  they have to demonstrate applicability of the book work.  And no one is liecensed as a surgeon without having to demonstrate applicability under supervision, so even your example weakens your argument.

      A multiple choice test, if well designed (which many are not), is an inexact sampling of the domain(s) being assessed, and may or may not be an an accurate measure of the underlying knowledge / skill.  

      I am an advocate of multiple methods of measurement, which you seem to ignore in your comment.

      I also believe the biggest single failing of multiple choice tests in the inability of the student to explain the answer she selected -  that a student picked a wrong answer gives heither her nor the teacher meaningful feedback of why she got it wrong.  

      Most assessment should be formative, helping improve and focus in instruction and learning, and not summative.

      And one time application of high stakes summative assessments inevitably leads to distortion of what is being measured.  Berliner and Nichols have demonstrated that Campbell's theorem is applicable at least as far back as the Chinese civil service exams for more than a millenium and a half.

      Oh, and if you don't won't be troll rated, express your disagreements in a less trollish fashion.  We have regular disagreements on this issue here without troll ratings all the time.  we seek to find common ground.  One of the members of the education working group, plf515, is a psychometrician, and we have among us been able to stretch all of our understanding.  You simply seem to want to dismiss anything with which you disagree.  Hardly conducive to learning.

      And on snark you get a failing grade.  Not particularly effective.

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Wed Jun 13, 2007 at 04:28:50 AM PDT

      [ Parent ]

    •  Yes, I agree that drilling is important (7+ / 0-)

      But I disagree that the memorization is all that is needed.  Do you want a doctor who knows absolutely everything that is in his book but cannot look at you when you have an unexpected reaction to a new (to you) medicine or when two diseases are interacting in your body in a way that is different from what is in the published literature (which happens a lot) and cannot do the critical thinking to figure out what is wrong.  There is more than memorization to the training of any successful specialist, whether medical or legal or educational.  When you don't have any way to test that ability, and you don't with many many high stakes tests, you stop teachers from taking time away from the drilling and memorization to teach students how to think (and in my field how to write).  Memorization is not the only thing that is important in learning.  

      •  Let's see (4+ / 0-)
        Recommended by:
        teacherken, Wee Mama, annetteboardman, cfk

        Suppose I need an appendectomy.

        There are two doctors available

        One can tell me the name of every bone, muscle and organ in the body.  But he's never actually DONE an appendectomy.

        The other doesn't know the names of any of the organs.  But he's done 100 successful appendectomies.

        I know which doctor I would want!

        Names and things can be looked up.  Techniques cannot.

        Memorization is VASTLY overrated.  It's almost never needed, and is often a hindrance to understanding.  (This is especially so in math, where memorizing formulas removes the need to understand them).

      •  You're right (2+ / 0-)
        Recommended by:
        annetteboardman, cfk

        Students can memorize the "right" answers until the cows come home, but they need to learn the process of how to get there.

    •  Would you (3+ / 0-)
      Recommended by:
      teacherken, annetteboardman, cfk

      Would you want a surgeon who's memorized the book but can't sew up a simple cut?  The current system reduces learning to only one of its components, then tries to force the result to stand for the whole.

    •  Didn't troll rate (3+ / 0-)
      Recommended by:
      teacherken, annetteboardman, cfk

      What is your aversion to tests and testing?  That you have to grade papers?  (Even those are machine graded these days.)

      MEMORIZATION and DRILL are ESSENTIAL for ANY ONE of the professions.  It is the ONLY way to KNOW that the person KNOWS what they have purportedly learned.

      Memoration and drill are important, but are at the bottom of Blooms Taxonomy of Learning.  This is something taught at every reputable education dept. I kept this list right inside my plan book in order to insure that my plans reflected the higher level of skills I was aiming to focus on. And I taught at a poor inner city school.  I always told my students that I was prepared as well as any teacher at the "science magnet" school and they were going to get the same education.

      Mexican American with Canadian citizenship. Ha! Please recommend?

      by TexMex on Wed Jun 13, 2007 at 05:44:25 AM PDT

      [ Parent ]

    •  It might help (2+ / 0-)
      Recommended by:
      Wee Mama, annetteboardman

      if you wish to avoid troll ratings, to not say that a highly respected kossack is 'getting ridiculous'.

      Ken and I disagree about a few things in education.  I have never seen him 'get ridiculous' and he certainly didn't do so here.

      You raise some interesting points.  You might try doing so in a way that invites discussion, rather than derision.

      •  I'll be happy to upgrade my opinion of Ken's ... (0+ / 1-)
        Recommended by:
        Hidden by:
        teacherken

        ...postings  the very first time he writes about teachers actually doing more work on their jobs for the same (or, better yet, less!) money...the same as every other worker in the United States these day.

        Or have you not noticed that the only common theme in his posts is (a) reducing the workload on teachers; (b) reducing their level of accountability; and (c) blaming every other possible cause for failed education except failed teachers.

        "The beginning of thought is in disagreement -- not only with others but also with ourselves." - Eric Hoffer www.InTheArena.bravehost.com

        by Thinking Republican on Thu Jun 14, 2007 at 06:25:49 PM PDT

        [ Parent ]

  •  yes (3+ / 0-)
    Recommended by:
    teacherken, Wee Mama, cfk

    Authentic and performance assessment is something I have always suported in my comments on your diaries.

    And in answer to the disabled question on performance assessments. Well trained cooperation between regular ed and special ed teachers in a collaborative manner get those kids through too.

    It has been a major part of the science reform movement to move to performance and use of projects and portfolios for some time well, at least till just before NCLB when the pressure to teach to the test has wiped out time for good science instruction at a time when other countries including Mexico are moving int the direction of hands on science methods.

    Mexican kids get evolution, but American kids get a new creationist museum.  Teaching to the test doesn't allow time to digest real ideas and complex concepts like natural selection.
    Question:
    How many legs does an insect have?
    quick one anwser

    Question:
    How did pollution in early industrial England influence the color of moths?
    not a quick one answer.

    Which question has more implications for the future of our country and hence a better question to prepare future citizens of the United States.

    Mexican American with Canadian citizenship. Ha! Please recommend?

    by TexMex on Wed Jun 13, 2007 at 05:38:08 AM PDT

  •  My observation has been... (5+ / 0-)

    My observation has been, just in dealing with people of all ages outside school settings, that people learn in different ways.  I remember what I read.  Other people do better remembering what they hear. Some people learn new skills or absorb new understandings only by hands-on doing.

    But while memorizing HTML codes or being able conjegate irregular verbs in Italian can be drilled, using those skills to actually DO something -- build your own website, write a story in Italian, or attempt to write English-to-Italian subtitles for a short video piece engages more than just the memory. It uses different patterns of thinking, different parts of the brain, and reinforces the basic information by putting it in context with the rest of what the student knows and understands.  It also allows the student to personalize this information by tying it into something else (the subject of the website, or story, or video) that is meaningful and creatively stimulating to them personally.  

    It also allows integrating multiple skills and disciplines together, so that a given project can touch on a student's knowledge and understanding of more than one subject at a time -- each reinforcing the other. And doing something with skills learned means the student herself can discover where she doesn't understand a particular point or concept, and get further assistance that can be immediately applied to something she's doing.

    You want drivers on the road who have practiced the basic manuevers enough to be safe. You want programmers or engineers or doctors who not only know the basic facts/data/terminology/procedures of their profession, but how to apply that learning in non-textbook situations, and how to solve problems that arise along the way, even anticipate and compensate for such problems before they happen. Experience counts, and performance-based testing/assessment is a way to get something of the learning-by-experience approach in a school setting, in a way that engages the student on multiple levels.  

    (I hope I'm making sense, I haven't had my coffee yet....).

    •  My best education experience (2+ / 0-)
      Recommended by:
      teacherken, annetteboardman

      was when I was in a program where everything was hands on. Apparently I was a hands on learner. This is all the way back when I was in 8th grade, in a public school where four teachers started a program for a limited amount of kids. Students would have the rare opportunity to personalize some of the curriculum and the students were always participating in the learning process. OTOH, my worst learning experience is where all I was required to do was memorize and the teacher would lecture for 55 minutes straight. It's safe to say I didn't do so well in that class.

  •  All of this focus on methods is driving me (3+ / 0-)
    Recommended by:
    teacherken, elmo, annetteboardman

    nuts. Drilling, memorization, etc., are just learning techniques that can be applied at various times in various situations to achieve learning success. When they are advocated, they become useless. I totally agree that the last sentence of the above second paragraph is vitally important as it acknowledges that teaching and learning is an interactive and dynamic process. The vital need for this is obvious but is lost in all the shrill advocation of teaching technique A and testing method B.

    In the schools my children have attended, so much has been focused on metrics that teaching and learning have gone by the wayside. I have a son who is not doing well in school. When I go to explore the issues with his teacher, I get the response "Oh, it's clear he really knows and understands the material.  He's just not [fill in the blank]."  Well, that was useful.  So the teacher has just fully acknowledged that his grade is not a measure of his knowledge or ability to learn. It's a measure of his ability to jump through the teacher's system of metrics.  F*ck that.

    There's so much wrong with how we teach I'm not sure this country will ever figure it out. IMO we're locked into a death spiral where bad teaching leads to poor thinking skills which leads to even more bad teaching which leads to worse thinking skills.

    OT, It's frightfully hard to "be objective" about the academic system when you're frustrated. I relieve some of my frustration with snark. Today's relief valve?  Thinking Republican is an oxymoron.

  •  What Is The Purpose Of These Tests? (0+ / 0-)

    I suggest that it is worth your while to become familiar with the material.  I also suggest that you pass on the link to anyone in authority with respect to schools:  that includes local and state school board members, and state and federal legislators, particularly if they sit on committee addressing issues of education.

    Are you suggesting that they add authentic assessment to NCLB? Why? So that states can label more schools as failing? Or is it your hope that subjective grading will be more generous than objective grading, which will be a legitimate reason for more schools to pass?

    If I was more Utopian, I would look at this diary and think that schools just need to make learning more authentic rather than about beating the test--then, students who currently can't find the area of a rectangle will be able to design buildings. Realistically, I have a lot of trouble with this. Schools should make learning more authentic, but it is far from a panacea. For one thing, whenever anything becomes high stakes, the focus is on how to beat that measurement, and that's far from the only problem.

    In my opinion, the problem with NCLB is that it encourages states to replace no academic standards with poorly measured high academic standards and then label schools as failing that don't meet those standards. It leads to a lot of schools being labelled as failures, and the states have no idea how to deal with them. Also, the measurements of schools are very poor, and your suggestion addresses a very small part of what makes them poor.

    If you replace the multiple choice tests with multiple measurements, that doesn't necessarily lead to a proper measurement of schools. Should schools still be held accountable for students with poor attendance? Students who moved into the district in the past two months? Students who have made over one year of progress in the past 12 months but are still below grade level?

    As you know, I have a lot of respect for you, and I often agree with you. However, I won't be calling my Congressman on this one.

    •  the idea is to allow use of authentic assessments (0+ / 0-)

      where possible, imbedded in the instructional process, and in lieu of external high stakes testing.

      Wish I had more time to explain, but I think if you ponder the complete document you might change your mind.

      BTW -  I know that the staff from both Senators from your state have seen this.

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Wed Jun 13, 2007 at 06:18:59 AM PDT

      [ Parent ]

  •  Good diary, but of course.... I have questions (2+ / 0-)
    Recommended by:
    oldpro, teacherken

    (you knew I would...., I am a pest)

    In the blockquoted text you have "often locally controlled" and then "large scale examples".  This isn't actually a question by me, but a comment.  I think this contrast points to two of the many uses of tests and performance assessments.  One is to help teachers teach.  Locally based methods may work better for this.  Another is to have some fair method of comparing kids from different schools.  Here, large scale is essential.

    Question 1: How does one standardize a performance based test?  I can see some ways, but it would depend  on knowing a lot more about the structure of the test.  For instance, there are ways of standardizing essay answers.  

    Question 2:  This is a biggie.  A performance based test assumes that we are trying to teach kids to perform some task.  Are we?  In some subjects, yes.  But what task?

    What, for example, is the purpose of a course in literature? Is it to teach a kid to read and then comment critically on a piece of literature? The courses I've taken seem like it.  Is that the RIGHT purpose?  Should not the purpose of such a course be to teach a kid to ENJOY reading literature?

    And how would one do a performance based assessment (or any other assessment) of that?  Well..... if one  had no budget restrictions and didn't mind violating a kid's privacy, one could follow the kid through the year and see how many novels he/she read that were not assigned.....  

    I love to read.  I rarely, if ever, write critically about what I read.  I regard myself as a success.  

    This same problem occurs with art, music, and so on.  If you teach a kid to enjoy playing the piano, then in my view, your teaching is a success, regardless of how WELL that kid plays.  But if the goal of your teaching (and the kid's studying) is a career as a musician, other standards apply.

    For most people, though, I like the following adage, which I made up

    Anything worth doing is worth doing badly

    To get to a more academic subject.  Math.  Here I think we have the purpose completely bollixed up.  We are teaching kids to hate math, while not actually teaching them any math at all.  That is, it is perfectly possible to graduate HS without ever having done any math whatsoever.  Most adults don't even know what doing math IS.  Nearly everyone confuses arithmetic, quantitative reasoning, and math.  It's not their fault.  The schools call it math.  But math is about proving theorems. Math is about the search for beauty.  

    How to measure THAT?

    well.... I could go on and on, but they actually expect me to do some work here, so, since they pay me....

    more later.

    •  and I am packing up classroom (2+ / 0-)
      Recommended by:
      annetteboardman, plf515

      because for some reason I cannot enter my grades from my room.  Now that finals are done 70% of students did not bother to show up today.  Not much to do.  Except packing up the classroom takes forever.

      Tomorrow I will have access to a computer only until Noon, then I will be disconnected at school, including for whatever time Friday I am here.  Oh well . . .

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Wed Jun 13, 2007 at 07:13:52 AM PDT

      [ Parent ]

  •  NCLB actually killed performance-based testing (4+ / 0-)
    Recommended by:
    teacherken, cfk, pioneer111, beobjective

    in many school districts. If the requirements of NCLB were not met by existing tests, school districts often were forced to discontinue their testing regimen--even if these tests were superior--because they could not afford to implement both.  

    Performance-based tests are more expensive to administer and to score--you need actual people, not machines, to score these tests. Our kids are so wonderfully creative that so far every effort to standardize open-ended questions and writing samples to the point a computer program can score them accurately has failed miserably.

    I was educated in German schools and had never seen a standardized test with multiple choice questions  until I came to this country. So even though English was not my native language, I had a huge advantage over my US classmates in college because I was used to responding to questions in writing and could provide in-depth and thoughtful explanations with ease.

    Standardized testing promotes memorization and the mindless repetition of predetermined information at the expense of critical thinking and independent  analysis. Cynic that I have become, it seems to me that this is not an unfortunate by-product of NCLB but its desired outcome. What is more to the liking of corporate and political hegemony? Passive consumers of education or active and knowledgeable citizens able to think for themselves?  

  •  Thinking out loud in a stream of consciousness (0+ / 0-)

    reponse to this diary and the comments it has generated...without enough coffee (yet) so this may be a bit of a rough go.  Still...I am thinking:

    What's the goal?
    What's the problem we are trying to solve?
    Is the drivers' license example germane to these questions?

    Here's the thing:  if your goal is to learn to drive, then jumping in a car - hopefully with a knowledgeable and responsible instructor, parent, peer - can achieve that goal with practice.  However, if your goal is to get a driver's license, it can't be done without taking the written test as well as the performance test.

    Funny...I've never heard anyone object to the high-stakes written drivers' license test or suggest that performance testing is enough.  Why is that?  The answer seems obvious.

    Ken, I think your drivers' license example undermines your argument and points us to some of the real-world issues:  attitude and motivation (both student and teacher), competence (both student and teacher) and a well-understood goal (both student and teacher).

    Every student I have ever known wants a driver's license.  They want to learn to perform driving, yes, but they want that license.  To do that, they have to pass the basics 'rules of the road' written test.  Nearly all do.  It's a rare student here in rural America who doesn't meet that goal...most, the first time they take the test.  Why is that?  Motivation.  Even a lousy driver's ed teacher can get a kid through that testing.

    Comments?

    Tell me how you spend your time and how you spend your money -- I'll tell you what your values are.

    by oldpro on Wed Jun 13, 2007 at 07:47:47 AM PDT

    •  you make several errors (1+ / 0-)
      Recommended by:
      annetteboardman
      1. it is not my example, but rather one the author's use
      1. the written test is insufficient - and it is actually fairly minimal as compared to the real driving test

      you seem to assume that the authors and I want no multiple choice tests under any condition.  Please demonstrate the evidence for that assumption.

      Such tests can serve a useful albeit limited purpose, but not that for which they are currently being used, which is as the sole and highstakes measurement.  

      Again, multiple methods/measures, and wherever possible imbedded in the instructional process.

      And in that sense the written test for driving is imbedded within the instructional process, since it is an intermediate step, or even a preliminary one.

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Wed Jun 13, 2007 at 08:37:23 AM PDT

      [ Parent ]

      •  Well, Hell, Ken...I make lots of errors! That's (0+ / 0-)

        how we learn, hmmm?  And heaven knows, I'm a lifelong learner.

        Picky, picky, picky...OK, it's not YOUR example but it is the one you passed on and used.  Pardon my leap.  I told you I needed more coffee (and more sleep, evidently).  Still...what's the problem with what I said about the example as it relates to something you didn't comment on - motivation?

        Don't interpret.  I may 'seem to assume' but I do not.  Assume, that is, that you and the authors "want no multiple choice tests under any condition."  No such thing.

        Personally, I'm not fond of 'multiple-guess' tests which I agree "can serve a useful albeit limited purpose."  As a teacher, I much preferred fill-in-the-blank or essay tests.  I agree with multiple methods/measures etc. etc. ad nauseam, whenever  possible -- as you say -- imbedded in the instructional process.

        I'm surprised and wondering at the hostile tone I think I detect in your response.  Is it me - my faulty interpretation?  (More coffee).  If not, what is the basis for it?  I'm perplexed...

        Tell me how you spend your time and how you spend your money -- I'll tell you what your values are.

        by oldpro on Wed Jun 13, 2007 at 09:25:35 AM PDT

        [ Parent ]

  •  In my old age... (3+ / 0-)
    Recommended by:
    teacherken, annetteboardman, cfk

    ...I've become more and more an advocate of project-based evaluation.

    Teacher's Lounge opens every Saturday between 11 am and noon. It's not just for teachers.

    by rserven on Wed Jun 13, 2007 at 10:09:16 AM PDT

    •  At night school, I took them through the task (2+ / 0-)
      Recommended by:
      oldpro, rserven

      first by modeling, then they practiced with a partner and group, then they did it by themselves.  I never sent the assignment home to do alone as it would never have been returned.  I was able to watch them do it so I knew they did it by themselves.

      But that takes time and I had 3 1/2 hour classes.

      Another assessment is the teacher seeing the student do the assignment called teacher observation.  

      Did they know how to do it?...ie listening skills and teacher instruction skills.

      Did they ask questions...good questions? an important skill, imo.

      Did they complete the task on time and as directed? I would think employers would value that skill.

      I had some high school kids come in for one term to make up a lost credit and they would try to sit and gab and complain they couldn't get the work done in the time given.  My night students who had had me before got right to work and got the task done and turned it in and it was well done.  The hs students noticed and got very quiet after that.

      "Other cultures are not failed attempts at being you. They are unique manifestations of the human spirit." Wade Davis

      by cfk on Wed Jun 13, 2007 at 11:22:03 AM PDT

      [ Parent ]

  •  A driving test is a good example (1+ / 0-)
    Recommended by:
    annetteboardman

    of a high stakes performance exam.  High stakes means that those who don't perform well are not allowed to go forward.

    However, the road test is also a good example of how high stakes tests corrupt the process of learning that underlies the exam.

    Very, very few, if not none, of the drivers who pass through a driving examination station would be ready to drive professionally--that is, have achieved a level of true excellence and mastery.

    It shows that high stakes exams result in a lowering of the bar to some minimum height that society is comfortable with calling "passing."  

    It says nothing about excellence.

    Education must be about achieving excellence and creating a learning institution that supports, fosters and produces excellence.  We are so far from that now.

    But, to start with, we have to acknowledge that there are limitations to the use of high-stakes examinations in terms of producing excellence.  The more people that have to scramble over the bar to advance, the more that bar will move lower and lower to the ground.

    Education? Teaching? NCLB? Read my book _Becoming Mr. Henry_

    by Mi Corazon on Wed Jun 13, 2007 at 11:35:31 AM PDT

    •  Of course, you're right about excellence and (1+ / 0-)
      Recommended by:
      annetteboardman

      minimum standards.  You're also right about reforming school culture ("It's the teachers, stupid!)

      I don't know, though, that the bar necessarily moves lower when you try to get everyone over it...is that the way it works in other countries?  Cuba, for instance?

      In theory:

      High stakes means that those who don't perform well are not allowed to go forward.

      In practice, however, reality is a funny thing...people who can't pass the driver's exams to get a license, drive anyway...by the millions!  (So do those who are 'punished' by having their license revoked). People who don't finish high school can go to college anyway...and thousands do.  People who don't finish college - or go in the first place - often do just fine in life (from Bill Gates to my local sheriff who makes twice as much as I ever did and both seem successful and happy as clams at high tide!)

      So, stigmatizing people with high-stakes tests doesn't seem to work so well...they don't stay stigmatized.  Not all, anyway.

      Tell me how you spend your time and how you spend your money -- I'll tell you what your values are.

      by oldpro on Wed Jun 13, 2007 at 01:57:40 PM PDT

      [ Parent ]

  •  Performance assessments really can drive teaching (1+ / 0-)
    Recommended by:
    teacherken

    One of my many pet peeves about the reliance on standardized multiple choice tests is the use of the phrase "data-driven instruction." It has been absurd to imagine that the kind of tests being given for AYP numbers, and the multitude of additional tests given to "prepare" for that test provide the kind of data that can actually inform teachers, much less inspire them.

    This proposal for performance assessment to be taken seriously is so exciting because it not only offers the possibility of providing everyone interested with real and valuable information, it must begin to drive the process of teaching toward helping kids develop the ability to "perform."

    Thinking, making connections, solving problems, working with others, knowing when they need help or guidance, acquiring the ability to access resources and so on become prized abilities because they are being measured! That means we teachers will be challenged to provide learning that contributes to the development of those kinds of qualities.

    Staff development, classroom materials, and professional conversations would all focus on teaching toward "performance."

    I think a lot of teachers would welcome such a shift (for many a return) with great joy.

  •  The comments are useful (6+ / 0-)

    The comments in response to teacher ken's posting of our work are very useful and I have enjoyed reading the discussion.  Maybe I can clarify a couple of things, though as tk will tell you, I am not good at this.

    First, out intent in getting this to lawmakers is NOT, believe me, to add new tests.  Rather, our intent is to make it possible for all those states and localities that had good performance-based assessments to get back to using them. The Department of Education has pushed all states to use standardized measures, measures that put bags full of money into textbook and test publishers pockets but tell us little about our kids.

    Second, we want to push back on the notion that a standardized measure is somehow better than tests that are used by teachers, at the point of instruction, and vetted by colleagues and peers.  There simply is no evidence, and if you know of some please post it here, that standardized paper and pencil tests are corrolated to performance outside of the test.  To get such a connection you either need to suppliment the test with performance (think bar exam followed by clerking or medical residencies, etc.) or replace it with the same.  This is especially the case for higher order skills, which NCLB CLAIMS it is about.

    Third, this is only one front on which to push back against the dumbing down of our schools when we focus only on test scores.  And before someone starts shouting about how some schools were already dumbed down let me say I know that--and that is why I left a tenured, full-professor position at a university 15 years ago to work as a school principal (and until you are ready to get in the middle of the work, you probably should not say too much).  But the evidence is in, see the stuff from both the Council for Basic Education and the Center for Education Policy (sorry, I don't know how to link to stuff) that shows that as schools have focused more on raising test scores they have cut back on the arts, student activities, literature, etc...especially in those schools that serve our most school dependent children.

    I am not against teaching to the test--my favorite teachers (Jim Leyland, manger of the Tigers; Scotty Bowman, legendary coach of the Red Wings; and Dean Smith, UNC Basketball coaching legend) taught to a test--a performance test.  The issue is not that we will all be tested, it is the quality of the test (or measure) dictates often the quality of the teaching and learning experience that leads to it.

    •  I am suggesting this as a top comment (1+ / 0-)
      Recommended by:
      soyinkafan

      so that it get some more visibility.

      Those who can, do. Those who can do more, TEACH!

      by teacherken on Thu Jun 14, 2007 at 07:44:31 AM PDT

      [ Parent ]

    •  Excellent commentary, George. A few thoughts: (2+ / 0-)
      Recommended by:
      teacherken, soyinkafan

      To boil the 'argument' re testing down to its most basic issues, the public thinks that performance in real life situations IS the important thing and that far too many kids have a graduation certificate but few basic skills or knowledge after 12 years of seat time.

      That is the basic issue which has led to 'minimum standards' and standardized testing ... a back-to-basics reaction which is understandable but not enough in the 21st century -- not nearly enough.

      We all know the horror stories, even lawsuits, of the kids who passed 'math' but can't make change...cannot read past primary level and some not at all...graduates with a diploma who cannot pass the college entrance exams or get a job at the local factory because they cannot fill out the forms successfully.  These stories are neither rare nor apocryphal.

      And, undeniably, students of color do not fare well in far too many public schools.  The gaps are real and must be addressed.    

      The public knows how much remedial work is necessary at the community college level, especially in math and reading.  They know that the school dropout rates have not improved significantly and are scandalously high in our country.  They know that adult literacy is not limited to immigrants whose first language is not English.  So,

      push back on the notion that a standardized measure is somehow better than tests that are used by teachers, at the point of instruction, and vetted by colleagues and peers.  There simply is no evidence, and if you know of some please post it here, that standardized paper and pencil tests are corrolated to performance outside of the test.  

      Push back is fine.  Performance tests are necessary and desireable.  So are paper-pencil tests. Balance is important and results must assure the public that their schools do not fail their children.  That is the most important test of all -- public confidence in public schools.

      Standardized tests are not going away any time soon so the question becomes...how to deal with them?  How to supplement them in a meaningful way with performance tests?  How to make sure standardized testing does not crowd out a liberal arts curriculum?  How to change habits and attitudes of everyone - from students to parents, teachers and employers -- from "Oh, no" to "Can do!"  How to reduce the 'threat/panic' level in educators whose schools are in danger of being labeled as failures?  How to change educators from fearful to proactive?  How to change without blame?  How to inspire?  How to train and retrain teachers?  How to evaluate them? How to recruit good teachers and keep them?  And so on...

      Your move to a principalship is admirable...even amazing.

      some schools were already dumbed down let me say I know that--and that is why I left a tenured, full-professor position at a university 15 years ago to work as a school principal (and until you are ready to get in the middle of the work, you probably should not say too much).

      "Getting in the middle of the work" is vital but that cannot mean that 'outsiders,' non-school empoyees (parents, employers, elected officials) shouldn't engage in the dialogue re 'how to improve education.'  As a former teacher, school board member, community college trustee, parent and student myself, for K-12 what I have decided (FWIW)is: "good principal, good school -- lousy principal, lottsa luck."  Your role, your leadership is vital to all progress.

      I'll bet you know that.  Others know it too.

      Tell me how you spend your time and how you spend your money -- I'll tell you what your values are.

      by oldpro on Thu Jun 14, 2007 at 11:42:57 AM PDT

      [ Parent ]

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site