Skip to main content

Teacher quality is a growing national and state concern in education reform, including teacher evaluation and pay scales. That trend can be traced to and remains reflected in the Los Angeles Times repeated publication of value-added models (VAM) of ranking teacher quality.

Catherine S. Durso’s National Education Policy Center review of VAM results calculated for and reported by the Los Angeles Times exposes that once again linking teacher quality to student test scores fails to identify fairly high-quality teachers. The body of evidence shows that test-based teacher evaluation is unstable and counter-productive to evaluating, paying, and retaining teachers, explains Durso:

“Many researchers have raised concerns regarding the use of value-added models (VAM) for teacher evaluation. Briefly, VAM do not provide guidance for improvement, are comparative rather than absolute measures, assess a small part of teacher’s responsibilities, force different kinds of teaching into one scale, do not produce consistent results for given teachers over time, and may not identify effects actually caused by the teachers.”

Test-Based Teacher Evaluations: A Failed Solution without a Problem

These conclusions about VAM in California and throughout the country are powerful but ignored evidence in states across the U.S., including my home state of South Carolina, now poised to link 40% of teacher evaluations to student test scores, policy as misguided as SC’s commitments to experimenting with students through expanding charter schools and hiring Teach for America (TFA) recruits.

And just as no solid evidence supports increasing faith in charter schools and TFA, test-based teacher evaluation should have no place in education and teacher quality reform in SC. Let’s consider the many reasons that addressing teacher quality through test scores is a waste of precious time and resources for any state at the expense of their students and teachers:

• As noted above, the relatively new but growing body of evidence on the validity and reliability of test-based teacher evaluations reveals that data are unstable [1]; in other words, as the populations of students change or the school settings change, the rankings of the teachers fluctuate. Test-based teacher evaluation can be of value only if it can offer a stable message that a teacher is strong or weak. If that label isn’t predictive, it has no positive contribution to policy and personnel decisions.

• In order to implement test-based teacher evaluation that shows student growth, SC will have to create and implement two tests per course for every teacher in the state and in every content area taught. Since there is no compelling evidence test-based teacher evaluation data are stable or valid, this investment in time and money is a catastrophic failure by our state leaders. How will we identify growth in music, P.E., and art, and how can we justify the costs associated with generating all of these tests?

• Test-based teacher evaluations create a competitive environment in which teachers must choose between the welfare of their students and their own professional security as that becomes threatened by the outcomes of other teachers and their students. In effect, each teacher must seek to use her/his students against the outcomes of other teachers’ students for their own personal gain. Education is best served by a collaborative, not competitive, environment.

• Test-based evaluations of teachers place far too much weight on flawed assumptions. First, high-stakes testing distorts how well any tests reflect student learning. Next, high-stakes testing decreases the quality of both teaching and learning since it encourages teaching to the test. Further, linking teacher evaluations to student outcomes confuses “high-quality teaching” with student test scores, although we have no way to insure that teacher quality is always positively correlated with those scores. And finally, test-based evaluations of teachers imply that student outcomes are or can be linked only to the teacher’s room that any student sits in when the test is administered. Here is the great failure of test-based evaluations of teachers: Data linked to a student are correlated with dozens of conditions (that cannot be controlled for) that distort any one teacher’s quality.

Claiming that teacher quality is central to education reform is a powerful and compelling message—although teacher quality is overshadowed by out-of-school factors. As I have noted about committing to charter schools, SC must stop pursuing solutions without identifying the primary problems, which include the inequity that creates poverty and the inequitable distribution of high-quality teachers.

Test-based reforms to teacher evaluation, pay, and retention have no place at the education reform table in SC since there is no evidence it is needed or that it works. To pursue test-based teacher evaluation while the state continues to struggle economically and educationally is an inexcusable failure of state leadership.

[1] See the evidence compiled HERE.

* Submitted in a slightly different version as an Op-Ed to The State (Columbia, SC), The Greenville News (Greenville, SC), and The Post and Courier (Charleston, SC).

Originally posted to plthomasEdD on Wed Jun 20, 2012 at 11:26 AM PDT.

Also republished by Teachers Lounge.

EMAIL TO A FRIEND X
Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags

?

More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  Thank You - N/T (3+ / 0-)
    Recommended by:
    chimene, slowbutsure, slatsg

    "Upward, not Northward" - Flatland, by EA Abbott

    by linkage on Wed Jun 20, 2012 at 11:54:45 AM PDT

  •  One of the "solutions" to the (4+ / 0-)
    Recommended by:
    FloridaSNMOM, chimene, slowbutsure, slatsg

    problem of art and PE is to... develop bubble tests for them. Talk about losing your way. Having the kids take and study for pointless tests just so we can assign a numeric score to the teacher? Let's just get out some dice and be done with it.

    Fry, don't be a hero! It's not covered by our health plan!

    by elfling on Wed Jun 20, 2012 at 12:07:38 PM PDT

    •  Or a Ouiji Board or Crazy 8 ball... (1+ / 0-)
      Recommended by:
      slatsg

      ...Son, those Elephants always look out for themselves. If you happen to get a crumb or two from their policies, it's a complete coincidence. -Malharden's Dad

      by slowbutsure on Wed Jun 20, 2012 at 04:53:57 PM PDT

      [ Parent ]

      •  Worse than this, (1+ / 0-)
        Recommended by:
        elfling

        Bill Gates is behind a study of "mood" bracelets for students to wear that will measure their "engagement."  He wants to show videos of teachers to see what works.  There are so many things wrong with this it is hard to believe he can't see it.

        1.  Students will react  differently to a film than to a human being.

        2.  Students learn in different ways--they are not a product on an assembly line.

        3.  How does  he know that the super-high score of engagement on the hormonal teenager's bracelet is a response to the lesson and not to the hot girl sitting beside him?

        The engagement that works with students is a personal relationship with a teacher that clearly has their best interests at heart--a person who tries different methods with different students in order to find what works for each one.  

        I read that Arne Duncan was amazed that most teachers who are being graded by their students' test scores don't want to transfer to at-risk schools to help those students.  Is he crazy?  Does he think the teachers want to be fired because of what might be lower scores than the wealthy schools have, even though they may reflect more hard work on the teachers' parts and an improvement for the students?  

        If he is that clueless, he needs to step aside and let someone who has actually taught in public schools take his place.  

  •  I hate the standardized testing... (1+ / 0-)
    Recommended by:
    slowbutsure

    and that's part (granted a lesser part) of the reason we don't use the public school system any longer. But, it would be more useful and more fair to the teachers if they would base things on the progress of each individual student from year to year rather than on a standardized score. It would be a better system for seeing how teachers are doing, would give teachers a better judge of what they personally could improve on, and wouldn't penalize teachers for having special needs students in their classrooms. Still shouldn't be used to punish school districts or teachers, but it would be more useful than what they're doing now.

    "Madness! Total and complete madness! This never would've happened if the humans hadn't started fighting one another!" Londo Mollari

    by FloridaSNMOM on Wed Jun 20, 2012 at 12:43:12 PM PDT

  •  Reason is Trumped by Other Agenda's (2+ / 0-)
    Recommended by:
    slatsg, iTeachQ

    The evidence is clear that standards based testing is distorting education and not leading to any improvement. After all, we are not creating "Jeopardy" champions, we are creating thinkers.The evidence that accountability driven education reform is a failure is abundant. The charter movement is almost exclusively about privatizing public schools. There is no improvement coming from charter school education. There are no silver bullets that magically improve education, but you are putting children at risk in schools with no track record. There has never been any evidence that value added measures are reliable yet politicians promote them. Smart people keep promoting these failed or unreasonable policies. Why? Clearly the primary goal is not to improve public education. However, destroying public education and replacing it will create a giant new profit center and that is a theory that explains what is happening. Get those tax dollars into private coffers.

    •  Thank you for exemplifying US anti-intellectualism (0+ / 0-)

      in your contemptuous dismissal of people who are not ignorant as "Jeopardy" champions.

      Knowledge and thinking go hand in hand.  One reason smart people keep promoting these failed policies is that smart people know that their intelligence and their knowledge are tightly-coupled, feeding back on each other continuously, and they are not impressed with analyses of reform that begin by dismissing the value of knowledge.

      To put the torture behind us is, inevitably, to put it in front of us.

      by UntimelyRippd on Wed Jun 20, 2012 at 03:04:12 PM PDT

      [ Parent ]

      •  If they continue to promote failed policies (1+ / 0-)
        Recommended by:
        iTeachQ

        that can't be all that intelligent.

        A proud member of the Professional Left since 1967.

        by slatsg on Wed Jun 20, 2012 at 06:05:01 PM PDT

        [ Parent ]

      •  Contemptuous Dismissal (0+ / 0-)

        Drilling students on facts without teaching thinking is bad pedagogy. For example, I dout that any of this years Presidential candidates remember the quadratic formula, but they all know how to diagnois a problem and formulate a response. The second skill is much more important. Forcing students to memorize facts for a multiple choice test undermines the desire to learn. We want life long learners not people who have predominately been drilled on facts. Students know those facts are not really that important. They know successful people do not remember many of the facts they learned in school and never use the quadratic formula. Knowledge is only important if it can be applied intellegently. Knowledge is the bottom of Bloom's taxonomy of learning domains. In other words, knowledge is important but it is not the goal.

        •  Prove it. (0+ / 0-)
          The second skill is much more important.
          you assert it.

          but you cannot prove it.

          you cannot even prove that the two skills can be meaningfully teased apart.

          To put the torture behind us is, inevitably, to put it in front of us.

          by UntimelyRippd on Thu Jun 21, 2012 at 03:18:28 PM PDT

          [ Parent ]

          •  you have no proof for your contentions either (0+ / 0-)

            Education is a social science. There are no objective proofs. I do however think that people who have spent a lifetime studying pedagogy should be given some credance.

            •  In case you haven't noticed, they don't (0+ / 0-)

              all agree with each other.

              Educational psychology, like pretty much all subfields of psychology, is more about the cherished dogmas of the "scientists" than it is about anything else. They can dress it up as science if they like, but few educational studies have the necessary power, controls, and methodology in order to demonstrate anything at all, and it's really more about ideology than anything else.

              As to proof of my contentions, you are correct, I have no firm and incontrovertible proof. I cannot even prove the anecdotal case, which is that I and the very smart, very knowledgeable people with whom I work must rely daily, and even hourly, on an extensive very deep, very broad knowledge base in order to think creatively and productively about the problems we are paid to solve.

              I can assert, however, categorically and without any self-doubt, that there have been many times in my life when my ability and willingness to plow through enormous amounts of information, much of it excruciatingly uninteresting, has been fundamental to my being able to do what I do. The ability to obtain and master large amounts of readily accessible mental data isn't a parlor trick, it's a powerful and valuable talent and skill, and only somebody who can't do it, or who is in a most peculiar variety of ideological denial, would argue that it isn't.

              To put the torture behind us is, inevitably, to put it in front of us.

              by UntimelyRippd on Thu Jun 21, 2012 at 07:03:39 PM PDT

              [ Parent ]

  •  It is no (0+ / 0-)

    coincidence that the testing companies are making a LOT of money on these tests and test preparation materials.  Is it a coincidence that some of the execs of the companies are friends of the Bush family?  Pearson?  

    It makes me sick that when school budgets have been cut so deeply,teachers have lost jobs, and class sizes have grown,  that millions are spent on these tests.  

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site