Skip to main content

View Diary: Ten Reasons Why Value-Added Assessments are Harmful to a Child’s Education (107 comments)

Comment Preferences

  •  No, I meant what I wrote (1+ / 0-)
    Recommended by:
    You find, for example, that you can predict a 3rd grader's test result if you know the 5th grade teacher she was assigned to.
    (I assume you meant that the other way around.) Again--that's only a valid concern if the evaluation criteria are stupid. The criteria should measure how each teachers' students do relative to expectations:
    If you run the model using the 5th grade teacher's scores as an input to predict the 3rd grade scores, you find a strong correlation. The exercise is done as a test of the model: if it finds it can "predict" scores with information that cannot possibly have influenced the score, then it means you have not adequately controlled your variables.

    For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and classes that teachers teach. One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year. Thus, a teacher who appears to be very ineffective in one year might have a dramatically different result the following year. The same dramatic fluctuations were found for teachers ranked at the bottom in the first year of analysis. This runs counter to most people’s notions that the true quality of a teacher is likely to change very little over time and raises questions about whether what is measured is largely a “teacher effect” or the effect of a wide variety of other factors.

    A study designed to test this question used VAM methods to assign effects to teachers after controlling for other factors, but applied the model backwards to see if credible results were obtained. Surprisingly, it found that students’ fifth grade teachers were good predictors of their fourth grade test scores. Inasmuch as a student’s later fifth grade teacher cannot possibly have influenced that student’s fourth grade performance, this curious result can only mean that VAM results are based on factors other than teachers’ actual effectiveness.

    (I note that I misremembered that it was 5th grade to 4th grade rather than 3rd. Sorry for the error.)

    Fry, don't be a hero! It's not covered by our health plan!

    by elfling on Tue Mar 12, 2013 at 02:11:58 PM PDT

    [ Parent ]

    •  What that shows... (0+ / 0-) that internal politics rule at these schools.

      The 5th-grade teacher with the most political juice gets the best kids assigned to her class.

      And/or, the parents who are most vocal and involved (and therefore have the best-testing kids) push to have their kids assigned to certain teachers.

      Did you know that being elected President "predicts" that you went to an Ivy League school?

      •  I submit (2+ / 0-)
        Recommended by:
        Mostel26, ManhattanMan

        that it could mean a lot of things, some educationally appropriate and some not. It can also just be that standard result we see with standardized tests: that the strongest correlating variable remains household income, and that the resolution of the data available doesn't allow it to be pulled out.

        But it does suggest that typically teacher-student matchups are not random.

        I am not certain to what extent this was controlled for individual schools; I suspect not. Few elementary schools would be large or diverse enough to have enough 5th grade teachers to see this effect with much confidence.

        Fry, don't be a hero! It's not covered by our health plan!

        by elfling on Tue Mar 12, 2013 at 05:38:50 PM PDT

        [ Parent ]

        •  You only need two... (0+ / 0-)

          ...5th-grade teachers. In fact the effect is more pronounced with just two, because each teacher you add costs your regression model a degree of freedom.

          1) You have a young one with no political pull and no social power.

          2) You have an old one who is close friends with the Principal.

          The Old One asks to have Genius, Hardworker, Imaginative, and Polite assigned to her class. She sticks the Young Teacher with Stupid, Unmotivated, Violent, and Lazy.

          Sure enough, a regression shows that being assigned to Old One's class is a powerful predictor of good 3rd-grade performance.

Subscribe or Donate to support Daily Kos.

  • Recommended (127)
  • Community (60)
  • Media (32)
  • Elections (32)
  • Trans-Pacific Partnership (30)
  • 2016 (29)
  • Environment (28)
  • Law (28)
  • Civil Rights (26)
  • Culture (24)
  • Barack Obama (24)
  • Hillary Clinton (23)
  • Republicans (22)
  • Science (21)
  • Climate Change (21)
  • Economy (19)
  • Labor (19)
  • Josh Duggar (18)
  • Jeb Bush (18)
  • Marriage Equality (17)
  • Click here for the mobile view of the site