Skip to main content

and want to be able to understand one of the key issues over which battles are being fought - including in the Chicago teachers' strike -  you need to understand Value-Added Methodology, and why it is NOT a silver bullet.

Fortunately for you, if you have 14 minutes, you can take advantage of the expertise of some professors oat Arizona State U (which has the nation's largest teacher preparation program -  full disclosure, I have done some consulting for them).

Here's the video.  I will offer some additional remarks in the tip jar

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  I want to point at a diary from several years ago (22+ / 0-)

    which will also help you understand the problems with VAM.  It is titled Problems with the use of Student Test Scores to Evaluate Teachers and it examines a policy brief put out by the Economic Policy Institute.  It is co-authored by ten of the biggest names in educational policy, and it examined all of the research then available (about 2 years ago) on Value-added.  I might add that there is nothing published since that undermines the conclusions drawn by the authors of that brief.

    "We didn't set out to save the world; we set out to wonder how other people are doing and to reflect on how our actions affect other people's hearts." - Pema Chodron

    by teacherken on Tue Sep 18, 2012 at 06:00:09 PM PDT

    •  Thanks, Ken (3+ / 0-)
      Recommended by:
      elwior, slatsg, Lujane

      I've not seen this... which will be very useful to pass on to the widget makers who think teaching must be evaluated similar to making widgets in a factory.

      "A civilization which does not provide young people with a way to earn a living is pretty poor". Eleanor Roosevelt

      by Superpole on Tue Sep 18, 2012 at 06:07:01 PM PDT

      [ Parent ]

  •  Okay, standardized test tracking (2+ / 0-)
    Recommended by:
    Lujane, ManhattanMan

    doesn't work. What does? How do we systematically identify the best teachers for promotion and raises, and the worst for demotions and firing? I think that's all people want. They want teaching job security and pay to not be mostly a function of seniority.

    •  your frame is all wrong (9+ / 0-)

      1.  Start by doing a better job of recruiting, training and inducting.  High Scoring Finland has a competitive admission process to teaching programs, and requires a masters and a gradual implementation

      2.  Do a better job of supervising people, including helping them when they struggle.

      Then your frame becomes irrelevant.

      And by the way, the current tests are poor, often badly constructed, and not even good measures of what students know and can do.

      And be clear on this -  most teachers do NOT want merit pay.

      "We didn't set out to save the world; we set out to wonder how other people are doing and to reflect on how our actions affect other people's hearts." - Pema Chodron

      by teacherken on Tue Sep 18, 2012 at 06:14:41 PM PDT

      [ Parent ]

      •  It is not my frame. (1+ / 0-)
        Recommended by:

        I'm a nobody. The public wants to see the same meritocracy they witness in their own private workplaces applied to public workplaces. It's what they know. The boneheads get fired, the top guns get rewarded.

        Every parent in the country knows two things - they know how great some teachers are, and they know how awful some teachers are. They sometimes witness great teachers, great teachers who are recognized as such by their principals, laid off while shitty teachers with more seniority are retained. That's just wrong, for everyone involved. Like I said, it doesn't have to be some Jeb Bush standardized testing method. But there's a vacuum, and the teaching profession would be wise to fill that vacuum with some acceptable alternative to today's seniority-based system (at least that's the system used in all the places I've lived).

        •  Yeah right... (9+ / 0-)

          Where have you worked where "boneheads" actually get fired and "top performers" get promoted?

          That's utter bullshit.  Every single place I've ever worked, it's the "boneheads" who get promoted to the top to get them out of the way, then they sabotage the "top performers" because they don't want to get shown up.

          Why don't we just let teachers teach instead of micromanaging ever aspect of their jobs?

          We all had some great teachers, a bunch of mediocre ones, and some bad ones.  

          That's just the way life is...even in Finland.

          •  My sister is a teacher. (1+ / 0-)
            Recommended by:

            Elementary school. She won some awards, was beloved by her students and the parents (and administration).  Layoffs came, strictly by seniority, by grade. She was fired while more senior teachers who hate teaching and are hated right back were left alone. She got her job back through a lucky break but this kind of thing happens all the time.

            I never said I was in favor of micromanaging teachers. I just asked what we can do to introduce more competition between teachers. It's not just me who wants this - that is why we've seen such movement toward the standardized tests. Because the public demands some accountability, and if there is only one idea out there being peddled, it will win by default.

            •  competition between teachers? (6+ / 0-)

              there's just no way competition between teachers makes better schools.

              The highest performing education systems, in fact,  reward collaboration -- between teachers in the same grade,  the same subject,  between teachers and college professors,  teachers and community institutions like libraries, social service agencies, parent groups.

              If Mrs. Jones has a great strategy for teaching long division and she won't share it w. Mr. Smith... who wins?

              It's not a fake orgasm; it's a real yawn.

              by sayitaintso on Tue Sep 18, 2012 at 07:15:20 PM PDT

              [ Parent ]

              •  If you announced that at the (0+ / 0-)

                end of the year, one teacher will be fired (out of an entire school) and one will be promoted, would you consider that competition? I just meant competition for being thought of as one of the better teachers. That does not have to mean competition between teachers in terms of knowledge and helping each other out. It could still be a collaborative environment. There would just be one evaluation period each year, and maybe during that short window - maybe just a week or so - there'd possibly be some infighting or sabotaging of each other, which is normal in any business environment.

                •  that would be idiotic (1+ / 0-)
                  Recommended by:

                  there are some schools where all of the teachers are good

                  there are others where few are

                  in the latter cases, that is because they may be the only teachers the administration can get to work in the school

                  you are trying to impose an after-the-fact nostrum that does NOTHING to address the problem

                  what guarantee do you have that replacements will be any better

                  sorry, but on this you do not know about what you are talking, and you totally misunderstand the real nature of education and teaching.

                  "We didn't set out to save the world; we set out to wonder how other people are doing and to reflect on how our actions affect other people's hearts." - Pema Chodron

                  by teacherken on Wed Sep 19, 2012 at 04:39:47 AM PDT

                  [ Parent ]

                  •  turnover at the top makes evaluation suspect (0+ / 0-)

                    I did not see, but my mother paraphrased Karen Lewis of CTU , saying that most Chicago principals turn over w. in 3-4 years,

                    In my area
                    1/2 NJ Superintendants are new

                    any useful evaluation of teachers has to be longditudinal
                    it has to be done by professionals who know the school environment, the community and the curriculum....
                    oh, but the curriculum changes with the speed of light, too

                    it's too complicated-- let's just fire people

                    It's not a fake orgasm; it's a real yawn.

                    by sayitaintso on Wed Sep 19, 2012 at 07:46:06 AM PDT

                    [ Parent ]

            •  I am sorry...but why does there need to be (3+ / 0-)
              Recommended by:
              tofumagoo, teacherken, Mostel26

              competition between or among teachers?
              As a Title 1 reading teacher...I was the only one in several of the schools I taught...who was I competing with?
              I tried many strategies/techniques/ideas/creative lessons/anything to encourage students to learn. I don't ever remember thinking about how to do something better than the teacher in the next room.....
              Are you referring to competition based on income?
              Most school districts have a scale with lanes and steps that are capped by years in teaching in the district as well as coursework beyond the degree. There were no bonuses for competitive teaching. The salaries were "capped" at either Master's  plus 45 or Master's plus 60. Honors and awards did not increase salary and salaries were frozen if no additional coursework was achieved within a certain period of time.

              I had a Master's plus 75 but it gave me no more money than a teacher who had a master's plus I am unsure what you mean by "competition"?

              I don't think you understand teaching. It is really not about the money. Money is nice...but, several times in my career, the union membership voted to freeze salaries in tight economic times for the it is not the prime motivator.

              Character is what you are in the dark. Emilio Lizardo in Buckaroo Bonzai

              by Temmoku on Tue Sep 18, 2012 at 07:50:43 PM PDT

              [ Parent ]

            •  Competition between teachers... (0+ / 0-)

              ...isn't a good idea.

              But competition between schools is an excellent one.

              I like VAM because parents can use them to find the best schools in their area.

      •  I think doc2 is right on-the-money. (0+ / 0-)

        You write:

        "Start by doing a better job of recruiting, training and inducting."
        How do we know what "better" is?  Do we recruit teachers with high SAT scores? Do we recruit teachers who are blond? What training methods work? How can we prove they work?

        Value Added is like the theory of Evolution. It does not explain everything. It does not even explain most things but it is the best tool we have so far.

        If teachers want to stop the "Accountability" steamroller (and since we are about to re-elect Obama, the steamrolling will continue) they need to present an alternate solution that works.

        If we are going to fire a teacher, I would rather do it based on hard numbers than based on the gut feel of some (possibly biased) principal.

        •  some colleges already screen (1+ / 0-)
          Recommended by:

          but not until the middle of the 2nd year of undergraduate by which time one has quantitative (grade) and qualitative( evaluations by professors) data on the applicants.  That is what University of Maryland College Park does with its undergraduate candidates for certification

          in the junior year they spend time on a regular basis - 1-2 days a week for a semester - in a classroom relevant to their certification area.  In senior year they student teach for about 1/2 the year.

          Sorry, but Value-Added explains nothing of value, especially when it is based upon tests that themselves are flawed and often inaccurate measurements.

          "We didn't set out to save the world; we set out to wonder how other people are doing and to reflect on how our actions affect other people's hearts." - Pema Chodron

          by teacherken on Wed Sep 19, 2012 at 09:01:46 AM PDT

          [ Parent ]

    •  Observe the teachers. (6+ / 0-)

      Track regular assessment--do the children make progress by THEIR standards?

      The problem with testing is that the tests are the same but the STUDENTS are different.  Some get anxiety, some have disabilities, and they ALL learn at different speeds and have different abilities.

      To TRULY evaluate teachers is a time consuming process to do it right.

      Because students are NOT repeat NOT business products or whatever.  Because Value-Added implies a SET PERCENTAGE of a "bad thing".

      "Nothing in all the world is more dangerous than sincere ignorance and conscientious stupidity." --M. L. King "You can't fix stupid" --Ron White -6.00, -5.18

      by zenbassoon on Tue Sep 18, 2012 at 06:17:22 PM PDT

      [ Parent ]

      •  The trouble is that students ARE BEING treated (8+ / 0-)

        as a product. This is a big reason why tests don't work as an evaluation tool. The other problem is that somehow we've latched onto a notion that schools need to produce a complete, finished "product". I'm sorry to say it doesn't work that way. What we need to produce is life-long learners and expose student to as many options that is possible (I was a HS teacher so this would be more practical at the HS level).

        Evaluating teachers is time consuming, but who evaluates the admn? In my tenure at my HS there were 11 different principals. By the time I retired I had "child" admn telling me what to do. I had been at the school longer than they had been alive.

        Only the weak & defeated are called to account for their crimes.

        by rreabold on Tue Sep 18, 2012 at 06:48:12 PM PDT

        [ Parent ]

      •  Peer observation (4+ / 0-)
        Recommended by:
        zenbassoon, bmcphail, Temmoku, Mostel26

        let teachers observe each other, it actually works

        I take political action every day. I teach.

        by jbfunk on Tue Sep 18, 2012 at 06:58:20 PM PDT

        [ Parent ]

        •  IF the focus is on collaboration and improvement (2+ / 0-)
          Recommended by:
          Temmoku, Mostel26

          not on hiring/firing.  Peer observation properly viewed is a way to get input from someone who understands what you are trying to do, better, rotating someones.

          We are the principled ones, remember? We don't get to use the black hats' tricks even when it would benefit us. Political Compass: -6.88, -6.41

          by bmcphail on Tue Sep 18, 2012 at 07:28:31 PM PDT

          [ Parent ]

          •  ALL (3+ / 0-)
            Recommended by:
            Temmoku, Mostel26, bmcphail

            Observation should be based on improvemnet and informing pedagogy as a whole. Hiring/firing needs in general to be de-emphasized and only addressed in the most extreme of circumstances. On that issue Im sure unions would agree wholeheartedly.

            I take political action every day. I teach.

            by jbfunk on Tue Sep 18, 2012 at 07:42:13 PM PDT

            [ Parent ]

    •  They also want good learning conditions-- (9+ / 0-)

      Libraries, small class size, textbooks, resources, buildings that have air conditioning and non leaky roofs, and stuff like that.

      "Nothing in all the world is more dangerous than sincere ignorance and conscientious stupidity." --M. L. King "You can't fix stupid" --Ron White -6.00, -5.18

      by zenbassoon on Tue Sep 18, 2012 at 06:18:31 PM PDT

      [ Parent ]

      •  I believe that all those things are (0+ / 0-)

        very important too. But we have to recognize that when cities across the country are literally bankrupt, there is not enough money to go around. It's not just the schools that are shortchanged - it's the cops, the sanitation depts, etc. In many places there just isn't enough money, and the bond-issue spigot is dry.

        •  As in (4+ / 0-)

          Our country is in a state of crisis.  Teachers are not the problem.

          I take political action every day. I teach.

          by jbfunk on Tue Sep 18, 2012 at 07:02:52 PM PDT

          [ Parent ]

          •  Just because bad teachers are (0+ / 1-)
            Recommended by:
            Hidden by:

            not "the" problem doesn't mean that they aren't a problem. If your kid has ever been stuck with an idiot for a teacher you know that it is a real problem.

            •  Almost (2+ / 0-)
              Recommended by:
              JanetT in MD, susanWAstate

              every person who is a product of public schools had a bad teacher at some point, I had several.  We've all dealt with bad doctors, dentists, lawyers, managers....the list goes on.   The US for decades was a leader in education, BEFORE ed reform 10 years ago starting pointing to teachers as the problem. Real solutions include making future teachers major in their subject as I did, serve longer internships, focus on education classes during internship, have peer evaluations, and finally ina free market, be paid more for there services. You pay for what you get.

              I take political action every day. I teach.

              by jbfunk on Tue Sep 18, 2012 at 07:32:37 PM PDT

              [ Parent ]

              •  Okay, you're right, we do (0+ / 0-)

                have bad doctors, and bad dentists, and lawyers, etc.  Let's say they're all fucked up. Why does that mean that you deserve the right to be fucked up too? Why can't we try to do better than the other professions, however messed up they are? Especially with kids involved. Those are good ideas for getting better new teachers by raising the bar and increasing training, but it also sounds like we'd almost be giving up on the teachers we've already got (and their students). Who knows? These are complicated things. We all think we know something, and we always think we are right, but half the people are always wrong, and we're bound to be in that half once in a while.

            •  Bad teachers are as prevalent as bad (2+ / 0-)
              Recommended by:
              susanWAstate, jbfunk

              parents and bad administrators.

              You need to define your parameters....some people would not agree as to what constitutes a "bad" teacher. My argument is if there really is a "Bad" teacher, then where is the administration? There is a set evaluation and remediation policy to address "bad" teachers and if it is not being followed then, perhaps the issue is a "bad" administration which is too lazy to bother following procedure.

              People are too ready to blame unions who protect "bad" teachers...but that is not the real issue. unions don't evaluate teachers, the union and its membership agrees with the administration on a fair policy/procedure for evaluating teachers. The administrators must do the evaluations and the evaluations should not be post dated or written before observations are done or contain information that was not observed during the evaluation.

              Character is what you are in the dark. Emilio Lizardo in Buckaroo Bonzai

              by Temmoku on Tue Sep 18, 2012 at 08:10:48 PM PDT

              [ Parent ]

    •  How (4+ / 0-)

      Do we systematically identify good doctors, good lawyers, good stock brokers, good computer programers?  We don't, ask anybody in any industry and he/she will rant on and on about incompetence in said industry.  As Malcolm Gladwell discovered in his survey of people's measurement of good doctors, he discovered people like friendly empathetic doctors regardless of their knowledge, competence, and ability to diagnose.  Identifying the "best" teachers is a red herring and an illusion.  Just know that millions of teachers across this country who had the opportunity to make more money and have a more respected job instead chose as I did to teach and dedicate our lives to working with and educating young people.  This country used to believe in its teachers, it can again.  

      I take political action every day. I teach.

      by jbfunk on Tue Sep 18, 2012 at 06:57:05 PM PDT

      [ Parent ]

      •  'Angie's List' or something for teachers? nt (0+ / 0-)

        "Had we gone the invasion route, the US could conceivably still be an occupying power in a bitterly hostile land." -- George H. W. Bush, "A World Transformed," 1998 memoir (explaining why the US did not occupy Iraq in the 1991 "Desert Storm" war)

        by nuclear winter solstice on Tue Sep 18, 2012 at 07:37:11 PM PDT

        [ Parent ]

        •  But you don't pick teachers that way.... (3+ / 0-)

          Not the way you pick a dentist or a kitchen remodeling service.  

          The school hires, the school assigns classes.  Depending on the grade level and subject, a student may have no alternatives, or few -- if they need to or want to take that class.  

          But putting all the pressure (and blame) on teachers when there are so many factors that influence learning and how different children do with a subject is short-sighted.

          Not all kids learn the same way.  We need to find ways of broadening opportunity and approaches for those kids who need it, not punish teachers and students because students aren't all cast from the same mold or come from the same background.

        •  Angie (3+ / 0-)

          of Angie's List is a big time GOP donor BTW.  Education is simply not measureable in a clean simple method.  Testimonials aren't reliable because ever student reacts to a teacher in his/her own unique way.  How many times have you taken a friend's personal reccomendation only to be disappointed?  Teachers are only one component of a child's development.  Parents, neighbors, extended family, community, environment, economic stratus all play into each and every child's individual development.  

          Want real education reform?  Start with government guranteeing breakfast for every child every day under age 20 and enrolled in any school, public, private, or charter.

          I take political action every day. I teach.

          by jbfunk on Tue Sep 18, 2012 at 07:52:09 PM PDT

          [ Parent ]

          •  I only picked 'Angie' for the recognizable name, (0+ / 0-)

            and I understand it's a complicated problem and mine was a flip answer.
                My own daughter was so traumatized by the testing that she would come home from school in tears, but I didn't take her as seriously as I should have, either, basically because the extreme amount of testing had gone up frog-in-a-pot style, insidiously, between the time of her education and mine. Also, I was an over-achiever who loved taking tests and so I couldn't relate, but I see in the end that all this testing appears more detrimental than helpful.
               Perhaps there is a completely outside-the-box new solution that could involve teachers and admin doing some kind of shared evaluations using the new ways like these Lists- a revolution of some sort. I sure hope so.

            "Had we gone the invasion route, the US could conceivably still be an occupying power in a bitterly hostile land." -- George H. W. Bush, "A World Transformed," 1998 memoir (explaining why the US did not occupy Iraq in the 1991 "Desert Storm" war)

            by nuclear winter solstice on Tue Sep 18, 2012 at 08:26:14 PM PDT

            [ Parent ]

      •  Be careful of what you ask for! (0+ / 0-)

        We have a method of identifying, "good doctors, good lawyers, good stock brokers, good computer programers". It's called the Free Market. It often sucks.

        In all of those professions, the customer can choose which person they get. They can make choices based on any factor they want.

        (When I chose my lawyer, I did not choose the #1 Lawyer in the state. I chose a friend-of-a-friend who I trusted.)

        Parents have no choice of teacher. The government chooses for you.

        I would rather have numbers-based methods than touchy-feely subjective methods.

        •  Do (0+ / 0-)

          you really want the responsibility of having to go out and "choose" good teachers?  Good luck finding a good/affordable public/charter/private school.  The US for the past 60 years before NCLB passed on 2001 had the best, not perfect mind you, schools in he world.  We can again.

          I take political action every day. I teach.

          by jbfunk on Wed Sep 19, 2012 at 06:31:24 PM PDT

          [ Parent ]

    •  I get what you are saying... (0+ / 0-)

      and I think it is a very valid point.  (I have a teaching background, and my husband is currently an Instructional Coach and has taught high school science for 14 years.  He has also taught teachers at the college level.)  

      Those who have an anti-education agenda, who really do NOT want to see public schools succeed, will certainly not support any sort of common sense and effective system to evaluate and promote teachers.  Those methods are out there and usually involve self and peer evaluation along with administrative evaluation.  We can look to other countries that retain and promote good teachers and discourage (or fire) ineffective teachers.  It isn't easy, but it isn't rocket science either.  What will really improve education are changes that are common sense oriented, bottom-up sorts of ideas that are multi-pronged and take time, wisdom, and yes money and commitment to implement.  

      The bigger issue is that the anti-education folks WANT ineffective teachers to remain in their jobs, and indeed they want great teachers to get so poorly treated that they quite in frustration.  They then are able to use this real issue (we all know there are some teachers who really, really are a disservice to kids) to play the "see, it is broken, so lets throw it out and get a new toy" game.  It works well with the more rigid-minded voters.  Instead of coming up with creative ways to solve the problem, they want use a sort of divide and conquer strategy:  keep the public frustrated with teachers in general because of the "few bad apples' syndrome.

      So, even IF we came up with the most brilliant system that "got rid of bad teachers" and promoted "good teachers" it would be fought tooth-and-nail by those who gain from the status quo mess.  So, yes, we need to create a fair system in order to improve the education system and THUS develop more trust among the general public.  But that won't satisfy those who are out to destroy public education.  

      I totally agree that in EVERY profession or workplace their is a tendency for the boneheads to get promoted -- education isn't any worse in that way than other professions.  However, the bonehead problem is USED by the rightwing as a tool to bash the entire profession.  They could do the same thing with doctors and lawyers, too, if they wanted to.  

      My husband works with teachers who need/want to improve their skills.  He is VERY good at what he does.  Teachers and students love him.  He knows what good teaching is and he can communicate it to teachers who are open minded enough to listen and learn.  What we need to do is clone my husband!  :-)  

      But seriously, teaching is a high art -- and so is learning.  There are so many awesome ideas out there, so many great teachers, parents, kids!  We need to stop blaming AND we need to take responsiblity for our profession and work towards a quality evalution/promotion system that includes REAL accountability.  

      I think real accountability that is fair is possible.  But anything that is top-down, data-heavy, and one-size-fits-all is not to be trusted.  Thus, the defensive posture you will see from teachers when this issue of "bad teachers being held accountable" comes up.  

    •  After teaching for over 20 years (0+ / 0-)

      I say ask the students. They are not idiots. Even from a young age students can identify effective teachers. I mean this most sincerely. The other thing is test scores are only a small part of the equation. "Soul Scores" are for more important. Teaching the youth to critically question this life and this world in which we live is paramount. Taking the white supremacist worldview out of American education could also produce positive change.

      But what do I know?  My contract was non-renewed three years ago. But I've got a "teacher grand baby" turning three next week. I was there in the delivery room with her mama welcoming her into this world. Ill be walking another student down the aisle soon. There are some aspects of measuring teacher quality that are most important, yet least considered.

      **I've got Magic Ladyparts**

      by consciousempress on Fri Sep 21, 2012 at 08:59:14 PM PDT

      [ Parent ]

  •  The video was informative (0+ / 0-)

    Thank you for posting it.  I also found it entertaining except that the narrator's voice was irritating.  Where does 'irritating voice' fit into teacher evaluation criteria?  : )

    We're ALL better off when we're ALL better off!

    by susanWAstate on Tue Sep 18, 2012 at 09:13:06 PM PDT

  •  The video was intellectually dishonest. (0+ / 0-)

    I have not read their full study, but the video was eye-popping.

    4:45 -- Video points out that principals' evaluations diverge from VAM scores.  This actually bolsters the case for VAM. If principal evaluations are so slippery and changeable, how can we rely on them?

    5:31 -- It gets worse. The video admits that principals evaluations are so slippery and subjective that they will change their evaluations to match scores. (Doesn't this contradict 4:45? No matter. it's a cartoon...) The video chooses to ignore a simple solution: Don't release the VAMs until the Principals scores are in.

    6:21 -- Video borrows a page from Mitt (Harvard Business School '75) Romney's alma mater and goes for the Case Study Method. They have mountains of data available, and they choose this unreliable method to test their assertion.

    Why? So they can commit another statistical error: Sampling on the dependent variable. They choose good people who got fired and ignore good teachers who got promoted or bad teachers who go fired.

    To put this into a perspective that will resonate with Kossacks, let's remember Ronald Reagan's "welfare queen" who supposedly ate steak and drove a Cadillac. Even if she existed -- and she probably did -- that did not mean the whole welfare program was corrupt.

    But the human brain is funny. One vivid example will outweigh a million data points. Reagan used this trick to win elections. Now this video uses it to smear VAM.

    6:50 -- OMG, we now learn that these people were only fired "in part" because of their scores. What other heinous acts did they commit? It's not stated. Even the case studies have incomplete data, so nothing is proved. But it is nicely animated and will probably fool many.

    8:00 -- Now we get to the most inexcusable bit of chicanery. The video claims that certain kids are more likely to generate low VAM scores. The video claims to know how to identify these kids. Once again the video chooses to ignore a simple solution: Just add this data to VAM and strengthen the model. This is the "House Republican method" of problem-solving:

    - Complain something is broke
    - Cite reasons why it is broke
    - Do nothing to fix it
    - Try to destroy it instead

    10:30 -- The video points out that many teachers leave because they get bad VAM scores and that they don't get counseling and training. This is not the fault of VAM -- the same thing happens with traditional evaluations.

    I guess that I am especially bitter-minded because this video was produced by a research institution. They are supposed to know better. It is not like this came from a union or even Super-PAC.

    Accountability measures are a very good idea. Teachers keep throwing up random objections, and every time these objections are met, they throw up more:

    + When tests were proposed, teachers said the tests were flawed.

    + When better tests were designed, teachers said some kids were easier to teach.

    + Now VAM is accounting for the fact that some kids are easier to teach.

    The objections are getting flimsier. The time for accountability will come soon. The only question is will teachers help design the systems (which will make them better) or be dragged kicking and screaming (which will leave us with bad blood and bad systems).

    •  just because you disagree (1+ / 0-)
      Recommended by:

      does not make the video as you describe it

      Had I time - i don't. I am on the road to volunteering for an important Congressional campaign - I would obtain permission to quote from comments on the film by several experts on assessment who have seriously studied Value-Added and found it very wanting.

      In fact, you would be hard put to find recognized experts on psychometrics other than those with financial connections to value-added approaches who approve of it.  Wonder what that says?

      As for "accountablity" -  teachers are only one part of the equation, and as much as we are allowed we take our responsibilities very seriously.

      Instead we have an approach where we will be told what and how to teach, and then punished when it does not produce the desired results -  a failure about which we warned you up front.

      "We didn't set out to save the world; we set out to wonder how other people are doing and to reflect on how our actions affect other people's hearts." - Pema Chodron

      by teacherken on Wed Sep 19, 2012 at 09:04:40 AM PDT

      [ Parent ]

      •  No matter which... (0+ / 0-)

        ...of the many specific VAM systems we look at, there will be flaws. Recognized experts will rightfully point them out.

        My point is that these flaws need to be addressedand fixed. They should not be used as an excuse to scrap the whole idea of VAM.

        What the teacher's unions are doing is this:

        1) They let a district propose a specific system.
        2) They say, "Gotcha! That system has flaws!"
        3) Instead of working to fix the flaws, they use them as an excuse to delay another year.
        4) Repeat step #1

        After the election we will have to sort this out. I have been canvasing in rural areas where education is not seen as a big issue. But we must stop delaying, people are getting impatient.

        •  you assume they are fixable (1+ / 0-)
          Recommended by:

          many psychometricians would disagree

          and all VAM does when accurate is address variance on students scores on tests designed to measure student knowledge, note ven student growth

          and as I have pointed out before, all 3 professional organizations dealing with educational measurement,  NCME, AErA and APA, warn that a test designed to allow you to draw valid inferences for one purpose - in this case what students know - ucannot be used to draw valid inferences for another purpose - in this case the teacher impact.

          "We didn't set out to save the world; we set out to wonder how other people are doing and to reflect on how our actions affect other people's hearts." - Pema Chodron

          by teacherken on Wed Sep 19, 2012 at 01:17:51 PM PDT

          [ Parent ]

  •  This conversation has also overlooked... (0+ / 0-)

    the responsibility of the student to do their part. You ask any teacher with a difficult class and often it comes to a select few students that are disruptive and don't want to be a part of the process.

    For the criticss out there, you should spend time in front of a class that YOU are responsible for, day in & day out for an entire school year. That will give perspective on how to make reforms.

    Unfortunately if you do this you won't be asked for input because teachers have been routinely ignored or excluded when iy comes to reform.

    Only the weak & defeated are called to account for their crimes.

    by rreabold on Wed Sep 19, 2012 at 08:11:38 AM PDT

  •  Prisons and schools and hamster wheels (1+ / 0-)
    Recommended by:

    In my IT department, we all know who the weak links are, and when the layoffs came, it turned out that our management knew who the weak links were, too, and we were all mostly in agreement.    Teachers and principals work together for several years, parents complain, teachers notice that kids coming from a certain class are less prepared, principals notice that a particular grade level doesn't seem to perform as well as other grade levels and tracks it back to a particular class.   Teachers and principals are on the ground, so they automatically adjust for a bias issue such as ESL.    Teachers and principals work together and talk to each other, and word gets around.   The grapevine is an effective tool.   Principals shift the less effective teachers into less critical slots.  It's how offices work.

    The first question that I wonder about, however, is whether teacher effectiveness is really the most significant factor affecting educational outcome.  It seems like common sense, but my own observation of my kids and their school lead me to different conclusions.

    It seems to me that kids have a tendency to have a default level of performance (the speed of the car) and expecting teachers to increase that level of performance in each class (such as achieving a constant increase of acceleration) is not realistic.   There are limits.  You cannot exceed the speed of light, nor the physiological limits of the ten year old brain.   It's like the stock market expecting that a corporation can sustain increases of 10% over last year same store sales, EVERY year.   It's crazy.   What goes up, must come down.   You can't sustain infinite acceleration.    At some point, you need to reach a cruising speed, and settle in for the ride.

    To me, the fairest question is, "Did the student learn what they were supposed to learn?", not "Did they learn faster and more this year over last year?"  -- because, where does it end?    The video alluded to this issue with the gifted and talented kids, who were already performing at such a high level there wasn't much more that you could get out of them.  They were already "accelerated".  How much more acceleration are you going to get?  

    The focus on teacher effectiveness feels a lot to me like the focus on prisons for all of society's ills.   It's too simplistic.  It disregards all of the other critical variables, some of the very same variables -- parenting, poverty, drugs, mental health issues, language and culture, socialization, etc.   If you ignore all of these other factors, you can keep throwing people in prison, and tuning teachers like crazy, the effects are going to be similar to running in a hamster wheel -- you'll go faster and faster, but you won't ever get anywhere.


  •  ... (1+ / 0-)
    Recommended by:

    I have taught in an upper middle class school with a lot of parental support for education and am now teaching in a Title 1 school.  If you graded me by my students' test scores, you would get two very different evaluations.  I am actually in the same school building, but the community has changed that much in 30 years.    

    What I wish people understood is that, having taught in both environments, the teachers I work with now work HARDER to get the scores that some would consider mediocre in the school where most students are on free lunch and it isn't unusual to see a child walking to school in the winter with no coat than we did before in the privileged school.  

    It is harder to work with children whose parents may not be educated and don't know how to help them or others who don't support the school or teachers.  Some parents work so much the students practically raise themselves.  

    The students who get the great scores probably do have good teachers, but even if they get a "bad" one, they will care enough and their parents will care enough that they will study and get tutors and will overcome it.  They are easier to teach and they are easier to get to produce good test scores.  

    The saddest thing about the evaluations on test scores is that good teachers won't want to go to the poor schools because they are far more likely to be fired, even though they are working hard and well.  Noble intentions are good, but when you have a family to support, you have to consider being in a job where you will be punished for the circumstances not under your control.  Guess who that leaves to teach the kids that need help the most?  

  •  ... (1+ / 0-)
    Recommended by:

    One thing that seems obvious to me:  if test scores are a measure of the quality of my teaching, why do the grades on a test vary so much in a class that I taught together at the same time?  If my teaching was that closely tied to their scores, wouldn't the group test at a level that was at least close to each other?  

    One teacher in Chicago said she had 43 students in a kindergarten class without enough desks for them.   Does anyone honestly think the test scores for that group will be as high as the class of 18 or 20 in a room with many resources and individual time with the teacher?  Is it her fault?  

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site