Daily Kos

THe state superintendent, Nancy Grasmick, met with the superintendents of all 24 school divisions in the state (23 counties and Baltimore City) and apparently got unanimous support in this action.  There has been legitimate concern about how long it takes to turn around the tests, primarily because the scoring of the constructed responses requires human eyeballs and takes time.  My students from last year have still not been informed of their scores, although I received a list about 10 days ago.  The state has switched to a new provider, and with the elimination of the constructed responses believes it can receive results in as little as 3 weeks, although with administering a test in May there is no real gain for the students sitting that year - if analysis of the scores were provided (so far none is given) it might enable those of us who teach courses tested by HSAs to modify our instruction before the start of the following school year.  That would be an advantage.

Let me quote the final two paragraphs of the article before offering some personal reactions, including an explanation of the title of this posting.

Making the tests completely multiple-choice could raise questions about how useful they are in evaluating mastery of the subjects. But state officials said that the exams would remain as challenging and accurate as before and that classroom instruction would not change.

"They now have a level of sophistication in the selected-response items they didn't have," Peiffer said. "The kinds of things we could only test with constructed-response items before now can be done in a valid and accurate way with selected-response items in a way that's just as good or better."

For that to happen, the state is going to have to severely improve both the quality and distribution across the curriculum of the items it tests.  I have in the past commented about items (questions) in my subject (government) that are inaccurate as phrased, having no correct answer or more than one correct answer, and have pointed out in a public hearing which included one of the state's top-ranking testing officials that one recent form of the government test had way too many questions on the economic areas, well beyond the state's guidelines for testing.

But there are more serious problems, and I wonder if there are not other issues.

First, the questions have 4 answers.   But there is as yet no correction for guessing as there is on the AP and SAT examinations.  On those, if there are 5 answers you get a raw score of the number correct minus 1/4 the number wrong.  Were you randomly guessing you would on average wind up with a raw score of zero.  Absent a correction for guessing the raw score received by a student may be inflated because there is no penalty for wild guessing and there may be a benefit.  I am thus not sure how such an approach provides an accurate measure of what the students know, even were the items individually and collectively an accurate and meaningful measure of what students know and can do.

Second - while there is no doubt that selected response (multiple choice) items CAN be sophisticated, the vast majority of items in my subject are at the lower if not lowest levels of Bloom's taxonomy, recall.  And given the ability to use process of elimination, even recall is not really required, only reasonable recognition.  Now, were life constructed of many situations where we get to pick one from four or five preprovided choices, then perhaps such a test might be a meaningful measure of what students have learned that is truly applicable enough to life to warrant our making high stakes tests.  But of course it does not.

And I strongly disagree with the first of the two paragraphs I have just quoted.   Given the high stakes of these exams, how they are constructed WILL drive instruction.  After all, besides the stakes for the students a propos their own graduation, the English and Algebra exams are used under NCLB for the purposes of determining AYP, and thus have exceedingly high stakes for districts, schools, and school personnel.  If you eliminate ANY writing requirement, including in ENGLISH for gosh sakes, you are inevitably going to see a decrease if not an elimination of the teaching of writing skills -  what is not tested will simply not be taught in schools where there is real concern about scores.

Totally unaddressed in this article nor in recent pieces accepting the state's bragging about scores on the test is that what is issued is a scaled score.  Like the SATs, a raw score of points gained or percentage correct is converted into a score along some arbitrary scale.  But the method of conversion has not to my knowledge been publicly released by the Maryland Department of Education.  Suppose I want to show my students "doing well" in my class -   I can theoretically take a test on which the median grade was 40% and by curving it  or redistributing the actual scores along some arbitrary measure present  report that says the median grade was B-.   i raise this point because of what I saw on this past year's test.  Granted, the state held out some kids who had failed a course for the year before they sat for the tests.  In my case, that applied to about 4-5% of my students.  The "pass" rate for the rest of my 10th graders was over 90% -  sounds like I am a great teacher, right?  The previous year my pass rate was in the low 80s for my 10th graders.  Even accounting or those not tested, I did not improve that much in my teaching between my 11th and 12th years of teaching.  I think, but cannot prove, that the supposed improvements reported by the state are at least in part - perhaps in large part - a result of manipulations of the conversion between raw and scaled score.  Why this past year? Because those students are in the class o 2009, the first for whom the tests really count - sudents in previous classes did not have to pass, merely sit for, the exams.

Why my title?  Our emphasis on testing ha been distorting our teaching, our schools, our educational processes.  But at last in MD, unlike Virginia, there was still some writing requirement, however minimal, that the students had to meet. Given the power the tests have to drive instruction, eliminating that requirement means we will now have even less instructional time on the basic academic and personal skill of writing.  Our students will have less opportunity to learn to express their own ideas.  

Better students who do not have to worry about passing the test will not see all of their instruction reduced to test prep, so long as they are in a school whose socioeconomics mean that most students arrive with sufficient background and skill to be successful in any environment.  But if you are in an inner city school whose students come from a deprived educational background, they will see their school time reduced increasingly to little more than preparing to take multiple choice questions.  We will truly be leaving them behind.  The elimination of any writing requirements on such tests can only serve to further narrow what many students in Maryland will learn.

I am both saddened and angry, but not at all surprised.  Things continue to get worse.

Peace.