Statisticians! pollster experts! teachers! and all the others! Lend me your ears and tell me if I'm crazy.
I think that test is fake.
Perhaps no test was actually administered and the 'results' have been simply invented.
Perhaps there really had been a test, but the responses have been manipulated.
Follow my reasoning.
I don't want to get into details about what sort of an organization OCPA is and other questionable 'information' it has disseminated in the past. Someone else can talk about their ongoing feud with the Oklahoma teachers' union.
I'll simply repost this mission statement from their webpage:
WHO WE ARE
Oklahoma Council of Public Affairs (OCPA) was founded in 1993 as a public policy research organization focused primarily on state-level issues. OCPA has been part of an emerging, national trend of conservative, state-based think tanks. The founders, led by Dr. David Brown, envisioned an organization that was capable of affecting the state’s public policy similar to national level think tanks, such as the Heritage Foundation. Throughout its 15 years of existence, OCPA has conducted research and analysis of public issues in Oklahoma from a perspective of limited government, individual liberty and a free-market economy. OCPA has promoted the conclusions from its research through an array of media that have steadily increased in breadth, scope and ultimately, effectiveness. Today, within the arena of public policy and politics, OCPA is regarded as "the flagship of the conservative movement in Oklahoma."
MISSION STATEMENT
OCPA's mission is to accumulate, evaluate, and disseminate public policy ideas and information for Oklahoma consistent with the principles of free enterprise, limited government, and individual initiative.
So this survey was commissioned by a conservative 'think-tank' with an axe to grind against centralized public education as inefficient and badly run. And look, they commissioned a survey with shocking results for you that will instantly convince you that they are right! How surprising and unexpected.
Here is why I don't think the results of this test are legitimate.
Let's just take this question for now: "What are the Two Major Political Parties in the United States?"
43% of the kids supposedly answered "Democrat and Republican"
11% answered "Communist and Republican"
46% answered "I don’t know."
Nothing about this makes sense.
- The 46% of ‘know-nothings’ is WAY too high. Even if you don’t know or can't recall on demand the author of the Declaration of Independence, there is no escaping constant, 24-7 political chatter on TV, on the Web, or between members of one’s community. And that chatter will almost always have the names of the parties in them. The answer to this question is part of the passively acquired background knowledge of Americana - no one would ever need to look it up or get it taught to them unelss they watch no television, listen to no radio, associate with no one and have no knowledge or memory of last year’s elections. It’s close to impossible that this could be the case for 460 kids randomly selected of 1000.
- There are no other options listed beyond these three. Given 1000 kids with such an apparent lack of standard knowledge, they should have gotten all sorts of options beyond "Communist and Republican." For instance, Libertarians aren’t anywhere in there, and you’d think they would be, in such a red state. And what about "Socialists"? If so many kids had been indoctrinated by their folks to consider the Democratic Party "Communist," then at least some tangible percentage would have had their parents use the word "Socialist." Where is the comparable 10-11% listing the non-Republican party as "Socialist"?
- These three options add up to the full 100%, as though no one answered anything beyond what got listed. And this goes for every other question in the survey as well! A bit irregular, if you ask me.
- The question actually has a wrong answer listed as correct: "Democrat and Republican" instead of "Democratic and Republican." And the 43% of the answers labeled as "correct" make the same mistake.
Some possible reasons for this
a) Not one among the surveyed kids actually called the party in question "Democratic," either out of universal contempt or total ignorance.
b) The same brain came up with both the question and the answer.
c) The pollster or OCPA lumped all variants of "Democrat" together into one, manipulating the response data to fit the predesignated and incorrect answer.
And that's just one question! Just to toss a few more thoughts out about the others:
- In the question about "the Supreme Law of the land," 4% listed the "Gettysburg Address," 2% listed the "Monroe Doctrine," and yet no one seems to have listed the most obvious wrong answer of all: "The Bible." This makes sense if whoever was writing these answers isn't actually in habit of thinking of the Bible as a wrong answer to that sort of question.
- 2% of the respondents to the test said Michael Jackson wrote the Declaration of Independence, and 7% said Barack Obama wrote it. That makes for a minimum 9% of obviously disingenuous respondents - not entirely unexpected with the surveyed pool being high school students. What
is surprising is that the listed margin of error for this poll is simply "plus/minus three percent," with nothing mentioned about the percentage of 'punkers.'
- Not a single person out of the 1000 surveyed got more than 7 out of 10 questions right. You would think a nice solid sample size like 1000 people would have at least one or two outliers. Even forgetting 'straight-A-student' 10-out-of-10 outliers - what, not a single one of these kids could score an 8 out of 10? That's only 80%, a B-! Not one of these kids could eke out a B-? Maybe a teacher or a statistician will correct me on this, but that seems unlikely.
Et cetera. I call shenanigans.
UPDATE: Anyone think we should contact Nate Silver of FiveThirtyEight to toss his two cents in?
UPDATE: frozenporklander pointed out something very curious about question 8: "How many Justices are on the Supreme Court?"
Here is the breakdown of answers:
5 Justices - 5% (50 people said this)
6 Justices - 11% (110 people said this)
7 Justices - 0% (no one said this)
8 Justices - 15% (150 people said this)
9 Justices - 10% (100 people said this)
10 Justices - 21% (210 people said this)
11 Justices - 0% (no one said this)
12 Justices - 7% (70 people said this)
Apparently, not one person answered 7 or 11. All other numbers from 5 to 12 are covered. How did that happen?