For the second time in three years, a diary sits atop the rec list that showcases a Harvard study comparing porn consumption rates across states, with some intriguing findings that flatter our ideological predispositions.
Unfortunately, a careful read of the actual study shows that it does not, in fact, make a convincing case that conservatives/Mormons/whatever are heavier-than-average porn consumers.
This diary, a modified reposting of one from 2009, is about social science – it is not meant as a direct refutation of the original diary, which is a funny, light-hearted jab at the wingnuts trying to control our reproductive decisions. While I fully support and encourage such attacks, I find it useful nonetheless to explore the social-scientific flaws in the predominant interpretation of the Harvard study.
Both in 2009 and now, the most common take-away from the study by progressive bloggers is an individual-level relationship between certain attributes (mainly Mormonism and general social conservatism) and porn consumption. In fact, I would guess that this potential relationship is the only thing that makes the study interesting to most readers (except for those instinctively attracted to any diary with “porn” in the title. Guilty as charged).
Unfortunately, this asserted relationship is a textbook example of the ecological fallacy. The ecological fallacy occurs when a researcher (or reader) attempts to make individual-level inferences from group-level data. For the sake of example, I’ll give an obvious (and hypothetical) case: imagine a study from the 1920s that showed a higher murder rate in states with larger African American populations. An ecological inference from this data could be that “African Americans are, therefore, more violent than whites.” But obviously, we don’t know that from the data. Sure, it could be true. But it’s much more likely (in fact, about 100% more likely, given what we know about history) that the murder rate is higher because blacks are being murdered by whites.
A more recent example is explored in Andrew Gelman’s excellent book from 2008. Conservatives frequently note the group-level relationship that blue states have a higher per capita income than red states. Therefore, so the story goes, wealthy people are more likely to be Democrats. You know, limousine liberals vs. salt-of-the-earth working class Republicans. However, it turns out that this is a classic ecological fallacy. Wealthy individuals within a given state are more likely to be Republicans than their less wealthy counterparts. But this gap is much larger in red states, and nearly zero in the bluest states – therefore explaining the disconnect between the individual- and group-level data. (It’s slightly more complicated than that – I’d recommend reading the book.)
This leads us back to porn (doesn’t everything?). The study uses two types of group-level data – consumption of paid online porn across states, and across zip codes. Any inference made about whom within those states and zip codes is using porn is just that – an inference. To his credit, Professor Edelman never claims to be making individual-level inferences. Instead, that claim was first made explicitly by a journalist at the New Scientist covering the study, and was picked up widely elsewhere (blogs, etc.). I should point out, though, that I’m a bit baffled that the term “ecological fallacy” appears nowhere in the published study. I don’t know enough about the journal or the field to criticize either – but such an omission is typically frowned upon in the peer-review process.
You might say at this point, “Ok, we don’t know for sure that conservatives are more likely to use porn. But, given the state-level data, isn’t the burden on you to show why it might be a fallacious inference?” Yes, that’s a fair question. So let’s look at other aspects of the study to form a clearer picture about what it is or is not saying.
Accepting the group-level data for what it is, for now, a careful look at the study shows that our fantasy of repressed conservatives and Mormons burning up bandwidth in their basements has little support.
First, it may be the case that Internet porn consumption is higher in states in which access to porn is otherwise limited. A reasonable counter might be that most porn is online now, so that shouldn’t be a factor. However, according to a chart in Edelman’s study, a significant amount of porn-related activity still occurs in the brick-and-mortar form. Retail sales in 2006, in millions:
Video Sales and Rentals: $3622
Granted, much of the porn on the Internet is free, if you know where to find it. But that can actually serve as a case against the “conservative porn” inference, as it may be that the subset of porn consumers who are repressed conservatives might also be less Internet savvy (an argument made explicitly in the 2009 diary).
Other variables analyzed make the story complicated, to say the least, for advocates of the “conservative porn” hypothesis. For example, porn subscriptions were more prevalent in urban than in rural areas. This is a particularly striking counterexample for our showcased state Utah, since its only urban area is also its most liberal/Democratic/non-Mormon area. Also, zip codes with higher marriage rates were much less likely to consume porn. Further, there was no statistically significant difference in porn consumption between zip codes with high and low church attendance. There was also no relationship between presidential voting in 2004 and porn usage at the zip-code level. In fact, the only number that appeared even remotely to make the “conservative porn” case was a zip-code level positive correlation between porn consumption and aggregate conservative positions on some social issues. But again, it could very well be the liberals in those areas who, feeling repressed by the dominant social norms, turn to porn. Or not. But the bottom line is, we can’t know from the data.
Again, please take this diary in the spirit in which it is intended – as a teaching/learning exercise. I enjoyed the original diary and the comments therein. But in the reality-based community, I find it helpful to be armed with the tools of valid social-scientific inference. Lord knows, conservatives give us enough bad inferences to shoot down on a daily basis.