Imagine for a second that I wanted to find out STD rates among straight men in big cities. Let's presume that I recruited a sample of straight men at night clubs, and let's say half of this sample were under the age of 32 years old, and let's say that 30% of this sample were straight men from very low income households, and let's say that two-thirds of this sample were straight men who were living alone and single, would you think that I should extrapolate from that that I had an accurate representation of straight men in big cities?
You'd probably discount my study entirely. You already know that I collected a convenience sample, and that you couldn't extrapolate from a convenience sample to larger population, and you'd probably discount my study entirely unless I made the proper qualifications. Well, this is what the CDC did recently but in reference to gay men and HIV rates.
Their conclusion is astonishing -- that 19% of gay men in big cities are infected with HIV. That is enough to set off a great deal of alarm. However this is exactly what the CDC study is: a bunch of bunk. In fact, other CDC data makes this claim preposterous. There is, in reality, no need for the alarm that the CDC's study may unwarrantly set off.
Let's go into the flaws of this study:
First this is a convenience sample. Convenience samples are always trouble when you try to generalize from such a sample to a larger population. The sample was recruited from gay bars, clubs, and social organizations. Let's think about the first two: bars and clubs.
While gay bars and clubs are a good place to go to meet other gay guys, there not such good places to go to conduct scientific research on the gay community as a whole. Lot's of guys go to gay bars looking for one night flings, they are people who engage in very risky behavior,... Maybe about 20 years ago gay bars and clubs may have been a bit more representative because they were the only way for g/bi men to congregate. But we have the internet, we have gay coffee shops, restaurants, gay resorts, and being publicly gay, especially in the 21 cities listed, is a lot easier than it was decades ago. You don't have to sneak into the gay bar to try and meet up with other gay people. Further, the CDC doesn't tell us what gay bars they recruited these men from. It would make a big difference to me, if I were in New Orleans, as to whether they recruited from Good Friends, the Oz, or the Phoenix. Each club could produce very different results because they have very different clientele.
Want proof that a convenience sample recruited from gay bars and clubs might not be representative and might skew towards a more promiscuous set of gay men?
- 2/3rds of the sample were men who lived alone. No roommates anything. Two things this indicates: they are single men, and many of them may lack sufficient social support. Such people, of any sexual orientation, are more likely to be engage in risky behavior.
- 30% had household incomes under $20,000 a year. No median household income was provided in the CDC report. This is particularly interesting because there is very good research that we could compare these numbers to find out just how representative this sample is of most gay men. In the cities surveyed, places like Boston, New York, San Francisco, Washington, DC, etc, with the high cost of living in those areas, you are talking people either in poverty or quite near to it. The sample skews to low income gay and bi men. Poverty is one of the big predictors of HIV infection. So, a sample that skews to low income gay and bi men would be problematic.
- The median age of gay men in this study was 32 years old. So, this sample skews young. Younger people are generally more promiscuous than older people, particularly when those younger people are living at or near poverty levels, are living alone, are single, etc...
Finally, there is some astounding CDC data that makes a 19% infection rate for gay men, even in big cities, highly unlikely. The CDC estimates that there are 1.1 million Americans living with HIV, and about a half of those got HIV through MSM contact (male to male sexual contact). That means that about 550,000 men with HIV got it from MSM contact. Some studies show that up to 14.2% of men have sex with other men in a single year.That would mean that roughly 23 million men a year have MSM contact. Simple math says 2.4% of those men have HIV. One would have to presume that HIV rates in major cities is almost 8 times as high as the national average to believe the CDC's numbers. While I can believe that the rates would be higher because HIV patients may be able to find more support services in the city and thus move to the city, there's no reason to believe the CDC's numbers which come from a highly questionable convenience sample.
Convenience or Accidental Sample:
Accidental sampling is a type of nonprobability sampling which involves the sample being drawn from that part of the population which is close to hand. That is, a sample population selected because it is readily available and convenient. The researcher using such a sample cannot scientifically make generalizations about the total population from this sample because it would not be representative enough.
Further, the CDC also didn't include anyone in the sample who didn't answer the behavioral survey portion of this research. What's wrong with this? The most modest people are the ones who might be less likely to complete such surveys. Those people might be at a lower risk for HIV.
I do believe that there is one bit of data from the CDC's study that we can find to be useful. That is the data the shows that a significant proportion of their subjects weren't aware of their HIV+ status. That information is useful. However, the 19% HIV rate for gays in the big city is not useful, and calls for unnecessary alarm.
Update: In the editorial notes of this report the CDC does limit the extrapolations of this data only to a segment of the gay/bi community, but does not make this distinction clear in the body of its report. It is a distinction that is necessary to keep the alarmist type thing that is happening at Huffington Post from happening.