If you're a regular reader of this blog, you would think that I spend a lot of time discussing autism spectrum disorders (ASD). But the goals of this blog is weed out and debunk pseudoscientific beliefs, especially some of the more popular ones. Probably the most ridiculous had been the
fraudulently invented and eventually
thoroughly debunked link between vaccines and autism. That belief is completely dead, right? Well, not really. Irrational and, frankly cult-like, groups like
Age of Autism refuse to give up. Of course, the Age of Autism is so deluded by their illogical and unscientific beliefs, they oppose funding for genetic research into autism.
Not everyone in the world of ASD has that fanatical hold onto the "vaccines cause autism" trope. Most real scientists accept that the likely cause of ASD is some combination of genetics and environment. But that hasn't stopped people from claiming that everything, but the kitchen sink, causes ASD. About three years ago, Emily Willingham, Ph.D., whom I consider to be one of the leading ASD scientific experts on this planet, wrote a hysterical, but still appropriate, article about all the popular causes of ASD. Older mothers. Older fathers. Depressed mothers. Fingers. Facial features. Facial features?
Dr. Willingham wrote about around 50 different causes of ASD, including refrigerator mothers (but not kitchen sink mothers)–it has nothing to do with actual refrigerators. That article was very important to me, because it formed the basis of my skepticism about popular beliefs regarding correlation and causality. It's not just ASD, but other areas, like cancer causes and cures, which seem to attract the everything-but-the-kitchen-sink science.
Recently, I was pointed to an article that attempted to outline an epidemiological correlation between agricultural pesticides and neurodevelopmental disorders, published in the journal, Environmental Health Perspectives. Before examining the methods, data and analysis, it's important to ascertain the overall quality of the journal and the researchers. The journal itself, Environmental Health Perspectives, has an impact factor of 7.26, which is not low, but certainly is not in the upper tier of journals. In many academic environments, any article published in journals with an impact factor less than 10 may not be used in consideration of tenure (although this guideline is not universally accepted).
The authors of the article are mostly from the University of California-Davis, a top research university, with a lot of focus in agricultural sciences. The first author is Janie F Shelton, who has published three articles about environmental pesticides and autism, and that's all she's published (she's just beginning her research career, in that she was a graduate student at UC Davis). In one of her previous articles, Dr. Shelton appeared to indicate that because mothers in the US have a higher burden of pesticides in the blood, there is some causal link with the higher rate of ASD in the US. I am unconvinced that she actually showed causality with such a general relationship. But what worries me most is that Dr. Shelton has an inherent bias towards what she's going to find. Instead of seeing where data might lead her, she has a hypothesis, pesticides cause autism, and she's trying to find data that supports that hypothesis. This isn't what a scientist should do.
Nevertheless, I can't criticize the journal, nor the bonafides of the researchers.. But that, in of itself, isn't everything. It's time to take a closer look at what was written
The Cliff Notes version of this study is that it appears to be a kilometer wide and a few mm deep. There's a lot of data, but I'm dubious that it supports the conclusions that pesticide use is correlated to (or even causal to) autism spectrum disorders. Of course, the popular press and the anti-chemical crowd is all over this article. It's so frustrating that they accept this research without actually examining it carefully. But I'll try.
Before we look at this study, we have to understand the data behind the CHARGE study, which is the basis of Dr. Shelton's paper. Basically, the CHARGE study is a case-control study, which is a type of observational study in which two existing groups differing in outcome are identified and compared on the basis of some supposed causal attribute.
Case-control studies are important in epidemiological research, but, in the hierarchy of evidence, it is of a fairly low rank. Why? Because data from case-control studies tend to be observational rather than objective, like in double blind clinical trials. Furthermore, confounding factors can bias the results if they are not fully understood–sometimes it's impossible to even identify confounders. Case-control studies have been valuable, but usually only when there are huge numbers involved. Some case-c0ntrol studies with vaccines include over 1 million patients, so that statistical analysis has much lower error rates.
Because California carefully tracks type, amount and date pesticide applications, the authors could examine how close the mothers of the subjects lived to where pesticides were applied. So, the authors could then find subjects who lived in areas that were relatively close to farms that had pesticides applied on a particular date.
The study populations were: 486 with ASD, 168 with development delay (DD) and 316 neurotypical. Moreover, all of these study subjects were found in an small area within a two hour drive of the University of California, Davis. I know there are all kinds of jokes about California (and Californians themselves make a lot of jokes about that area of California), but it is difficult to accept an epidemiological study whose subjects represent a tiny segment of the general population. Not being an expert about environmental factors in that area, I still know enough to ask questions. Water quality. How about soil. Heavy metals in the environment from the gold rush (yes, we're still paying for that damage).
I'm being a bit flippant, but the fact is the authors didn't really consider many confounding factors. They state that:
Other potential confounders explored but found not to satisfy criteria for confounding based on inclusion in the DAG or the change in estimate criterion were: distance from a major freeway, maternal major metabolic disorders (diabetes, hypertension, and obesity), gestational age (days), latitude of residence, type of insurance used to pay for the delivery (public vs. private), maternal age, paternal age, and season of conception.
They seem to ignore all other environmental factors that would be important for this type of study. It's possible that such a study would become complex, but that's why it's hard–so that we know the results will stand up to scrutiny and in-depth analysis. Discussing and analyzing the full breadth of confounding factors is, for me, one of the most critical parts of a case-control study. And this study completely dismisses confounders with barely a sentence.
Unfortunately, that's not all. I actually remain unconvinced that data shows you anything, irrespective of confounding information. Here's my analysis:
- The study is missing evidence that the pesticides scattered beyond the target areas (according to wind patterns or whatever). As I stated above, agricultural operations despise wasting pesticides and attempt to be accurate in application. What convincing evidence did the authors present to make me, or any scientific reader, think that the pesticides actually landed near the subjects? That would be hardly anything at all.
- The study is lacking evidence of a dose-response effect–the only way to confirm a toxic effect is to establish the range of pesticide "dosage" (in this case applications or something quantifiable) that caused what level of ASD. If they had a large enough subject population, and they knew the actual pesticide burden of each mother, they could develop a dose (pesticides) response (ASD) graph.
- The results were based on such small numbers, which left us with huge statistical errors. If you looked at each result, it ranged from all the way from exposure reduced the risk of autism to exposure tripled the risk of autism. But in the end, given the size of error, the increased risk seemed to be clinically small–you might think that a 58% increased risk is high, but with such small subject numbers and lack of analysis of confounding factors, it becomes difficult to determine if that 58% is important. In other words, the relative risk was so tiny that it could be overwhelmed by any of the ignored confounding factors.
- The authors spent an inordinate amount of time discussing other animal studies, tying pesticides to neurodevelopmental disorders, as if they had to add some meat to their article because the actual case-control study was so thin.
- We have no evidence of blood levels of these pesticides in the mothers. Do we know if being closer to or farther from the site of application had any influence over the blood level of the pesticide? It is very possible that there was no difference in blood levels of these pesticides between the mothers of the control group and the ASD group–that would be key evidence for another factor in the those areas.
- Where is the causality? Even if this data made sense. Even if the size of the subject groups were 100X larger. Even if we could show blood levels. This study has done nothing to provide a physiologically plausible pathway between exposure and ASD, which would be the basis of causality.
Orac also did a review of this study, almost for the same reasons as I did–here comes ANOTHER cause of autism. His review of the data lead him to this observation:
The sine qua non of a good study demonstrating an association between an environmental exposure and a condition requires the actual verification and quantification of the environmental exposure under study in the cases. Sometimes this involves measuring the actual levels of the chemicals in question, either in the research subjects (ideally) to document exposure or in the places where they live and work.
Precisely. We have no idea, and the authors offered no insight into this issue, whether there actually was any additional exposure to pesticides. I could write a whole paper on the whole litany of factors that might make it impossible to determine if the mothers were exposed even within the 1.5 km of the pesticide application that was used as the cutoff for the study participants. Think about this–farms that apply pesticides do it with pinpoint precision in low wind days. Why? Because pesticides that land on a home 1.5 km away is a very expensive proposition. They want the pesticides on their crops, not on a rooftop in a residential area. If we had actual blood levels of the pesticides from the mother, we'd have some actual evidence that the agricultural based pesticides contaminated the mothers.
The authors appear to be cautious about their study. They state that "this analysis serves as exploratory research to identify environmental risk factors for ASD and DD, and contributes to a broader understanding of the potential risks to neurodevelopment from agricultural pesticides in a diverse population of California residents." That's typical scientific verbiage that can be translated to "we need more data to confirm this."
Except in the press release from the University of California Davis, Janie Shelton claims:
This study validates the results of earlier research that has reported associations between having a child with autism and prenatal exposure to agricultural chemicals in California. While we still must investigate whether certain sub-groups are more vulnerable to exposures to these compounds than others, the message is very clear: Women who are pregnant should take special care to avoid contact with agricultural chemicals whenever possible.
Oh and there's this from the same press release:
Research from the CHARGE Study has emphasized the importance of maternal nutrition during pregnancy, particularly the use of prenatal vitamins to reduce the risk of having a child with autism. While it’s impossible to entirely eliminate risks due to environmental exposures, Hertz-Picciotto said that finding ways to reduce exposures to chemical pesticides, particularly for the very young, is important.
I guess Shelton thinks it's some sort of hit-it-out-of-the-park home run of evidence. But scientifically, she's shown us very little, but this study may start a panic equivalent to what we say from the vaccines cause autism myth.
Let me close with Orac's observation about this study:
I would agree that pregnant women shouldn’t be handling industrial strength pesticides, but if there’s evidence that living within a mile or so of areas where pesticides are applied during pregnancy will cause a woman’s child to develop autism, certainly neither Shelton nor Hertz-Piccioto has provided it, either in this paper or elsewhere. Sadly, that didn’t stop the press from dutifully responding to the press release from the MIND Institute as though this study were slam-dunk evidence that pesticide exposure during pregnancy causes autism. It’s not, not by a long shot. It barely qualifies as maybe hypothesis-generating evidence. Wait. Strike that. I don’t think it qualifies even as that.
Key citations: