In Progress Analysis Criticizing Vaccine Safety Studies

(Posted by David AuBuchon) I’m presently working on a paper criticizing the credibility of observational vaccine safety studies, with an emphasis on MMR-autism studies. The first part of the paper is a literature review documenting overlooked and potentially devastating sources of confounding and bias. The second part is a sensitivity analysis grounded in the literature […]


(Posted by David AuBuchon)

I’m presently working on a paper criticizing the credibility of observational vaccine safety studies, with an emphasis on MMR-autism studies. The first part of the paper is a literature review documenting overlooked and potentially devastating sources of confounding and bias. The second part is a sensitivity analysis grounded in the literature review. The analysis illustrates numerically why existing MMR-autism studies are (at best) literally meaningless, as they would probably be incapable of detecting even a large autism risk owing to these flaws.

All constructive criticism is welcome. Subscribe to the comments thread at the bottom of the page if you wish to follow discussion on this research.

Tentative Title

Fatal Flaws in Vaccine Safety Research: An Analysis of Confounding by Contraindication and Exposure Dilution Bias

Tentative Abstract

The most important questions of vaccine safety are mostly addressed by observational studies. However, these require careful design and analysis to address many potential sources of bias and confounding. Two such sources which have received inadequate attention are confounding by contraindication (CBC) and exposure dilution bias (EDB). In studies of childhood vaccinations CBC occurs when children who have observable attributes that are associated with increased risk of adverse outcomes are simultaneously less likely to receive vaccines than are other children. This generally occurs because of parents’ perceptions that vaccines are not safe for their children with such “high-risk” attributes. Subsequently, adverse outcomes become concentrated in control groups, working to conceal any potential harms of vaccines, or even to produce spurious “protective” effects. EDB occurs if subjects in both the exposure and control groups have received other vaccines not under study that pose identical risks. The baseline pool of those susceptible to the adverse outcome may be shrunk due to harm from other vaccines, leading to a reduction in observed harm. EDB biases any true elevation in relative risk downward towards one.

I review the literature documenting the certain or near-certain existence of CBC in a variety of vaccine safety studies, with particular attention to at least 10 types of contraindications that stand to confound MMR-autism studies. I also document the likely existence of EDB in a number of contexts. I review efforts researchers have made to address these two weaknesses, sometimes successfully, and sometimes not. I document a pattern of conflicting results that suggest when CBC and/or EDB are credibly addressed, harms from vaccines are reported, and when they are not credibly addressed, an absence of those same harms or even protective effects are reported.

Fine and Chen previously derived a mathematical expression to model the effects of CBC in vaccine cohort studies. I enrich their expression to model CBC in greater detail, and to simultaneously model the combined effects of CBC and EDB in the context of MMR-autism cohort studies, demonstrating the sensitivity of reported risk ratios to both CBC and EDB. I illustrate using plausible variable values that in the event MMR and possibly other vaccines cause ASD that existing studies would very likely be incapable of detecting even very large risks.

Lastly, I offer four suggested observational study designs that reduce CBC and/or EDB. Most importantly among them is a retrospective design comparing fully vaccinated children to unvaccinated children in a manner that is ethical, probably feasible, virtually eliminates both CBC and EDB, and addresses concerns about differences in baseline characteristics between vaccinated and unvaccinated families.

I posit that CBC and EDB are the “elephants in the room” that render large swaths of vaccine safety research not credible.

In Two Simple Pictures

This first image illustrates confounding by contraindication in MMR-autism studies:

Confounding by Contraindication

Children already at high-risk of being diagnosed with ASD are selected by their parents to not receive MMR. This concentrates ASD cases in the control group, thereby concealing risk. Note that atypical development refers to an entire family, and not necessarily just an individual child. For example, if a child has an older sibling with ASD, even though the younger child may not have any signs of ASD, this child is nonetheless at both increased risk of ASD (because autism runs in families) and at higher risk of skipping MMR.

The next image illustrates exposure dilution bias:

exposure dilution bias

MMR-autism studies are potentially affected by this kind of bias. MMR is analogous to Vaccine H. For example, what if many of the people prone to being caused ASD by MMR would already be destined to receive an ASD diagnosis due to injury from other vaccines, regardless of whether or not they receive MMR? Studies could be comparing one cumulatively injured group of children to another cumulatively injured group of children, and hence find no risk, or a reduced risk, associated with MMR. These kind of vaccine cohort studies are based on the assumption that other vaccines not under study do not pose identical risks to the vaccine being studied. In general, this is an entirely unfounded assumption. Several plausible shared mechanisms by which multiple vaccines could cause autism exist. These include aluminum toxicity, detrimental immune activation, cross-reactivity, adventitious viral infection, and others.

Examples of Conflicting Results

Several strange anomalies are present in vaccine literature. When CBC and EDB are controlled for, either risks of vaccines are found, or protective effects vanish. When they aren’t controlled for, either no risks are found, or protective effects reappear. I have identified such examples of conflicting results with regards to:

  • Cumulative vaccine exposure and risk of autism
  • Cumulative vaccine exposure and risk of allergies
  • DTP and mortality risk
  • Flu vaccine and mortality risk
  • Flu vaccine and risk of nontargeted infections
  • Vaccines and risk of Sudden Infant Death Syndrome (SIDS)

Though not yet explicitly illustrated by concrete studies, I have also found what I believe is a major source of CBC in studies of measles vaccine and mortality. The alleged nonspecific effects of measles vaccine I believe are in large part – perhaps even entirely- nothing more than artifacts of a previously unconsidered source of CBC.

I will be reviewing all of these examples in the paper.

Sneak Peak at the Paper and the Model

I have preliminary documents available to qualified persons, only for the purpose of inviting informal peer-review. I realize the mathematical model is over most peoples’ heads, but for those who are mathematically inclined, I am looking for serious criticism. To request the files, email [email protected], introduce yourself, and acknowledge that you agree not to distribute the files.

The paper preview is a portion of the overall paper. This particular segment reviews the literature that proves that children who are already at high risk of ending up with an ASD (autism spectrum disorder) diagnosis are also less likely to receive MMR. This is confounding by contraindication, and it results in ASD cases being concentrated within the control group of studies, thereby masking any real ASD risk that may be posed by MMR vaccines. If you need access to full texts of papers I cite, you can enter the papers’ DOIs into Sci-Hub.1–3

The model preview introduces an algebraic framework for testing how sensitive the results of MMR-autism studies are to the combined effects of both CBC and EDB. This is an enrichment of a simpler model derived by Fine and Chen in 1992.4 Fine and Chen model only CBC,  but not EDB. I model them simultaneously. I also enrich a number of details about how CBC is modeled to make it appropriate to the context of MMR-autism studies. They have 6 variables, whereas I am ultimately am going to have about 13 or 14 variables. For anyone who wants to understand my model, it is virtually required that they read the Fine and Chen paper first.

What’s Lacking

There are a a several things I need to fix or further develop with the paper and the model that I will be working on going forward. These are as follows:

  1. I have documented 8 contraindications in the above-mentioned preview. It turns out that having a child born preterm is probably a 9th relevant contraindication, and potentially a very severe one, as preterm births are very common. There is definite evidence that preterm children are at higher risk of ASD, and further, that that risk may be mediated specifically by vaccine injury. It seems preterm children do have greater tendency to delay or skip vaccines, but I am looking for research specifically on the MMR receipt rate of children who were born preterm. I am not sure if this data exists. There is also a 10th relevant contraindication that can be constructed as “children who have an elevated autism risk score”, which is derived from a variety of known factors.
  2. I have assumed in the model that risky vaccines (i.e. vaccines that may cause autism) which are scheduled prior to MMR are equally received by both those who do and those who do not ultimately receive MMR. In other words, I assume that background vaccine exposure (i.e. any vaccine other than MMR – the vaccine being studied) is matched between the MMR-exposed and the MMR-unexposed. This turns out to be a false assumption, as vaccine refusal is correlated. The very fact that those in the control group of an MMR-autism study have not received MMR increases the likelihood that they skipped 1 or more earlier vaccines as well. This means that the “pool of susceptibles” (i.e. those susceptible to being caused ASD by vaccines, and who have not already become destined to receive an ASD diagnosis due to risky vaccines received prior to MMR) is not as diluted in the control group as it is in the MMR-exposed group. In short, my present model overestimates the implications of exposure dilution bias (EDB). The good news is that I know how to fix this and will do so in the next draft of the model.
  3. The actual variable values I have assigned to this present model in my illustrating numerical examples are not entirely feasible. I need to put some additional plausibility checks into it for the next revision. For example, the proportion of the study population that has the highest absolute risk of “getting” ASD at the time of any given risky vaccine would be those who incur both the independent relative risk associated with a contraindication, as well as the independent relative risk of the vaccine. In the terms used in the model, this absolute risk would be either S*R*D or S*R’*D, depending on which vaccine is being received. However, this product exceeds 1 in various numerical examples. Since it is not possible for anyone to have an absolute risk greater than 1, these are therefore not plausible values.
  4. This biggest issue I have yet to model is the fact that in the event vaccines do cause autism, that contraindication status and susceptibility status would not be independent. The present model treats them independently. In reality, contraindications would be concentrated within the group of those susceptible to vaccine injury. For example, the fact that ASD runs in families could merely indicate familial susceptibility to vaccine injury. So the very fact you have an older ASD sibling could indicate you are also susceptible to being caused ASD by vaccines. Or for example, if a parent notices signs of ASD in a child, it could be that those signs exist precisely because the child is susceptible to vaccine injury, and was injured by earlier vaccines. The implications of contraindications being concentrated within the susceptible are potentially huge, as confounding by contraindication (CBC) would not just result in those with contraindications becoming concentrated in the control group of MMR-autism studies. It would also result in those susceptible to being injured by MMR also being concentrated in the control group. This could result in a devastating concealment of risk.
  5. The model assumes that, aside from MMR, only vaccines schedule prior to MMR can potentially be risky. In other words, risks potentially posed by those scheduled after MMR are ignored. I need to attempt to generalize the model so MMR can be treated as any generic specific vaccine under study that appears in any place within a schedule of other risky vaccines.
  6. Only vaccine avoidance has been modeled. Vaccine delayal has not. If delaying vaccines reduce risk, this would add a whole additional level of risk concealment that exists in MMR-autism studies. I need to attempt to model this as well.
  7. The model treats all contraindications as being compacted into one overarching contraindication with its one set of variable descriptors. I need to attempt to generalize the model to allow for “i” types of contraindications, each with its own variable descriptors. This will make it easier and more convincing to apply variables values directly from the literature, and demonstrate both the individual and cumulative risk-concealing effects of contraindications. But this may not be practical since contraindications may be contaminant with one another and not always distinct, and that contaminant contraindications may cause synergistic confounding.
  8. There are still approximately 200 studies on my list which I need to review to search for data describing the significance of the 10 contraindications I have identified. Additional contraindications may also potentially be identified.

Support This Research

I’ve been quietly working on this project for a long time, without pay, and am only about 50% done. Once it’s done, I think it can and will get published in a high impact journal. Make a donation if you wish to show your support. An increasing number of reputable journals are offering authors a choice between subscription-based access and open access. I will opt for open access, in which case the author(s) are expected to cover the fees of publication. This can range from $1000 to $5000.

A secret link when Sci-hub is not working…(Jun 13 updated). NovoPro. Published June 13, 2016. Accessed November 6, 2018.
SCI-HUB…to remove all barriers in the way of science. Sci-Hub. Published November 6, 2018. Accessed November 6, 2018.
The latest Sci-Hub working domain. Love Science,Love Sci-Hub! Published July 29, 2019. Accessed July 29, 2019.
Fine PEM, Chen RT. Confounding in Studies of Adverse Reactions to Vaccines. American Journal of Epidemiology. 1992;136(2):121-135. doi:10.1093/oxfordjournals.aje.a116479