In the case of Keys’s hypothesis, the annoying evidence was consistently disregarded from the beginning. Because the totality of evidence was defined as only those data that confirmed the hypothesis, Keys’s hypothesis would always appear monolithic. Annoying observations could not force a reanalysis of the underlying assumptions, because each of those observations would be discarded immediately as being inconsistent with the totality of the evidence. This was a self-fulfilling phenomenon. It was unlikely, however, to lead to reliable knowledge about either the cause of heart disease or the routes to prevention. It did not mean the hypothesis was false, but its truth could never be established, either.

One other method can be employed to judge the validity of the hypotheses that dietary fat or saturated fat causes heart disease, and that cholesterol-lowering diets prevent it. This is a technique known as meta-analysis, viewed as a kind of last epidemiological resort in these kinds of medical and public-health controversies: if the existing studies give ambiguous results, the true size of a benefit or harm may be assessed by pooling the data from all the studies in such a way as to gain what’s known as statistical power. Meta-analysis is controversial in its own right. Investigators can choose, for instance, which studies to include in their meta-analysis, either consciously or subconsciously, based on which ones are most likely to give them the desired result.

For this reason, a collaboration of seventy-seven scientists from eleven countries founded the Cochrane Collaboration in 1993. The founders, led by Iain Chalmers of Oxford University, believed that meta-analyses could be so easily biased by researchers’ prejudices that they needed a standardized methodology to minimize the influence of such prejudice, and they needed a venue that would allow for the publication of impartial reviews. The Cochrane Collaboration methodology makes it effectively impossible for researchers to influence a meta-analysis by the criteria they use to include or exclude studies. Cochrane Collaboration reviews must include all studies that fit a prespecified set of criteria, and they must exclude all that don’t.

In 2001, the Cochrane Collaboration published a review of “reduced or modified dietary fat for preventing cardiovascular disease.” The authors combed the literature for all possibly relevant studies and identified twenty-seven that were performed with sufficient controls and rigor to be considered meaningful.*26 These trials encompassed some ten thousand subjects followed for an average of three years each. The review concluded that the diets, whether low-fat or cholesterol-lowering, had no effect on longevity and not even a “significant effect on cardiovascular events.” There was only a “suggestion” of benefit from the trials lasting more than two years. In 2006, the Cochrane Collaboration published a review of multiple-risk-factor interventions—including lowering blood pressure and cholesterol—for the prevention of coronary heart disease. In this case, thirty-nine trials were identified of which ten (comprising over nine-hundred thousand patient years of observation) included sufficient data and were carried out with sufficient rigor to draw meaningful inferences. “The pooled effects suggest multiple risk factor intervention has no effect on mortality,” the authors concluded. Although, once again, a “small” benefit of treatment, perhaps “a 10 percent reduction in CHD mortality,” may have been missed, they added.