Bradford Hill on Observation versus Experiment

A highly quotable classic was presented by Bradford Hill at the Cutter Lecture on Preventive Medicine at the Harvard School of Public Health on March 25, 1953. Hill introduced his ideas as an antidote to a prior Cutter Lecture, provoked by the claim of Oxford nutritionist Hugh Sinclair that: “the use of the experimental method has brilliant discoveries to its credit, whereas the method of observation has achieved little” (Hill 1953, 995).

Hill, the advocate and pioneer designer of randomized clinical trials (RCT), indicates that, in fact, few lifestyle and environmental effects on health are amenable to experiment. Accordingly, he lays out a map for rendering observational studies as much as possible like experiments, making efforts to sharpen data collection and reduce biases, all the while seeking, as his London School colleague Major Greenwood wished, to permeate “statistical research with the experimental spirit. In other words, can observations be made in such a way as to fulfill, as far as possible, experimental requirements?” (ibid.)

Hill warns presciently that “the methods of partial correlation, enthusiastically accepted a quarter of a century ago, no longer seem to have an unlimited power to penetrate the secrets of nature” (ibid.). Then he illustrates how the powerful observations of John Snow on the distribution of London cholera led him to explore the “natural experiment” of the rival companies water supplies to the populace. “… Such ‘experiments’ or, at the least, effective ‘controls’ would not … really prove to be so rare if one invariably cast one’s eyes round for them after vital statistics, or similar observation, had given an appropriate lead” (ibid. 996). [Ancel Keys, for example, stung early by tough critiques made of his ecological observations with government data on diet and coronary disease, turned to the natural experiments provided by stable populations having centuries of differing traditional diets.]

Hill asks pointedly, in reply to naysayers about the statistical treatment of observations, “How many researches which have led to real advances in medicine would ever have been started had there not first been some statistics to suggest that here was a problem to be investigated?” (ibid. 998)

Advances made by such pioneers in CVD investigation as John Gofman and Ancel Keys illustrate the advantage of fearlessly treading both paths, observation and experiment. Hill maintains: “There are…no grounds for antagonism between experiment and observation. The former, indeed, depends on observation but of a type that has the good fortune to be controlled at the experimenter’s will. In the world of public health and preventive medicine each will–or should–constantly react beneficially upon the other. Observation in the field suggests experiment; the experiment leads back to more, and better-defined, observations” (ibid.).

Hill actually holds out for the greater task or talent of the observer: “The observer may well have to be more patient than the experimenter–awaiting the occurrence of the natural succession of events he desires to study; he may well have to be more imaginative–sensing the correlations that lie below the surface of his observations; and he may well have to be more logical and less dogmatic–avoiding as the evil eye the fallacy of post hoc ergo propter hoc, the mistaking of correlation for causation” (ibid., 1000). His colleague, William Topley, suggests that the worker at the bench, the clinician in the ward, and the epidemiologist in the field are all making experiments, “all shading through almost imperceptible stages of increasing intervention into the fully developed experiment in the laboratory” (ibid.).

Fifty years later, we find a vigorous return to this fundamental concept through an innovative strategy in which the observational design of a prospective study that showed an apparently favorable effect of hormone replacement therapy was reformulated as closely as possible to conditions of the clinical trial of the same therapy (Womens’ Health Initiative HRT trial). The observations then yielded the same null results on cardiovascular risk as those in that momentous trial (Hernan et al. 2008). (Henry Blackburn)

References

Bradford-Hill, A. 1953. Observation and experiment. New England Journal of Medicine 248; 995-1001.

Hernán, M.A., Alonso, A., Logan, R., Grodstein, F., Michels, K.B., Willett, W.C., Manson J.E., Robins, J.M. 2008. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology. 6:785-8.