Comments on the Analysis of the Burlington Randomized Trial

Abstract

In view of the continuing importance of the large-scale study described as "The Burlington randomized trial of the nurse practitioner" (Sackett 1974, Spitzer 1974), it is worthwhile discussing some of the difficulties in interpreting the published account of the statistical analysis of this study. These comments are offered for the purpose of clarification and in no way are meant to minimize the considerable achievement of this study. The purpose of a statistical significance test is to assess the numerical evidence for the inclusion of a new parameter. Following the principle of parsimony of Occam's Razor, which may be taken in the form "entities are not multiplied without necessity" (Jeffreys 1961), one commences with the situation in which all the variation is treated as random. Parameters are introduced one by one which appear to explain a sufficient amount of the variation to merit their inclusion and the remaining variation is treated as random. This latter part never completely disappears. At any stage in this process of successively introducing fresh parameters the present status quo may be described as the "null hypothesis". This is the "working rule" presently achieved. The "alternative hypothesis" next considered usually contains one additional, or new, parameter or, one additional degree of complexity. The evidence for the inclusion of this new parameter is sometimes simply assessed by considering the ratio of the estimate of this new parameter to the standard error of the estimate. However the assessment of the evidence is made, the "null hypothesis" always describes the present working rule and the alternative hypothesis describes a rule of greater complexity, the "thesis" or proposition which the experimenter has in mind whilst designing the experiment. Often the main purpose of the experiment is to compare the experimenters' thesis with the present working rule. If the sample size is very small the estimate of the standard error is often too vague to be able to reject the null hypothesis. When the same size is very large the evidence for the introduction of the new parameter may be