|
Computing Sample Size for Scientific Studies: A Non-Technical Overview
Power and sample size calculations are an expected and desired part of the protocol of nearly all scientific research in the social sciences and medicine, with the notable exception of purely non-clinical laboratory studies. Review panels at the National Institutes of Health expect to see these calculations in most grant applications, and their absence may raise serious concerns about the feasibility of the proposed work. This is especially true for studies involving human or animal subjects in which sample sizes that are either too large or too small have ethical implications as well as scientific and budgetary ones.
Too large a study means not only a waste of resources, but also the expectation that, in a comparative study of interventions, too many people will have received the inferior "arm." Furthermore, an excessively large study is sensitive to differences that are below the threshold of practical utility or clinical significance. In this case, the participation of too many participants, whether it was painful or painless, risky or benign, was wasted. At the other extreme, having too few participants leads to inconclusive studies that cannot answer their primary questions and so are also disrespectful of the contributions of men, women, or mice.
All calculations of power and sample size are linked to hypotheses. Indeed, power only makes sense in terms of the ability to test a hypothesis. In a study with several hypotheses, power is often computed only for the primary hypothesis, less often for secondary hypotheses. If there is to be a power and sample size calculation, hypotheses must be specific predictions about what is likely to be found. While the wording of the hypothesis need not mention every paper-and-pencil instrument or laboratory test, ultimately the calculation must be about a specific result from such a data source: "in regard to test A, there will be a difference between groups," or "features x and y will be positively correlated," or "there will be an increasing trend in outcome z."
Although studies vary enormously in design, four elements appear in every calculation of power and sample size. They are the sample size itself (N), the chances of a false positive finding ("type I error," also called a), the chances of a true positive finding (power, also called 1-ß), and the smallest meaningful effect size (d). Given any three of these elements, relevant formulas yield the fourth element. Most commonly, one computes the sample size, N, for given values of a, 1-ß (typically 0.05 and 0.80 or 0.90), and d. Or, one might check whether a study of a given size is likely to detect a true, meaningful effect: 1-ß for given values of N, a and d. Less frequently, d is computed as a function of the other three design elements; if the detectable d is too large, the study is insensitive to some meaningful effects.
Details of how these elements appear in the sample size formulas vary according to the study design. Thus, in a study with multiple groups, N may actually be a list of numbers, perhaps unequal as when a decision has been made to assign an active intervention to more participants than a placebo. Furthermore, if there are multiple groups, d may also be a list of the differences of the outcome in one group compared to another. The outcome itself may be a number (e.g., a test score), or binary (e.g., "yes/no," "good/bad," "alive/dead"), an event time (e.g., time to treatment failure), or something more complex. While the formulas relating the four design elements vary according to these key details, the elements themselves always have the same inherent meaning.
The simplest power and sample size analyses can be done by pocket calculator, more sophisticated ones by pre-written, specialized software packages. In any case, as with other such specialized, technical tasks, the best advice is to consult an expert, ideally a statistician familiar with both the hypotheses and the outcome measures.
Recommended Reading and Additional Resources
Dennis, Michael L; Lennox, Richard D; Foss, Mark A. Practical power analysis for substance abuse health services research. [Chapter] Bryant, Kendall J. (Ed), Windle, Michael (Ed), et al. (1997). The science of prevention: Methodological advances from alcohol and substance abuse research. (pp. 367-404). Washington, DC, USA: American Psychological Association. xxxii, 458 pp.
Go to Drugnet.net for more information about power analysis and other statistical calculations.
|
