Glossary                                     Display Your Outline

Design and Methods

How you will do the study. See sample: Design and Methods. (Adobe Acrobat File help)

• This section should contain the following information:


1.  Design of the study (for example, descriptive, comparative, longitudinal, case-control, quasi-experimental, or randomized) and why that design was chosen (if necessary, use a diagram to clarify). Show reviewers that you know what you are doing and talking about by explaining why you chose the methodology you have chosen rather than alternative methodologies that you elected not to use. To the greatest extent possible, the reviewer should see your decision as one that was based on scientific merit and factors rather than on the basis of convenience or simplicity.

2.  Description of sampling methods, including sampling frame, study settings and sample size—what are your criteria for inclusion or exclusion, anticipated sampling list, method of selection, and group assignment? Describe the results of your power analysis. Indicate estimated attrition along with information explaining the basis for that estimation and how it will be handled.

3.  Research protocol: How will the data be collected? Who will collect the data? What procedures will be used? What will the basic research protocol be for data collection? How many data collection points will there be and why? How will subjects be recruited for the followup? How will the research protocol differ in the followup?

4.  Instruments and data processes or data collection forms, and descriptions of their domains of information collected. (Briefly describe any instruments in the text, including previous reliability and validity data for the instruments—include a copy of each instrument in an appendix, if allowed.)

5.  Procedures for training of researchers or interviewers, data collection, and handling of human subjects and confidentiality issues.



6.  Data analysis and evaluation. What statistical procedures and methods will you use to analyze the data for each hypothesis you are testing? How will you deal with missing data? A strong analytical plan should go beyond simple statistics but need not be excessively complex. Reviewers especially like to see multivariate analyses and analyses that control for the effects of extraneous variables or conditions. Be sure to address the issue of missing data and how you will handle it in your study, as this is a particularly important issue in the field of drug abuse research, where attrition tends to be high for long-term studies or repeated measures design. Per NIH policy, also be sure to present subset analyses for examining gender and racial or ethnic differences.

7.  A brief discussion of the limitations of the proposed study and alternative methodologies for carrying out the proposed research plan, if necessary. A delicate balancing act is needed here. If you provide too much information about potential limitations of your work, it gives reviewers ammunition to score or evaluate your project less favorably on scientific grounds. On the other hand, if you fail to address key limitations of your study (including threats to external validity) and the reviewers notice that, you could be held accountable scientifically (in the form of a poorer grant score or evaluation) for failing to address the limitations of the research you are proposing.

8.  A timeline of tasks to be completed during the project period.

9.  Possible hazards to research personnel and study participants, along with procedures to prevent dangerous situations.




• Make sure that the study you describe corresponds with the specific aims you listed earlier in the proposal.



• Justify all of your decisions—provide explanations for what you did propose to do and why you did not propose something else. This shows reviewers that you have given careful thought to the study you are proposing and that you have considered alternative approaches to doing the study you are hoping to do.



• Gender and minority representation must be discussed, and a justification provided if adequate representation is not applicable or not possible. Remember, reviewers can reject applications that do not represent all relevant groups appropriately. All reasonable steps should be taken to guarantee representation of women, children, and members of racial and ethnic minority groups.



• The goal of the research design and methods section is to minimize the number of assumptions others (reviewers) must make about your project and to show that you are using sound scientific techniques.

Four of the most common research designs are discussed below. Answer the questions in the relevant section for the design you have chosen. Be sure to address all appropriate threats to validity for the design you select.

(1) One Group Posttest Only Design
(2) One Group Pretest/Posttest Design
(3) Nonequivalent Comparison Group Design; and
(4) Experimental Design

One Group Posttest Only Design

This design neither documents change nor corrects for threats to validity. May be used for a pilot test to identify potential problems before you conduct a more expensive evaluation. It can tell you how well a client is doing or how satisfied a client is with a program or product; but it cannot tell you, for example, if a client has improved since being exposed to an intervention or what effect the intervention has had on the client.

One Group Pretest/Posttest Design

With this design, you can measure change. You cannot, however, obtain information needed to discount any other possible explanation for change. The threats to validity that must be addressed in your proposal are the following: maturation, testing, instrumentation, and statistical regression.