Is Research or Evaluation Needed?

Most State Administering Agencies (SAAs) and policy makers with full plates and limited funds, will not have the capacity or resources to engage in or support rigorous longitudinal research projects that study a group of individuals over a relatively long period of time. Indeed, many will feel stretched to allocate funds which some will argue are better invested in programs to evaluate one or more of those programs. Therefore, your first task is to decide if research or evaluation is warranted. Would one or the other assist your organization in discharging its responsibilities or completing its duties?

Cirumstances Where Evaluation May Not Be Needed

There are four circumstances where evaluation may NOT be worthwhile:

1. When a program has few routines and little stability.

2. When the people involved in the program cannot agree on what it is trying to achieve.

3. When the evaluation sponsor or project manager sets stringent limits on what the evaluation can study thereby putting off limits many important issues.

4. When there is not enough money or no staff sufficiently qualified to conduct the evaluation.

An additional situation when evaluation may not be needed is when implementing a “blueprint” program (a program that has already been extensively evaluated), as these have the highest level of proof of effectiveness (control groups, external evaluators, effect lasting one year beyond the end of the program, and replication in more than one site and demographic group). With this sort of program, evaluation may not be needed, but careful implementation of a project, based on the model, is a requirement so collection of data demonstrating a site is complying with critical components of the model is important.

Things to Consider When Investing in Evaluation

Since few states will have sufficient resources to evaluate all the programs that merit evaluation, let alone engage in independent research that will contribute to the larger knowledge base, the following information is principally focused on evaluation though some of the information could apply to both.

Once a decision has been made to invest some funds in evaluation, you’ll need to decide what will be evaluated and by whom. You also need to keep in mind that not all programs warrant independent evaluation. As noted above those based on established models that have been evaluated may only need to regularly submit performance measures (i.e., objective measures that show the extent to which a program is achieving its stated goals, objectives and activities) to demonstrate they are performing as intended. You also need to remember there is no “one size fits all” evaluation method. The most appropriate method depends on:

1. What you want to know/question(s) to be answered;

2. How easy/difficult it will be to obtain that information and answer the question(s);

3. Resources available for the evaluation; and

4. Time available in which to complete the evaluation.