evaluation

Evaluation: Measuring Success

Mathilda Harris

Donors will always ask how you will know if your idea is succeeding. Your response can make or break your chance to win the grant, and your evaluation plan holds the answer. Reassuring the donor that you have a strong evaluation component – and that your project will make a substantial impact – will add to your credibility and demonstrate your good stewardship potential. Before you design your assessment, you need to consider what you want to accomplish and how you will gauge change along the way. This ensures that you can achieve what you promised in your proposal. The questions you need to ask and the approach you take will depend on your measurable objectives. The example we use below will involve the following objective:

Our objective is to transform eight different communication courses over a period of one year via the integration of a two-week instructional module on negotiating skills, to be included in each course.

The process is as follows:

Questions

What do you want to know to determine if your objective was realized? For the example above, you would want to know if:

  • criteria for course transformations were met
  • subject matter was appropriate for the module
  • criteria for negotiating skills are achievable and being met
  • measurable outcomes are accomplished along the way

Indicators and Evidence

How will you know that the above questions are being answered? Examples include, but are not limited to:

  • pre- and post-project tests
  • surveys, observations, interviews, and other methods that gather qualitative information
  • quantitative data collection such as: number of students, instructors involved, course transformations, tests, and grades
  • Involvement in and success rates of mock negotiations

Timing

When should you collect this data? Your project will determine the timing; however, gather your data incrementally according to the percentage of change for established baselines of your proposal. A timeline demonstrating each of the changes as they are measured and happening is an extremely valuable tool.

Data Collection

The data-collection process assists the evaluation team in revising the design of the program and methods based on resources – both financial and human. The team can also examine how the evaluation process is received by participants and other people from whom information is collected. Assessment of the usefulness of the information collected is another factor the team can determine. This will include:

  • Sources: Who has the information? What are the measurable criteria and expected outcomes for each of the objectives? In our example above, sources may be the evaluator, the faculty, the course transformation experts, a committee of experts who will evaluate each module, and of course, the students.
  • Methods: How and when will the information be gathered? Your approach might be quantitative, qualitative, or mixed method research. Very often an evaluation consultant is included to ensure the statistical design and methodology are well-developed and logical.
  • Sample: From whom will the data be collected?In large studies, a stratified random sample may be obtained by separating the population into mutually exclusive sets, or strata, and then drawing random samples from each stratum. Since the population for our example objective is not very large, chances are that the data will be collected from everyone involved in the project.
  • Instruments: What tools will be used in gathering the data? These can range from quite simple to extremely complicated. Developing the instruments may be a lengthily process and relate directly to what needs to be known about each objective. Some examples may include attitudinal instruments, surveys, and pre- and post-tests. Once again, you may require the assistance of an evaluation consultant.

An evaluation plan is a written document that states the objectives of a project and the questions that will be used to determine its success. It also includes the information that will be collected and the timeframe of data collection. You can think of the evaluation plan as the instructions which will guide you through each step of the process. It is the key to demonstrating accountability to the donor.

Mathilda Harris

Director at Grant Training Center
Over the past 18 years, she has written grants, conducted capital campaigns, developed strategic plans for grant procurement, and assisted individuals and institutions to write winning proposals for various donors.

Latest posts by Mathilda Harris (see all)

2 comments on “Evaluation: Measuring Success”

  1. The article above was quite helpful. Hope you accept questions. Isn’t it important to substantiate claims of success in real terms? What if I’m doing animal rescue and say I got 100 feral cats neutered? Don’t I need copies of the vet papers?

    Reply

    1. Thank you for the question, Dr Harris provided the following response:

      “It depends on the reason for your need of evidence. For example if you are applying for a job, the papers might be helpful as proof to an employer. If you are writing an article, you might begin by saying that these animals were neutered in such a such a place. If I received a grant and I need to measure success for the donor, I would have a chart that shows when the neutering took place, how many animals were neutered, and where it took place.”

      Reply

Leave a Reply

Your email address will not be published.