STEP 7: Measuring Achievements

In this section you will map out ways of monitoring the achievement of your objectives to ensure that your program stays on track. This will also help you improve your program on an ongoing basis, and justify requests for project funding.

You will:

  • Consider the use of pilot testing.
  • Decide what measures to monitor frequently.
  • Select the ones you will measure only at major milestones.
  • Consider setting up a control group.

Note for health promoters.

 
1. Whenever practical, pilot test your program before implementing it widely.

Examples

The purpose of Class5’s RMC pilot was to adapt its existing SEE program to a healthcare setting.

Stockholm’s Congestion Pricing initiative is a rare example of a pilot program that introduced then removed and then reintroduced an intervention. It illustrates the importance of timing referendums after rather than before trials or pilots.

BRIDGE piloted Nditha! Sports in the Mzimba district. Based on the overwhelmingly positive results, BRIDGE scaled up the activity to the remaining seven districts.

A few years prior to the development of its formal telework initiative, AT&T conducted a few, localized telework pilot projects in three regional offices.

Your Program

What groups or locations are available for pilot testing your program?
Login to Save Plans for Tools of Change
Which of the measurement methods that you listed in "Setting Objectives" would be appropriate for such a pilot?
Login to Save Plans for Tools of Change
 
2. Decide what measures you will monitor frequently to ensure that your program stays on track.

Examples

 Stepping it Up monitored participation through ‘quick and dirty’ hands-up surveys

BIXI used a wireless communications network to track real-time data on bicycle usage.

COAST, Drama Downunder, and the Healthy Penis Campaign had access to ongoing data on testing rates and the use of online program resources.

In The Clean Air Commute pilot, results were collected by the coordinator and marked on a display at the end of each month.

Your Program

Which of the measurement methods that you listed in "Setting Objectives" could you use frequently or on an ongoing basis?
Login to Save Plans for Tools of Change
 
3. Choose the measures you will track less frequently, e. g., at major program milestones.

Examples

Rossland conducted pre- and post-program surveys and home energy assessments.

Haliburton Communities in Action used a community on-line survey as well as observation counts of walking and cycling

Love to Ride used short, high-response surveys when participants first register to establish their baseline behaviours, then again at 3 weeks and 3 months post-challenge.

BIXI conducted yearly participant surveys

Your Program

Which of the measurement methods that you listed in "Setting Objectives" could you use on a less frequent basis?
Login to Save Plans for Tools of Change
 
4. Wherever possible, make use of control groups & randomly selected groups of people who were not exposed to your program or, at least, not to the parts of it that you are planning to monitor. This helps filter out background changes and influences so that you can be more confident in measuring the changes that are actually a consequence of your program. A less reliable alternative is to compare resource use and/ or people's actions before and after implementing your program.

Examples

Opower created a group of all eligible customers then randomly allocated them to a control group or a task group, which were considered statistically equivalent. The task group received reports and the control group did not.

Smarter Travel conducted telephone interviews with a sample of 1,500 Sutton residents, and an additional 500 in neighboring Croydon borough, which served as a control area (comparison group, Quasi-experimental Design).

North Shore Recycling’s Compost Coaching Program estimated impacts using pre- and post-pilot surveys.

Loreto Bay used the same survey methodology and protocols before and after its campaign to measure changes in knowledge, attitude, and practice.

Tip: Plan how you will select your control group before you engage participants, or you may find it difficult to find the control group you need.

Your Program

How might you set up a control group for your measurements?
Login to Save Plans for Tools of Change