STEP 7: Measuring Achievements

In this section you will map out ways of monitoring the achievement of your objectives to ensure that your program stays on track. This will also help you improve your program on an ongoing basis, and justify requests for project funding.

You will:

  • Consider the use of pilot testing.
  • Decide what measures to monitor frequently.
  • Select the ones you will measure only at major milestones.
  • Consider setting up a control group.

Note for health promoters.

 
1. Whenever practical, pilot test your program before implementing it widely.

Examples

The Environment Network pilot tested its starter kits with 150 participants.

When Go Boulder first introduced its Guaranteed Ride Home, it pilot-tested it with a small number of companies. No limits were set on its use so that any possible abuses would become apparent. Similarly, its student transit pass was first introduced at the University of Colorado on a one-year trial basis.

The Roach Coach Project approach was first piloted in Toronto, then promoted to others for their use.

Resource Alert for Canadians:
Canadas EnerGuide for Houses Program was first piloted with a few organizations such as Green$aver, before being offered more widely across the country.

Your Program

What groups or locations are available for pilot testing your program?
Login to Save Plans for Tools of Change
Which of the measurement methods that you listed in "Setting Objectives" would be appropriate for such a pilot?
Login to Save Plans for Tools of Change
 
2. Decide what measures you will monitor frequently to ensure that your program stays on track.

Examples

In The Clean Air Commute pilot, results were collected by the coordinator and marked on a display at the end of each month.

On an ongoing basis, JEEP monitored energy outputs at the local power plant and estimated energy savings using information from home visit reports.

Quinte Regional Recycling continuously tracked curbside lift counts, the weight of waste going to landfill, and the weight of Blue Box materials.

Resource Alert for Canadians:
EnerGuide for Houses facilitates the ongoing measurement of energy-efficiency improvements resulting from the Program. Its blower door test provides accurate before and after data on the decreases in air leakage achieved for each house. Even if there is no second (i.e. after) home visit, the programs software can use the collected data to make sound estimates of the likely energy savings from each recommended action taken.

Your Program

Which of the measurement methods that you listed in "Setting Objectives" could you use frequently or on an ongoing basis?
Login to Save Plans for Tools of Change
 
3. Choose the measures you will track less frequently, e. g., at major program milestones.

Examples

When employees arrived at work on the day of The Clean Air Commute, they marked the activity they undertook on a chart. Pollution Probe collected the charts and tabulated accumulated points. A telephone survey was conducted at the end of the Clean Air Commute pilot to collect information on clean commuting practices undertaken by members of each of the groups during the three-month period. Respondents were also asked about their intentions for repeating these practices the following summer.

The Environment Network used a telephone survey at the end of its pilot to measure purchasing intentions regarding pesticides, synthetic chemical fertilizers, and toxic cleaning products.

Go Boulder conducted a Biennial Diary Study, for which 1,000 residents kept a log of their travels for a randomly assigned day in the second week of September. Six years of data on the progress of the program had been accumulated by 1995. Similar studies were undertaken with employees, who were asked to record their travel habits for a full week.

Quinte Regional Recycling conducted periodic studies on waste composition and the use of backyard composters.

Your Program

Which of the measurement methods that you listed in "Setting Objectives" could you use on a less frequent basis?
Login to Save Plans for Tools of Change
 
4. Wherever possible, make use of control groups & randomly selected groups of people who were not exposed to your program or, at least, not to the parts of it that you are planning to monitor. This helps filter out background changes and influences so that you can be more confident in measuring the changes that are actually a consequence of your program. A less reliable alternative is to compare resource use and/ or people's actions before and after implementing your program.

Examples

Of the seven companies which agreed to participate in The Clean Air Commute pilot, three were randomly assigned to serve as a control group.

The Environment Network randomly selected one hundred households that had already received a Green Home Tune-Up; 50 received a sustainable landscaping kit, 50 received a household hazardous waste kit. A third group of 50 residents who had participated in the Green Home Tune-Up program were randomly selected as a control group, and received neither kit.

In one Montreal Dietary Dispensary study, two children born to the same mother were compared, the first untreated and the second (born after the first) treated with the Higgins Method. In another study, the pregnancy outcomes of treated adolescents were compared to those of women who were untreated but were the same age and gave birth in the same year at the same hospital as the treated group.

The AIDS Peer Education Program used a control group of 74 students attending another high school in another city. The comparison group received the same questionnaire with no intervention. For ethical reasons, the peer educators later visited these students and provided the presentation.

Tip: Plan how you will select your control group before you engage participants, or you may find it difficult to find the control group you need.

Your Program

How might you set up a control group for your measurements?
Login to Save Plans for Tools of Change