Your Impact Evaluation Plan
You’ve had a chance in the previous sections to consider why impact measurement and evaluation matter and what makes them effective. Now we want to help you articulate a plan.
First, if you haven’t done so already, take the Strategic Plan you developed in Chapter 2 ('Planning'), and transfer your impact goals to the Evaluation Plan template. Here’s a way to think about this in three simple steps.
Step 1: Clarify Goals
By now, you probably have a sense of what you hope to accomplish with your impact campaign. You have a sense of what your film’s strengths are, in other words: its core messages and how it’s likely to move people. And you have a few aspirations that relate to these. Now it’s time to make sure they are clear and achievable over the course of your campaign.
Ask yourself: to achieve this goal, who needs to be affected or moved? How do we need them to react and engage? What would change as a result of their engagement?
For more on setting goals, refer to section #1 of the excellent Deepening Engagement For Lasting Impact (pages 6-10), a valuable tool from Learning for Action.
As you proceed with your campaign, and as you collect more data and learn from it, you’ll likely challenge and/or refine some of your Impact Goals. That’s all part of the process!
Step 2: ID Indicators
Next, you want to define your Impact Indicators for those outcomes. In other words, what do you expect is going to change that will help you get closer to those goals? And which of these are measurement-ready? Because, as we’ve discussed in previous sections, not every way a film has an impact can be measured. And sometimes social progress isn’t linear or predictable.
An example of an Impact Goal that might be difficult to measure would be something like: “Get rape on the radar of issues in the US military”.
A better articulation might be: “Get every soldier in the US military to see the film and understand how its themes relate to their lives and choices.” Impact indicators for this goal might include:
- The number of soldiers who viewed the film
- The number of soldiers who engaged in discussion about the film (on or offline)
- The number of soldiers who indicate that they see connections between the story and their own lives
- The quality of post-screening conversations, as reported through qualitative data
Another clear Impact Goal might be: “Ensure rape survivors in the military are able to report abuse without fear of retribution.” Impact indicators for this goal might include:
- Increases in soldiers’ desire to build support for survivors within the military
- New support among lawmakers and military leadership for relevant policies
- Launch of new programs that support survivors who report abuse
- Implementation of new practices that support survivors’ ability to safely report abuse
What you are doing here is identifying observable data that shows change. Some of the indicators above are more immediate or short term than others, and that’s okay. So long as you can clearly trace the role the film plays in the changes that result in your activities, then they belong in your plan.
Step 3: Collect Data
Now that you know what you’re looking for, it’s time to figure out what tools or practices can help you find and collect that data. What are the specific methods of data collection you might be able to use to gather data for each of your Impact Goals? You can start with a single piece of data you could collect for each one.
In order to do this, the main thing you’ll need to do is have a look through the next section of this chapter, the Evaluation Toolbox. Here we’ve got the long list of all the ways we know of to collect data. This will help you understand what’s possible, so you can then apply it to your own challenges.
For more on identifying impact indicators and collecting data on them, refer to section #3 of the Deepening Engagement report (pages 18-28).
Because learning is such a big part of evaluation and impact assessment, what you end up with is ideally a feedback loop. The data you collect helps you to see the role your film is playing with respect to the issues it addresses, which helps you refine your goals and/or the outcomes you’re aiming for, which helps you refine the data you collect, and so forth.
Now you’re ready to start collecting evidence.
