Muir, K. & Bennett, S. (2014). The Compass: Your Guide to Social Impact Measurement.
Sydney, Australia: The Centre for Social Impact.
The primary purposes of outcomes measurement are to provide evidence of what works and what doesn’t, and why and how to improve effectiveness and efficiency.
The 3 Ps to achieve social impact: Purpose (what’s our purpose, why are we measuring impact, what are we trying to achieve?), Process (how are we going to get there?), Performance (have we made impact?).
Indicators are measures that show whether progress is being made on individual outcomes or goals. They may show no, positive or negative change over time. Change might be intended or an unintended side-effect.
Indicators can be qualitative or quantitative. Qualitative indicators seek to understand how the world is understood, interpreted and experienced by individuals, groups and organisations (usually through the eyes of the people being studied and in natural settings). They help to unpack the ‘why’ and are often richly descriptive, flexible, relative and subjective.
Quantitative indicators seek to explain something by using numerical data: how many, how much, how often. They are highly structured and based on theory/evidence and usually objective, but they can also capture subjective responses such as attitudes and feelings.
Common Indicators
If common indicators are used and the outcome data is de-identified and shared, outcomes will be comparable not just at a population level by also at an organisational, group, sector and/or social issue area.
… For example, if your organisation provides housing services, you might track and report tenant housing stability and wellbeing (outcomes) along with information on the client demographics, housing type and other information about your organisation (what you do, how you work, how many people are housed etc).
If similar outcome indicators are used, the housing stability and wellbeing of one group of residents could be compared to other residents in the organisation, in other organisations, in different geographic areas, across the housing provision sector, or to the broader population.
A simple problem can generally be thought of as having a linear cause and effect relationship.
To improve the child with a disability’s social participation, one of the problems the initiative is trying to solve is access to the school playground because of a mobility restriction and steps.
If the problem is that the school playground has steps and needs a ramp, this is a relatively simple problem. The relationship between cause and effect is clear: the steps are causing a lack of access so if you put in a ramp, the outcome is access to the playground.
But problems can be more complicated or complex. A simple problem usually requires a standard approach and the problem will usually be addressed quickly or over time. The solution can often be replicated by others in different situations. Measuring the change that has occurred with a simple problem is also fairly straightforward.
A Complicated Problem
A complicated problem might have a linear cause and effect relationship between the problem and solution. However, there are usually multiple, interconnected components and feedback loops.
The complicated problem might be access and inclusion in a mainstream school that is not set up for a child with a mobility impairment. Modifications may need to be made to the physical space, resource allocation and practices. The problem, however, can be solved over time and outcomes can be measured.
A Complex Problem
A complex problem is one that has many possible interrelated cause and effect pathways. The behaviour of each part will affect other parts and the overall system. Outcomes might be intended or unintended and positive or negative.
An uncertainty that the problem will be resolved, measuring outcomes is also more difficult, and attribution for which group or initiative was responsible for the outcome cannot usually be accurately determined.
Achieving improved social participation overall in this scenario not only relies on practical changes and resource investments, it is also affected by social acceptance, cultural beliefs (e.g. disability is ‘hidden’ within certain cultures), legislation to enforce equal access and the right to live free from discrimination, parent resources (to purchase goods and services needed beyond those accessed publicly) and access to integration supports – to outline just a few contingent factors.
Key Navigation Points
In summary, there are three steps for integrating measurement into your organisation:
- Clarify your purpose
- Determine and articulate the process of how social impact will be achieved
- Measure your performance, the markers of change and the conditions of how this occurs
In undertaking these steps, consider the complexity of the problem and interrelated systems that will affect change.
Resources
W.K. Kellogg Foundation (2004), ‘Logic Model Development Guide: Using Logic Models to Bring Together Planning, Evaluation, and Action’,
https://www.wkkf.org/resource-directory/resources/2004/01/logic-model-development-guide
Baker and Bruner (2010), ‘Participatory evaluation essentials: An updated guide for non-profit organizations and their evaluation partners’, The Bruner Foundation.
http://www.evaluativethinking.org/docs/EvaluationEssentials2010.pdf
