Devising success criteria by defining outcomes and outputs
All organisations wish to have a positive influence on the people that interact with their services .
In order to better understand our services’ impact, SLT core staff attended a half day workshop hosted by Think:Learn:Do , facilitated by Voluntary Action Islington, and designed as a cost-effective model for charities to receive impact consultancy. It involved a personalized consultation on our impact measurement approach and outcomes framework, as well as a group session. We share what we learned about impact, success criteria, outputs and outcomes. We also highlight the importance of involving participants in defining some of these key concepts.
Impact: an assessment of your ability to produce change
Think:Learn:Do define s impact measurement as
“a process for assessing a services ability to produce change in its users.”
During the workshop we had an opportunity to review our service’s ‘Vision’ of Intended Impact and Theory of Change model. We were asked to identify any areas of change that could be made more specific and focused .
Success – what does this look like?
Having reflected on SLT’s mission, we identified some initial success criteria during the workshop. These were:
- Long-term improved wellbeing amongst those who have experienced mental distress or social isolation
- Increased wellbeing amongst the population in Islington with mental health issues
- More ‘community’ in Islington for those with mental health issues.
- Increased knowledge and awareness amongst organizations and communities around MH and wellbeing
Following the workshop, we held an internal consultation with Stuart Low Trust staff to review our outcomes framework.
“Outcomes frameworks are tools for informing data collection… they enable organisations to link the data they collect to the outcome it is meant to measure. Outcomes frameworks ensure that data is ‘about’ an outcome, and actually measures it.” Think:Learn:Do .
Here, we did 3 exercises in groups and reconvened at the end to discuss and develop a consensus around what we felt SLT’s priority measurable outcomes were .
Impact outcomes, Engagement outcomes and Delivery outputs: the cornerstone of Monitoring and Evaluation
As a service SLT is participant-centered, so we repeated this consultation process with our participants who identified views on this too. Equipped with insights from our participants on what they felt the impact of the service should be we identified the below list of key outcomes for SLT.
The desired Impact Outcomes of SLT – ‘what success looks like’:
- Fewer people are socially isolated in Islington and surrounding boroughs
- People living with mental distress who are more at risk of being socially isolated, have the opportunity to make more social connections
- Participants experience decreased feelings of loneliness
- Participants experience more feelings of belonging to a ‘community’
Engagement Outcomes – ‘ how people think and feel about our activities ’:
- Participants are satisfied with the services
- Staff/Volunteers are supported with Personal Development
- SLT are leaders in Community of Practice- we share are knowledge and expertise with other professionals and organisations
Delivery Outputs – ‘what and how we deliver our activities ’:
- SLT services are inclusive
- SLT services reach those who are / have been Socially Isolated
- SLT services reach those experiencing / have experienced Mental Distress
- SLT provide activities for participants outside of office hours
- SLT services reach local people
Loneliness, wellbeing, and social isolation: impact indicators
There is a strong correlation between a person’s feelings of loneliness, wellbeing, and social isolation. For example reduced feelings of loneliness imply better wellbeing; and those who are socially isolated are more at risk of having feelings of loneliness.
As such we can identify a few secondary factors that would be intrinsically linked to the above desired impact:
- Improved self-confidence
- Increased knowledge
- Improved overall wellbeing
- Feelings of contentment in oneself
- Number of positive connections
These are helpful indicators that could help monitor whether we are achieving our desired impacts.
Why is it so important to involve participants?
Participants are much better placed in answering questions around what they need. In our case we were able to understand the participants perspective on:
- What success looks like in participants own words – allowing us to model impact
- Wording – to ensure we don’t use triggering or excluding language
- Collecting feedback – and how to get better engagement in the feedback collection proce s
Our Next steps
Covid-19 and the resulting social distancing measures have made it more difficult to consult with participants. Once the Covid-19 restrictions have lifted we intend to do a further consultation with participants on the format of our new M&E e.g. in person questions.
However during this time we launched a new online version of our M&E forms which we only circulate on a termly basis. We have adapted out outcomes framework on a temporary basis to ensure we continue to monitor our impact through the changes in circumstances. Look at our reports section to find out how what this looked like in practice.
How you can better define impact and success criteria
Invite staff, volunteer reps, participant reps and trustees for discussions on impact. Ensure that there is a diverse set of voices that represent difference experiences across your organisation. Work with them to:
- Define the key concepts relating to impact and success criteria
- Explore what success looks like for your beneficiaries
- Think about how you want people (your staff, volunteers and beneficiaries) to think and feel about our activities
- Consider what and how you deliver your activities.
