This is a guest blog by James Noble, Deputy Head of Measurement and Evaluation, New Philanthropy Capital (NPC), and Matt Wall, National Secretary, Community Chaplaincy Association.

NPC have been working with Community Chaplaincy Association since 2014 (initially as part of the ‘Improving your Evidence’ project delivered in partnership with Clinks between April 2013-14) to support voluntary sector organisations to improve their capacity to identify, produce and use good quality evidence and evaluation. Below, James and Matt share what they’ve been up to, and how the process could be followed by other organisations in the sector.

What impact does chaplaincy mentoring have on desistance from crime?

James: We have been helping the Community Chaplaincy Association equip themselves to answer this question.

It all started with the development of a Theory of change. This is based on the idea that you can’t measure the impact of a project unless you can first describe who the beneficiaries are, what you want to achieve, and how you will go about it. I have blogged about the merits of this approach before.

When we worked on the chaplaincy theory of change I felt it was a real step forward in how we describe rehabilitative mentoring; firstly because it stresses the importance of building trust with people; and secondly because it shows how mentoring is best seen as process of change, in which mentors deliver different activities and outcomes at each stage of the journey.

Matt: Our Theory of Change has been a valuable reminder of the core focus we have needed to hold on to through times of transformation for all those supporting people leaving prison. By outlining a set of shared outcomes, it has also given us new clarity on what areas of impact we need to measure.

Simply having a theory of change helps organisations to engage stakeholders and funders, but the real benefit is when you start to collect evidence to test it…

A new tool to measure change

Matt: The Theory of Change helped us identify what outcomes are really important to us and led to us increasing our emphasis on the development of the client’s internal strengths and well-being. This meant that we needed to find a way of measuring these factors alongside more tangible outcomes such as housing, finance and employment.

To do this we developed a tool for specifically measuring ‘Internal Change’ which explores areas such as resilience, decision making, motivation and self-esteem. Initial feedback has been that the tool has enabled workers to ‘dig deeper’ with clients and start exploring some of the under-lying issues behind their offending behaviour.

A robust system for collecting data

James: This brings us to the next stage of the project, which was to develop an online system for mentors to record the progress made by all service users through the theory of change. The challenge was finding a system that would work across a widely dispersed network of mentors. Happily, with a wide range of off-the-shelf solutions available, charities no longer need to develop their own systems, and with funding from the Impact Readiness Fund, we were able to purchase iizuka – a package explicitly designed for case management which allows mentors to enter their data offline.

Matt: By collating evidence through the online system we have the potential to report on greater quantities of comparable data from network members in different parts of the country.

This new impact system will have tremendous benefits:

1. It enhances the mentoring relationship by being clear about goals and providing a user-friendly system for tracking progress against them.
2. It’s a really valuable learning tool – with the potential to reflect on what has worked well and share examples of effective interventions to further improve quality standards.
3. It is a huge asset in demonstrating the full value of our interventions to funders and other stakeholders, as well as demonstrating a commitment to testing, learning and improving.

The system is now live in four pilot organisations for which NPC has provided comprehensive training. We look forward to making it available as a national resource for Community Chaplaincies within the next few months and equipping others to share the benefits of a more holistic and robust system of measuring the impact of their work.

Find out more about the Improving your Evidence project and its resources here, and download Community Chaplaincy Association’s Theory of Change here.

The Social Investment Business have just announced a further round of Impact Readiness Funding. If you are interested in applying you can find out more here.

With special thanks to all the Community Chaplaincies who have done the hard work in testing the new system - including Open Gate, Peninsula Initiative, New Leaf, Inside Out and Yellow Ribbon.