A blog by Professor Sharon Collard, Research Director at the University of Bristol’s Personal Finance Research Centre – part of the What Works Fund Evaluation Learning Partner (ELP). The ELP includes Ipsos MORI, Ecorys and the Personal Finance Research Centre.
With the main What Works Fund programme coming to the end of its delivery phase, the focus now is on producing good-quality data analysis and final evaluation reports. Delivering useful evidence is at the heart of the What Works Fund programme – to strengthen the evidence base and help shape future financial capability services. The second WWF Webinar was an opportunity to consider how projects and external evaluators can generate useful evidence from their evaluations, with the support of the ELP and MAS.
We had 50 participants take part in the webinar and their feedback was positive – they found the webinar useful and 1.5 hours was generally felt to be about the right length of time. In the webinar, we heard from two WWF projects that have worked with external evaluators:
- The Money Charity, working with NFER, has designed and implemented a randomised control trial to evaluate the impact of its Money Workshops in secondary schools
- MyBnk, working with Substance, has used a mixed methods approach to evaluate its Kick Start Money programme in primary schools.
Among the learning from their experiences, these projects highlighted:
- The tricky job of getting the balance right between adapting delivery and evaluation to real-world contexts (in their case schools) while still robustly testing their Theories of Change and gathering meaningful data on financial capability outcomes.
- The challenge of making sense of different data sources and analysing data from multiple datasets.
- The importance of close collaborative working between projects and external evaluators, throughout the analysis and reporting process.
The webinar also touched upon other considerations for projects and external evaluators to guide their analysis and reporting such as taking care not to over-state or over-interpret outcomes and impacts; and reporting on the factors that help explain a project’s outcomes and impacts e.g. the profile of users, how the intervention was implemented, and the external environment in which projects operated.
You can listen to/view the webinar here.
The next webinar is planned for March, on the topic of sharing and learning from the WWF.