I recently came across this gem: a report by International Initiative for Impact Evaluation (3ie) titled ‘Behind the scenes: Managing and conducting large scale impact evaluations in Columbia‘. This is a current discussion – the paper is dated December ’11. There is substantive investment in IE in Colombia and they are seeing the real benefit of investment in IE.
The focus of the authors Briceño, Cuesta and Attanasio was development projects in Columbia, and I think there is some useful guidance that would apply to the Student Services sector.
I particularly like this concluding section, “Five Key Lessons”.
Five Key Lessons
1. Invest in the preparation of good terms of reference (TOR).
I agree, it is essential to think hard about exactly what you want to find out when starting to design the process of an evaluation.
2. Decide on the best methodological approach to address the evaluation questions.
AMOSSHE’s Value & Impact Bank is a resource that may help you here. I have collated evaluation materials used by UK HEIs which can give you an insight into how others have approached improving the impact of their services. The AMOSSHE Value & Impact Bank is a member’s resource and you can find it here: http://www.amosshe.org.uk/projects/vip/bank.
3. Ensure evaluation quality.
You may want to meet with professional researchers in your institution to discuss research methods – a resource on your doorstep.
4. To foster evaluation ownership, lay out the incentives for dialogue between the involved parties.
To put it another way, do not fear tackling head-on the questions “what’s in it for me? / What’s in it for you?” with your team for a positive process and outcome.
5. Conduct quality dissemination.
In other words, share your findings. A small note on the departmental newsletter will likely not do justice to your efforts in evaluation. Be creative about the best ways to share your findings.
You can download the full report for free, here.*
*The emboldened conditions for successful IE on pp6-8 are worth looking at – they very much echo the issues around implementation that our pilot Value & Impact HEIs experienced.