When managing a project or delivering training it is important that we know what our manager, sponsor or client needs from us. Clarity is required. Without it we are unlikely to deliver a good quality product.
There is exactly the same requirement when undertaking an evaluation of a training or learning event. But for some reason evaluators often obtain the information they think is appropriate rather than what the sponsor requires.
Here are the 10 key questions to ask your sponsor in order to unlock their exact requirements and ensure you deliver what is required:
1. Do you require statistical or anecdotal data? A person requiring statistical information probably wants to be able to scientifically back up the findings ensuring it is provable and measurable. If the sponsor prefers anecdotal information they may well be looking for thoughts and impressions as to how it appears to others.
2. Do you want simple answers to simple questions or complex process issues to be addressed?
3. When obtaining data, do you want a large sample size of respondents to be used or a case study approach? With the former we would interview or question lots of people, with the latter we would follow individuals or ideas from start to finish, like a thread.
4. Is the report intended as a management tool or a practitioner’s tool? In other words, is it for the sponsor or another manager to use to improve a situation or is it for the providers to help them?
5. Am I, as the evaluator, required to make a judgement with sanctions, or am I non-judgemental in my approach? With the former, I may be required to report on whether people are following a specific policy and name those who were not. With the latter, the sponsor does not want people to feel threatened and wants them to develop issues.
6. Are my findings to be specific or anonymous? This indicates whether the sponsor is more interested in who is doing (or not doing) what or more interested in what staff really think about the issues.
7. Is it a set agenda or a progressive agenda? The former means the evaluator sticks to the initial remit, the latter gives permission to go with new or unexpected issues that arise.
8. Is the final report (assuming there will be one) to make recommendations or pose questions? The former gives the readers a clear idea of what is required whilst the latter involves everyone in how the goal or noteworthy practice can be achieved.
9. Is the information gathered to be closed or open? With the former, only the sponsor gets to know about the findings, whereas with the latter everyone knows tham as they become apparent
10. Is the feedback to be ‘official’ (summative) or ‘formative’? The former would usually be a full report with recommendations to act on whilst the latter would be a product enabling everyone to understand what is happening so that those involved are in the best position to decide where to go to next.
As you will have seen, all the questions contain alternatives. If your responses are all towards the first alternative within each question, you will be undertaking what can be called a ‘Bureacratic’ evaluation. If your sponsor’s preferences are more towards each of the second alternatives, your evaluation methodology will be more ‘Democratic’.
Get this right and you have sound foundations for your work. Get this wrong or omit to do it, you are almost guaranteed problems. In my next post I’ll explain why, and how, to avoid the potential pitfalls.
Paul