This is what Zenex Foundation did at its Breakfast Seminar in Gauteng on the 11th of October 2017, when the results of the evaluations of its Learner Support Programme was presented. By publicly sharing evaluation results without any sugar-coating, Zenex Foundation showed how serious it is about the evaluation of its programmes. This is congruent with the rigour of the Zenex Foundation’s approach to evaluations.
Having undertaken one of these multi-faceted evaluations over a period of three years, Susannah Clarke and Leanne Adams of Creative Consulting & Development Works, presented the findings to the audience at this Seminar.
One of the comments from the audience at the breakfast meeting was of appreciation for the fact that Zenex Foundation did not try to hide what did not work so well. In fact, the presenters mentioned that Zenex Foundation wanted them to “tell the truth”.
Professional evaluators know that the courage to make a judgement comes with the territory – the 1983 Michael Scriven adage “what is good is good and what is bad is bad and it is the job of evaluators to decide which is which” is well-known in the evaluation community. This seems quite simple, but when faced with high levels of complexity, the good-bad dichotomy becomes an uncomfortable continuum with a fuzzy in-between space where the mist of conditions, dosage, assumptions, unforeseen consequences, changes in the environment over time and many other variables challenge the rigour of an evaluation and the sound judgement of the evaluation team to be able to see the wood for the trees.
Although the judgement element is inherent in an evaluation, it is not always welcomed. Michael Patton refers to the uneasiness with evaluator’s incisive questions in his prelude to his 1997 book titled “Utilization-Focused Evaluation” where he pictures the archangel asking God on the seventh day of creation “…how do you know that what you have created is ‘very good’?”. Then, on the eighth day, God responds by saying: “Lucifer, go to hell”. (See Text Box) Indeed, the evaluator may be the proverbial bearer of bad news, who risks being killed (proverbially). “Killing” the evaluator comes in many forms, ranging from requests to “soften” findings, to omit certain troublesome findings, to obscure certain issues, and even refusal to accept a report. These are risks that come with the territory and are real challenges to ethics in evaluation.
Attempts to influence evaluators’ independence is as shocking as naivety about the politics of evaluation is surprising. When an evaluation is conducted, there is a lot at stake. Hopefully, evaluations influence decisions or rather sound decisions. It is, unfortunately, a reality that, in some, and especially in dysfunctional environments, evaluations could be used for the wrong purpose. It is therefore refreshing to encounter a situation where the data is allowed to “speak”. Only when programmes are willing to hear what has not worked so well, is it possible to learn and improve, and to share that knowledge with others, who may want to replicate it.
The Zenex Foundation approach to take the good and the bad on the chin and to commit to learning is commendable. It is in the mix of the judgement and the learning that progress can be made. The Breakfast Meeting on the Zenex Learner Support Programme has demonstrated very clearly that a programme with an authentic intention to make a difference, will be brave enough to learn.
No comments yet.