Client: Tshikululu Social Investments
Duration: April 2013 – January 2014
The FNB Fund Community Care Programme (CCP), through their corporate social management consultancy Tshikululu Social Investments (TSI), appointed Creative Consulting and Development Works (CC&DW) to undertake an outcome evaluation of the programme from 2010 -2013. The overarching goal of the evaluation was to assess the effect of the ten projects in their communities according to priority CCP objectives. It also aimed to assess the effectiveness of the project partners in implementing the CCP from 2010 – 2013, identifying strengths and challenges and lessons for future strategy planning.
PROJECT NAME: FNB Fund Community Care Programme Evaluation
CLIENT: Tshikululu Social Investments
PROJECT DURATION: April 2013 – January 2014
SECTOR: Corporate Social Investment, Community Care, Gender Based Violence, Youth At Risk, Family Preservation
SERVICE PROVIDED: Evaluation
The evaluation aimed to establish a retrospective baseline to measure the outcomes achieved by FNB funded project partners. The aim of qualitative research is to understand and describe opinions, feelings and experiences without manipulation – in the case of the CCP, to focus on the experiences of beneficiaries, as well as staff of the programme. Data collection focused heavily on qualitative feedback from beneficiaries, key informants and staff of project partners. Evaluators conducted fieldwork with 258 individuals to collect data on the outcomes achieved by the project partners. To establish a baseline, evaluators relied on programme documents as well as beneficiary and key informant reporting on the needs and problems of the community prior to their involvement with the programme. All qualitative data was thematically analysed with the assistance of computerised qualitative data analysis software, Atlas Ti.
The evaluation was the first external evaluation study completed for the 2010 to 2013 funding round for the FNB CCP. The evaluation findings were centred around the assessment of the FNB CCPs achievement in relation to their strategic objectives and on the levels of relevance, effectiveness, cost analysis and recommendations.
The evaluation provided key findings around administration of the grants including recommendation to the monitoring and evaluation process being implemented by TSI. Additionally, the report provided rich qualitative stories of change of individual beneficiaires. Key recommendations and lessons learnt were collated for project partners – taking into account the limited time the evaluation team was able to spend with them. Most importantly, the evaluation provided suggestions around strategic alignment between the FNB CCP as well as project partners.
KEY LIMITATIONS OF THE EVALUATION
Key limitations of the evaluation included the lack of real baseline data, as census data is not collected during the exact time of programme start and does not target a specific enough area to relate to CCP projects and unreliability of recall data. Additionally, the lack of a real comparison group meant it was not possible to directly compare the beneficiaries to people who did not receive services. However, due to the high quality of the qualitative data collected, beneficiaries gave key insight into the change in their situation as a result of interaction with the FNB funded programmes. Additionally, a weakness of the evaluation was the limited amount of quantitative and financial data made available by project partners – this made it difficult to comment on the amount of funding allocated per strategic objective and compare the relevance and effectiveness based on cost analysis. Quantitative data reported by projects was inconsistent from year to year as well as project to project. Additionally, data reported has not been quality assessed – even in some cases where there is clear over-reporting. Therefore, the quantitative data drawn from the project partner documents is extremely unreliable and does not lend itself to comparison. The data has been presented, however, it must be highlighted and emphasised that these figures should be viewed with caution.