Elements of a successful ECB intervention in Africa: Participant’s perceptions.
Introduction
The last decade has witnessed the growth in accountability, especially on the side of donors who demand to understand how their funds were utilized to deliver respective results. This has led to a precedent increase in the demand of employees with skills in Monitoring and Evaluation (M&E). Expertise not only increases projects delivery, it also helps assess program performance, impact, results and its sustainability. At the continental level, the growing trend of results-oriented development, has led to the mushrooming of M&E programmes. This is also in response to the fact that both donors, and citizens are expecting their governments to be accountable. Despite the boom in M&E trainings offered both onsite and off-site learnings, less research focusing on soliciting participant’s perception in terms of what works in M&E training programmes have been undertaken. This blog documents the perceptions of participants of the Development Training Programme in Africa (DETPA) cohort delivered by the Centre for Learning on Evaluation and Results (CLEAR-AA). A mixed method approach which entailed semi-structured interviews and survey was used to solicit perceptions of the three cohorts for the period 2017 to 2019 who were enrolled in the programme. It is authored by Mokgophana Ramasobana (Programme Convener) and Nagnouma Nanou Kone (2019 participant).
Overall, the research questions were aimed at determining what worked, what did not work, what should be retained and what should not be retained in relation to the programme. In summary, the following constructs were documented as findings. Below the findings are presented.
1) Kindly enlist what was successful about the programme
(a) Top class coordination process and organisational ownership
A well-thought implementation process and organizational ownership was demonstrated throughout the delivery of the programme. For example, one of the respondent indicated that “the provision of the logistical note prior the commencement, and the entire coordination of the programme was professed to have been well executed by the programme team. Another respondent further corroborated this sentiment by stating that “I can’t emphasise the importance of how organized the programme was. It was really a highlight, the support of the entire CLEAR-AA “team work showed, and I was impressed”
(b) Contextually-fitted curriculum content
“The two tracks (fundamentals and advanced) approach which catered for new entrants and experienced practitioners was a highlight for me”. This ensured that the modules were sequenced and structured appropriately and fit for purpose. One of the respondents alluded that “the use of case studies, site visits, peer learning and group work were some of the learning approaches that scaffolded her skills and knowledge during the delivery of the programme. Another respondent gave a similar response by stating that “the Department of Evaluation and Monitoring (DPME) site visit was a highlight because it entrenched the concepts discussed in the classroom with a practical National Evaluation System (NES). Most importantly, the Made in Africa Evaluation (MAE) module was highlighted as one of the key components of the programme.“This module gave account of Africa’s evolution and how M&E fits in the broader development” said one respondent. This is a particularly important aspect as its reflects the objective of the programme, which is “to build a community of M&E professionals equipped with skills, knowledge and tools, which are fit-for-purpose to address local and global development challenges”.
(c) The quality of facilitators
The use of experts with technical expertise in various fields of M&E provided enriching theoretical and practical knowledge to the participants, enriching the delivery of the programme. As an illustration, one of the respondent asserted that “facilitators presented concepts in-depth as well as provide relevant examples”. A different respondent further noted that “dual facilitation during sessions contributed to the success of the programme”. In addition, it was proposed that other institutions should consider applying this approach in delivering training programmes as such a mix approach responds more to the needs of participants from diverse backgrounds, skills and countries.
2) Kindly enlist what was not successful about the programme
(a) Pre–survey
Some of the concepts such as systems thinking remain complex and difficult to decipher. Therefore, one respondent recommended CLEAR-AA to “consider conducting a pre-survey in order to gauge the level of participant’s understandings prior the commencement of the programme.” This is envisaged to ensure that concepts are pitched on par with participant’s levels of understanding, expectations, as well as to ensure that the modules are tailored to cater for diverse different contexts.
(b) Time
The programme duration was considered short. One respondent argued that “in some cases lots of theories were covered with less time allocated for the application” Another participants mentioned that “there were time constraints witnessed during the site visit”. This was caused by traffic delays between Pretoria and Johannesburg.
(c) Standardize the facilitation style
A minority of the facilitators had poor facilitation skills. For example, “one of the facilitator was perceived to have not been eloquent and used traditional teaching style to lecturer”. It was therefore recommended that “CLEAR-AA standardize the facilitation style with all facilitators”. Standardization of the teaching style among facilitators will help to curb such minor occurrences.
3) Kindly enlist what should be retained about the programme
(a) Curriculum and learning approach
Majority of the respondents consented that the curriculum structure and the learning approach applied in the programme should be retained. For instance, one of the participants mentioned that “Learning how to package the evaluation reports especially how to report to various stakeholders. This includes steps on how to commission evaluations. The decolonisation seminar was useful therefore it should be retained. This will assist practitioners to adapt existing concepts and frameworks as well as empower them to navigate their practices. Most importantly, the site visit enlightened me because it illustrated how a complex M&E system works”.
(b) Two different streams/tracks
The two streams offered by the programme should be retained. In emphasising this sentiment, one respondent cited that “I like the fact that there are two streams/track. One focuses on theoretical concepts whilst another pays attention to technical approaches”. Beyond the classroom, one of the respondent proposed that “CLEAR-AA should think about a platform that connects the alumni’s as well as avail space to learn from each other and beyond the programme”.
4) Kindly enlist what should not be retained about the programme
(a) Mixed reactions
In answering the question of what should not be retained, different responses were solicited from the respondents. Although most of the respondents mentioned that all components of the programme should be retained. As an illustration, one of the respondent mentioned that “the programme was well structured and learning occurred”. On the other hand, some of the respondents raised issues relating to the heavy curriculum content. For example, one respondent mentioned that “the programme was quite content heavy. A multiplicity of events such as lunchtime lectures, evening activities, and weekend tours etc., were overwhelming as they provided less time for processing the information taught. Based on the sentiments, recommendations such as “CLEAR-AA reducing the intensity of such activities. Rather explore more social activities after the classrooms so that people can relax and connect in a relaxed mode outside the formalities”, was proposed, in addition to extending the programme duration.
Although the findings of this study cannot be generalized. However, there are few conclusions that could be inferred as well contribute to the growing discourse of evaluation capacity building in the region. Firstly, the authors argue that these responses reinforce the importance of the evolving field of M&E, and the urgency to customize training initiatives which are in sync with the skills needs of African practitioners. Secondly, there is an increasing acknowledgment that M&E is an evolving field in Africa, therefore, suppliers of training are urged to be cognizant of the context in which they are operating in. Lastly, the key conclusion drawn from this blog is that the continent requires more skilled personnel trained in M&E in order to track implementation and outputs systematically, and measure the effectiveness of programmes. Therefore, it remains important that training programmes provided are fit for purpose and contextually relevant.
By: Mokgophana Ramasobana and Nagnouma Nanou Kone
The Republic of Ghana Public Sector and M&E Culture
In 2017 the Government of Ghana added a new tool to its development process by establishing the ministries for Planning (MoP) and for Monitoring and Evaluation (MoME). The MoME and the NDPC (National Development and Planning Commission) and the local M&E Association – the Ghana National Monitoring and Evaluation Forum (GMEF) – have developed a National Monitoring and Evaluation Policy. Once cabinet approves the Policy, it will be an important instrument in fostering the culture and practice of evidence-based decision-making in the public sector. The institutionalisation of M&E calls for better planning, use of resources, implementation and learning from outcomes about what worked and what did not.
With the establishment of the MoME and the recent launch of GIMPA’s (Ghana Institute for Management and Public Administration) Masters programme for evaluators, a better coordinated ecosystem for evaluation appears to be forming. However, the government system still faces many challenges, such as:
- weak M&E capacities,
- low demand for, and utilisation of M&E results, limited resources and budgetary allocations for M&E,
- non-compliance with M&E reporting timelines and formats by MDAs (Ministries, Departments and Agencies) /MMDAs (Metropolitans, Municipalities, Districts and Agencies)
- inconsistent data quality,
- data gaps, and
- limited management information systems.
Given that building evaluative thinking and systems is a relatively new endeavour, research into the current M&E culture in the public sector would prove useful. It is for this reason that the Ghanaian government partnered with Twende Mbele to undertake a study to establish a baseline of the M&E culture in the public sector that can be used to measure changes in practices and attitudes over the coming years. The study included interviewing 43 senior management officials from 14 ministries and two agencies using a formulated survey. The results below are based on the Report on the Ghana M&E Culture Baseline Survey.
All (100%) respondents to the survey indicated that evaluation reports are not structured to hide results. Futhermore, most (95%) reported that evaluation results of poor performance are not ignored and 92.50% that senior management does not reject evaluation reports with findings of poor performance.
The MoME is currently working on frameworks and policies to standardise and improve use of M&E across government. This includes the development of a mechanism to obtain information to understand the cause of poor performance. The above suggests a positive management attitude and environment within which to practice M&E in the public service.
According to survey results, recommendations are implemented and learning outcomes are documented and used to improve future results – 92.50% of respondents stated always. The implementation of performance management and performance contracts with managers of the ministries has yielded some results as performance measurement and results are now a basic requirements by Senior level Managers, Ministers and Cabinet for decision-making. There was however, a perception that the officers responsible for poor performance are not sanctioned, (32/38 of respondants stated never), which needs to be addressed in order to motivate staff. The report therefore recommended that a rigorous sanction and rewards regime must be implemented to regulate staff performance in the Ghanaian public service.
An enduring M&E system cannot be developed overnight. However, the Ghanaian government has proven that it is committed to pursuing a desired change based on the work of existing institutions including NDPC, MoME and GIMPA. It has been a steady process and one of the building blocks required was behaviour change. There is also the need for a paradigm shift towards the incorporation of M&E into the routine work of public service staff in addition to deepening M&E processes by which public service activities are conducted in order to build a stronger, learning-focused M&E Culture in Ghana.
By: Rendani Manugu
Twende Mbele, South Africa, Country Coordinator