Five stages of Building a Local Measurement & Evaluation Team

As Monitoring and Evaluation (M&E) Program Manager for Nuru Kenya, I consistently work with my team to build up different strengths that enable us to objectively monitor and evaluate Nuru Kenya’s impact programs. I want to share my experience building a credible and sustainable M&E system in Kenya. Below are the five critical stages our team has experienced in the last five years.

1. Define what impact means for your organization

Back in 2011 when I first joined Nuru, I was the only local staff from Kuria West (where Nuru Kenya operates) in the newly established research program. I worked under the guidance of Jamie Frederick who was an expatriate strategic advisor from Nuru International. Jamie helped me set up systems and processes as well as design and implement strategies aimed at measuring impact programs’ effectiveness and scaling.

2. Understand your organization’s approach

Nuru Kenya provides robust agriculture, financial inclusion, healthcare and education programs to smallholder farmer households. In my initial weeks at work, I was completely preoccupied with questions like:

• How are Nuru Kenya’s programs doing?
• How are its theories of change articulated?
• How effective or ineffective, adequate or inadequate are they?
• How can Nuru Kenya accomplish its goals?
• How can people be involved in making sense of the data generated by M&E processes and in assessing the implications for the project strategy and operations?
• What contribution should I offer?

With these questions lingering in my mind, I began reading programs’ logic models and holding discussions with impact program managers programs about two topics: fine tuning logic models and designing M&E framework to guide M&E strategy.

3. Implement baseline surveys

In order to effectively support Nuru International’s overarching mission to end extreme poverty in remote, rural areas, Nuru Kenya had to implement a baseline poverty assessment. As Nuru believes poverty is multifaceted, our team selected a well validated, standardized tool called Multidimensional Poverty Assessment Tool (MPAT) to fully understand the principal underlying causes of poverty in the communities where Nuru Kenya works. After its first implementation, our team thought critically about the practical implications of gathering the required information and how we would use it to assess impact. During this baseline phase, we evaluated what information could realistically be collected, summarized and analyzed given available human and financial resources.

4. Build a sustainable team and processes

We learned several lessons from our initial experience with the MPAT that prompted us to reflect critically on new strategies. While we no longer utilize the MPAT to measure poverty, we still were able to improve operations from this initial experience:

• Firstly, the MPAT took a long time and it was not efficient to conduct an evaluation for two months. If this trend continued, then M&E would be unable to meet its goal of timely and effectively supporting all four impact programs.
• Secondly, there was urgent need to hire more staff to alleviate staff deficiencies and improve the quality of our data processes through enumerator training and field supervision. The latter were supported by the development of a standard curriculum and user guides for each impact assessment survey. To build out our team, we ensured that everyone was clear about his/her responsibilities. This also helped us to avoid gaps and contradictory efforts.
• Finally, it was desirable and also sustainable to have local staff engaged fully in the entire evaluation processes from collection to analysis. Working closely with Nuru International expatriates, our local team instituted capacity-building initiatives and embraced them consistently through in house training. Additionally, Nuru International facilitated exchange program visits to organizations with reputable M&E systems, such as One Acre Fund, as well as attending international M&E conferences, such as the African Evaluation Association Conference. These opportunities broadened our networking circle.

5. Control, process and disseminate quality data

One of the final processes we worked hard to perfect was data management. In order to ensure data integrity, our team applied basic quality control procedures at different levels.

• In translating the survey from English to Kikurian, the local language, we strive for conceptual equivalence in order to stay true to the meaning of the questions. Staff use pilot studies to correct discrepancies and errors emanating from translation in order to have an intimate understanding of surveys and the intentions of each question.
• During data collection, various team members check, score and code raw paper data, with each stage completed by a different person for a high degree of quality checks prior to the actual analysis.

As we strive to provide timely data for effective decision making, we have found that we can shorten the final communication of analysis to programs through proper planning, developing Stata analysis files well in advance and commencing data entry as soon as data collection starts.

With these strategies in place, and with competent, well-motivated staff, the team has been capable of executing quality evaluations. The team has become so skilled at this that we’re currently capable of implementing multiple surveys simultaneously.

Moreover, we have institutionalized proper feedback mechanisms to quicken reporting to program managers and other stakeholders. We have established systematic meetings as part of our communication strategy with impact programs to provide forums for sharing evaluation and monitoring formal reports, seeking feedback about interim findings as well as addressing program specific challenges on how to use information for results based management.

Overall, the questions I posed at the beginning of my journey with Nuru Kenya still linger and continuously evolve.

The most important aspect of my time with Nuru Kenya, however, has been the transition of leadership, management and technical skills—first from Nuru International expatriates to me and more recently, from me to my team.

Because of the knowledge passed down, Nuru Kenya M&E is fully equipped to objectively monitor and evaluate its impact programs.