MY ‘PIALA’ EXPERIENCE: NINE MONTHS OF IMMERSION
In the Africa Evaluation Blogs started by Zenda Ofir, Glowen Kyei-Mensah reflects on the challenges encountered in piloting our Participatory Impact Assessment and Learning Approach (PIALA) in the impact evaluation of the IFAD-funded the Roots & Tubers Improvements and Marketing Programme (RTIMP) in Ghana.
Sitting across the table from two soon-to-be clients –the International Fund for Agricultural Development (IFAD) and the Government of Ghana (GoG)–, I didn’t think this evaluation would differ much from many others I had done. Our organization, Participatory Development Associates (PDA), thrives on participation. Little did I know that nine months of my life would be submerged in the Participatory Impact Assessment & Learning Approach (PIALA).
The PDA PIALA journey began in 2013 when we applied for a nationwide evaluation of the IFAD-funded Roots and Tuber Improvement and Marketing Programme (RTIMP) of the Government of Ghana (GoG). The proposal submission was dramatic, with Beatrice, a Senior Programme Officer, racing through traffic to submit it on time. Little did we know, this was just the beginning of our challenges.
“What is PIALA?” my team was asking, as we prepared to meet the IFAD consultant, Adinda Van Hemelrijck, who would lead the evaluation design and support us throughout the process. We soon found out that PIALA isn’t just a participatory methodology but a novel approach for impact evaluation developed through an innovation initiative funded by IFAD and the Bill & Melinda Gates Foundation (BMGF). PIALA was new to us, previously piloted only in Vietnam. The novelty excited us.
The challenge of rigour and participation at scale
The evaluation needed to be conducted full-scope and full-scale across the entire country. Expectations on IFAD’s and the GoG’s side were extremely high. I managed it, coordinating three local teams across North, Central, and South Ghana. We planned, budgeted and took a leap of faith, hoping to complete fieldwork in about 35 days on average. At PDA, we had never sent researchers into the field for more than 25 continuous days, so I was concerned. One of the local teams ended up spending 56 days in the field.
For the multi stage sampling and quantitative inquiries, we collaborated with a seasoned statistician from the Ghana Statistical Services, who was as excited as we were to try something different. Investigating supply chains of fresh yam, plywood cassava flour, gari, and high-quality cassava flour with diverse populations of stakeholders was complex but rewarding. Defining and sampling these supply chains and sub-sampling households and stakeholder groups across diverse geographies and contexts was challenging but turned into an interesting piece of work for our statistician.
As PDA is known for participatory research and evaluation approaches, this was where the teams excelled. Despite our experience, the P-factor, Participation, became our biggest challenge in this comprehensive and large-scale mixed-methods evaluation. Using familiar participatory tools on such a large scale without compromising process quality was intriguing. Our local researchers understood the context, culture, and political economy, and spoke the local languages, enabling them to build trust and handle power dynamics under time constraints.
Reinforcing ‘inclusive rigour’
We organised and facilitated 23 district-level Participatory Sensemaking workshops with 650 participants (70 % beneficiaries), and a national workshop with over 100 participants (30 % beneficiaries). These forums allowed local, regional and national stakeholders to cross-validate and debate findings, engage with the data analysis from their districts and learn about the broader picture, and collectively value contributions to impact. Most fascinating were the Theory of Change discussions, in which participants reconstructed the various pathways with the evidence and compared these across the different zones in the country. This exercise went beyond classic validation and resonated with PDA’s principle of inclusion.
The Configurational Analysis was really new to us, and quite challenging but rewarding. Analyzing large amounts of qualitative and quantitative data rigorously allowed us to draw robust conclusions based on different configurations of contexts, mechanisms, and performance.
Rewarding feedback
Despite the ups and downs, completing the PIALA evaluation was one of the most fulfilling professional experiences for my team and me. The icing on the cake came when Robert Chambers, an internationally acclaimed guru of participatory methods, praised our work:“This is the most innovative combination of methods I remember ever having come across. It could be that I am simply out of date and out of touch. Probably, but I hope not entirely. All of you who have been involved - Adinda, PDA, Glowen, Irene, PDA colleagues and others deserve congratulations and thanks. The congratulations extend to IFAD and BMGF for having done what we have been advocating so long and fruitlessly with donors, to fund methodological innovation. I can’t suppress my excitement. What a pa
