Q&A WITH ASYA TROYCHANSKY, ROOT’S IMPACT ASSESSMENT OFFICER.
Asya Troychansky, Root Capital’s Impact Assessment Officer, recently returned from Guatemala where she has been overseeing Root Capital’s Cluster Study.
Due out in spring of 2014, the Cluster Study builds on Root Capital’s previous impact studies on clients such as COOMPROCOM in Nicaragua, COOPCAB in Haiti, and Fruiteq in Burkina Faso and will help us, our clients, and collaborators understand impact on the ground. Here, in the first installment of a series on Asya’s work, she details the study, reflects on the challenges she and her team of consultants have encountered and explains how deep-dive studies like this further our understanding of Root Capital’s impact.
Q. Please give a brief, high-level summary of the study.
The Cluster Study, as we’ve come to call it, is actually four separate deep-dive studies, each with a different rural coffee cooperative client in Guatemala, that we conducted in collaboration with the Multilateral Investment Fund and the Committee on Sustainability Assessment (COSA). Between July and October, a talented local team of researchers who we hired and trained conducted field research using mobile devices in remote villages in Guatemala’s highlands.
Like our other impact studies, the research delved into two main questions:
- What are the impacts of our clients on the small-scale farmers that supply to them and on the farmers’ families and their communities?
- How does Root Capital, through our lending and financial advisory services, help our clients to amplify their impacts?
We administered a survey to 400 cooperative members and 200 unaffiliated farmers, which, beyond impact, gives us an opportunity to deepen our understanding of farmers’ socioeconomic situations, their on-farm production practices, the challenges they face, especially in light of the devastating coffee rust epidemic, and their aspirations for the future. As part of our Women in Agriculture Initiative, we also zeroed in on the different roles and impacts for men and women cooperative members.
Q. Why a “cluster study”?
With a diverse portfolio of more than 200 clients working in 25 countries and 36 industries, no study of a single client will definitively prove our impact. Therefore, we’re examining impacts in our portfolio with strategic assessments by sector and region. We’re also using various approaches and tools, such as the Progress Out of Poverty and Coping Strategy indexes to look at poverty and food security, respectively, and pilots with various mobile tools. Coffee in Central America is a logical focus area because Central American coffee clients comprise 28% of our entire portfolio (coffee clients comprise 69%).
By grouping several studies in the same country and sector, we’re also able to look for common results that are not a function of one group’s unique situation, but that are likely to be more broadly relevant in the Central American coffee context.
Q: Why is the Cluster Study important for Root Capital?
Deep dive studies allow us to obtain granular data on specific communities and farmers. They supplement the portfolio-wide data we collect through our social and environmental due diligence process, which give us a bird’s eye view of client benefits to producers but is often self-reported or estimated.
Deeper studies like the Cluster Study verify and elaborate in more detail on specific impacts, such as payments to producers, for a subset of clients. For example, incorporating the Progress Out of Poverty 10-question index into our surveys has allowed us to corroborate the income level that Loan Officers estimate as part of due diligence, giving us a more accurate idea of the farmers our clients serve.
Q. How does this study differ from Root Capital’s previous impact studies?
For Root Capital, the Cluster Study marks impact assessment 2.0, as we’re aspiring towards greater generalizability through simultaneous studies with four different clients and broader comparability by aligning our questions with COSA and other partners. We also introduced a comparison group (the 200 unaffiliated farmers) to identify the effects of enterprise interventions on producers and their families.
We won’t be able to prove impact, say in yields or incomes, if differences emerge between members and non-members because we are not conducting a randomized experiment. But having a control group, even an imperfect one, is a big step forward for our studies and will help give us an approximate sense of impacts.
Q. What challenges have you faced while designing and conducting this study?
What made this study complex was implementing the same plan – identical methodologies and timeline – with four groups in very different contexts. The first challenge was language. The survey, written in English, translated and adapted to Guatemalan colloquial Spanish, then had to be applied for a subset of surveys in the indigenous languages of K’iche, Ixil, and Mam. The first step of finding a native speaker to conduct the research was particularly difficult for Ixil, which is spoken in just three towns in the Guatemalan Highlands. Although we ultimately identified a researcher, he had to adjust his accent and vocabulary to communicate effectively in the local variant of the language in another town 45 minutes from his.
Our consultants also faced logistical hurdles in the field. In one group, for example, producer members were extremely dispersed in communities many hours apart and often not connected by main roads. Another group near the Mexican border was difficult to survey, first because of a local police operation against a drug trafficking group, later because of a multi-day town festival. Applying the surveys digitally using a mobile device, while cutting down on the time needed for data entry, created issues during power outages or in areas with limited electricity. In the latter half of the data collection, the rainy season also entered in full force, with mudslides and sinkholes making travel and monitoring visits that much more challenging.
Recruiting control groups to serve as a point of comparison for cooperative members was also difficult. Ideally, our control group would mirror cooperative members exactly, with the only exception being their cooperative membership status. Yet, we needed a more practicable strategy that balanced rigor with field realities. After conferring with the cooperatives and our researchers, we decided on a combination of interviewing incoming cooperative members for the subsequent harvest (strongest control group), members of neighboring coffee cooperatives that weren’t Root Capital clients (moderate control group), and producers who decided not to affiliate but were from the same communities as members (moderate to weak control group).
Q. What has been your favorite part about leading this study?
It’s been extremely fulfilling to get to know these cooperatives in many dimensions: their successes, their warts, challenges overcome and those yet to overcome, and the diversity of the members who comprise these groups. I’ve enjoyed visiting the groups and meeting periodically with the research team to hear about everything they’ve learned through surveys and meetings and activities in the communities. (Now the exciting task for us will be to synthesize all of these results!) In the process, I’ve also loved the food in the campo, and have logged many eggs, beans, and tortillas for breakfast and dinner.
