Deja Boo Hoo - Have We Learned Anything from Impact Evaluations?

Grab a cup of chai (or two), this is a long one…

On Wednesday, Dr. Brigit Helms, CEO of Unitus published a scathing op-ed in the Seattle Times on behalf of a consortium of microfinance practitioners, including ACCION International , FINCA, Grameen Foundation, Opportunity International and Women’s World Banking. They followed this op-ed with a declaration titled Measuring the Impact of Microfinance: Our Perspective. The piece and the follow-up four page note square off against the now infamous research results.

I’ll start with one of the finer bits, and perhaps the nexus, of the op-ed: “these studies are giving the inaccurate impression that increasing access to basic financial services has no real benefit.” Jump to their formal response, which contradicts the entire op-ed by admitting “the media’s interpretations of several recent studies on the impact of microfinance…. have questioned whether microfinance has made a quantitative improvement in the lives of borrowers, or has had any effect on poverty alleviation on a systemic basis.”

It baffles me that, although their official take alludes to the irresponsibility of the media’s thirst for sound bites, the consortium’s introductory op-ed cloaks its argument in a series of them. In its plea, we find no evidence that they have read the studies in question, if even their executive summaries. Drs. Abhijit Banerjee, Esther Duflo, and Dean Karlan have already responded to the folly of such mis-interpretation months prior, and I won’t repeat what’s been eloquently addressed. In addition, I would invite anyone to refer to the study-specific policy briefs via J-PAL and IPA’s respective websites…or better yet, the actual working papers.

Rather, the practitioners in question seek to showcase their “evidence” in a series of anecdotes revolving around previously down-trodden women whose lives have miraculously transformed after taking an initial loan.

It would be an understatement to say that I’m all for the empowerment of women. If it weren’t for my belief that microfinance can empower individuals (and not only women, let’s not leave out the men!), I would never have quit my previous job to come seeking the evidence. Although I haven’t seen “thousands of women” whose lives have taken a turn for the better, I’ve seen enough to write a heartfelt story or two. Take the woman above, one of a community of women in Kutch, Gujarat that previously could not leave their village due to tradition. After joining a government-assisted self-help group, these women have not only ventured outside of their village but to Ahmedabad, Mumbai, Delhi, and more to sell their handicrafts in exhibitions. If that’s not empowerment, I don’t know what is.

That said, there is an ugly side to microfinance that a practitioner would (understandably) never choose to highlight. In a similarly alarmist Wall Street Journal article last year by Ketaki Gokhale, we find the following anecdote: "Baleshwari, 23 years old, and her sister Balamani, 40, started taking microcredit two years ago when their father, the sole breadwinner, died. Between the two of them, they have taken loans from four different microlenders and owe payments totaling 4,430 rupees, about $95, each month. During the monsoons, when their combined monthly income, drawn from selling bamboo baskets and catering food, dips to about $65, they turn to the local pawn broker for short-term loans to cover their microfinance debt. The interest rates she pays to pawn brokers range from 36% to 48%, she says, and she had to put up gold jewelry as collateral. Her microfinance loans have interest rates of 18% and 24%. "Group pressure makes us go to moneylenders" to cover their microfinance loans, says Baleshwari, who goes by only one name, as does her sister. "We get small loans for 15 days to fill the gaps when we can't pay. If you lag behind, the rest of the group members can't get new loans."

When the WSJ or others publish stories such as this, certain microfinance practitioners cry foul, claiming that the individuals cited are not representative of the larger picture, the larger impact of microfinance. What, then, is the difference between anecdotes from practitioners and anecdotes from others? And whose are we to believe? What happens to stories of drop-out clients for whom the “magic” of microfinance never quite materialized? Let’s be honest, I’ve yet to see many practitioners showcase their failures. Research is necessary precisely for the reason that anecdotes are biased. See David Roodman’s latest blog for an illustration of the filtering that occurs before we hear of the success stories that dominate the discourse on microfinance

The truth is that nobody yet understands the exact nuances by which microfinance (or development in general) lead to empowerment. We have a broad understanding that, yes, extra income in the hands of women does increase household spending on nutrition, health, children’s welfare and education. However, we have yet to prove that the act of getting a loan in and of itself changes household decision-dynamics in favor of the wife. Esther Duflo goes into detail here. This is just one of the (many) components of microfinance’s purported benefits that the studies in question are examining.

It’s essential that academics, policymakers, and practitioners both acknowledge and explore the reasons for microfinance failures, lest we keep re-inventing the wheel. Randomized control trials, the methodology by which the Hyderabad and Philippines studies were conducted, allow academics to zero in on what actually works. And yes, although each study is conducted in a highly specific context (which the principal investigators readily acknowledge), replications, which are currently taking place (not only with the same households surveyed in India, but in Morocco, Mexico and Peru), allow a synthesis of results with overarching policy implications. Rather than calling a halt to microfinance the world over, such empirical evidence calls instead for innovation, testing, and tinkering for better solutions. After all, if it hadn’t been for Muhammed Yunus’ inventiveness 30-odd years ago, we might not be having this discussion today.

*Thanks to my colleague, Alex, for the title and edits!