What Client Poverty Monitoring Over Time Is and Is Not Good For

I was recently out for a meal with several young employees of a profit oriented Indian microfinance institution (MFI). Like good young microfinance professionals, we got to speaking about the best ways to efficiently measure the impact of the small loans distributed by their organization. Though it was the most methodologically sound method, we all agreed that it was unrealistic for every MFI to be expected to run randomized evaluations (Randomized evaluations allow researchers to compare the effects of products on a set of randomly selected households which was offered the MFI’s products to a randomly selected set of households not offered the product from the same population). This is unrealistic because randomized evaluations call for an immense investment in time and money that would almost certainly not fit in the impact monitoring budget of even the largest MFIs. So what to do?

Last week, a former colleague kindly sent along the note “Scoring Change; Prizma’s Approach to Assessing Poverty,” published by the Microfinance Centre’s (MFC), a membership-based microfinance resource centre for MFIs in Eastern and Central Europe. The document describes the “scorecard” strategy developed by MFC, as a part of the “IMP-ACT” program, which was designed to monitor the economic profile of MFI clients, as well as observe the changes in their poverty status over time. The document also details how Prizma, an Eastern European MFI, implemented the approach. To oversimplify, the “scorecard” approach requires MFI’s to collect information for 7 consumption/education/wealth indicators that are used to create a poverty score. By using indicators that are also found in national living standards surveys, the correlation between these indicators and poverty can be ascertained. Thus, the poverty score can be used to make a robust guess at the economic status of that household. I have no ability to vouch for the methodology behind the approach, but nothing I read in the document led me to believe it is anything but credible.

The “scorecard” strategy seems to me to have several important uses, but not to be a strong tool for evaluating impact (and although developed under the “IMP-ACT” program, I am not sure that its creators themselves would claim it to be). Probably the most valuable information the “scorecard” can provide is whether an MFI is “reaching who it seeks (and claims) to be reaching…” I am personally not altogether convinced that serving poorer clients equals social performance (perhaps wealthier clients better use credit to develop the local economy), but if an MFI does believe this and asserts to their investors/donors that they serve a certain section of society, they should be able to prove it. MFC mention that other potential uses for a “scorecard” system are to “track dropout by poverty status enabling the Institution to better understand the appropriateness of its service to different [economic classes]” and to “better understand its cost structure…. and locus of cost associated with outreach to poorer clients.”

This type of information has proved to be valuable to MFIs before. See this link for how time series data of their customers reoriented the strategy of Indian MFI Basix.

As far as impact, it is rather cryptically suggested in the note that, “the System enables analysis of more or less discrete change in poverty status over time. While this does not assume attribution, measuring change in household poverty status over time does provide important signal on which to make inferences about outcomes of medium to long-term service provision and highlight areas for further investigation.” I believe what they are trying to say is if you find that your clients, or a segment of your clients, are doing much better or worse than their peers, you should look into this by doing a more thorough study.

Returning to the larger question of this post about how MFIs might understand the impact of their products in a way that has validity but is cheaper than a randomized evaluation, I think my colleague Lakshmi Krishnan’s suggestion that “there is no shortcut to impact” is almost surely right. This of course does not mean that tracking the status of clients over time is not useful, just that we should be clear on what it is useful for.