|
Peter Burgess peterbnyc@gmail.com
Mar 3 (7 days ago)
to MicrofinancePr.
Dear Colleagues
As I read this, the question of 'Why we want to do measurements?' comes to mind. Throughout my career, I have made use of measurement and data, perhaps in part because I started out as an engineer, and without measurement you don't build anything. In the corporate world the purpose of measurement (accounting) was to report profit performance to the investors and to help (with cost accounting) to get everyone in the company pulling in the same direction. I would also add that in the corporate world measurement had to cost very little and the use of these data had to be very effective!
So how does this fit into the measurement of poverty? I argue that the measure we should be looking for is one that links progress (reduction in poverty) with the resources deployed (and the activities undertaken) in order to get this progress. My impression is that in the development environment people tend to measure only one part of this ... not both parts. I may be mistaken but I don't think so.
One of the strengths of double entry accounting is the idea that change in state (the balance sheet) is a measure of profit. This same idea can be used to measure progress. How much poverty is there at the beginning of the period, and how much at the end of the period. What is the change? How much resources and what activities went into achieving this progress ... or lack of?
I believe a system designed around these concepts would be much more useful than what we have at the present time. Better still, if a system based on these concepts was to be linked to mobile apps for data input and analysis, all sorts of amazing things could become possible.
Peter Burgess - TrueValueMetrics
Multi Dimension Impact Accounting
steve wright steve@conches.org via yahoogroups.com
Mar 4 (5 days ago)
to MicrofinancePr.
Peter,
I have two comments on your excellent insights. The first is in response to your comment: 'Better still, if a system based on these concepts was to be linked to mobile apps for data input and analysis, all sorts of amazing things could become possible. '
At Grameen we have built TaroWorks (http://www.TaroWorks.org) explicitly in response to the dynamics that you describe. This is a tool with an Android 'front end' for data collection, and not just data collection. The mobile app also provides tools for performance and task management of a field force of employees. The cloud based 'back end' provides management and data analysis tools. We have integrated the PPI in to the tool and have made it possible to integrate other standard or custom indicators/surveys. A next step will be to make data aggregation of these standards easy. Again, we will start with PPI data but will ultimately be able to aggregate data from any predictable set of standard indicators.
My second comment is in response to: 'The measure we should be looking for is one that links progress (reduction in poverty) with the resources deployed (and the activities undertaken) in order to get this progress.'
A colleague of mine recently pointed me to an excellent paper that is about integrating Measurement & Evaluation (M&E) in to the 'design space'. The paper starts with an excellent look at the history of M&E from which comes this quote. 'In practice however, ‘project evaluation’ has been used to mean three completely different things, for which we propose three distinct terms: 1) project valuation, 2) implementation evaluation and 3) impact evaluation' The first is focused on the cost of inputs and the benefit (in terms of cost) of the outputs. It is a set of aspirational and speculative questions that one asks to determine if it makes sense to allocate funds. The second, answers very practical questions. Did the intended activities occur? Were the expected outputs produced? The third is concerned with causality - did the activities which generated the outputs cause outcomes?
Embedded in your comment are all three questions. The coming innovations will enable us to efficiently and effectively manage our work explicitly at each stage from ideation to pilot to implementation to scale. From the paper mentioned above: 'We introduce structured experiential learning (which we add to M&E to get MeE) which allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real-time performance information with direct feedback into the decision loops of project design and implementation.' This of course necessitates effective and transparent process and data management of the type that we work to enable with our TaroWorks product.
The social enterprise's progenitor - the financial enterprise - only had to manage a P&L to understand success. Our analog for profit is multifaceted and, while it includes financial sustainability, it also includes vibrant human communities which we must learn to realize with the same rigor and intention with which we realize profit.
Steve Wright
VP Poverty Tools and Insights
Grameen Foundation
There is an inherent tension between implementing organizations—which have specific objectives and narrow missions and mandates—and executive organizations—which provide resources to multiple implementing organizations. Ministries of finance/planning/budgeting allocate across ministries and projects/programs within ministries, development organizations allocate across sectors (and countries), foundations or philanthropies allocate across programs/grantees. Implementing organizations typically try to do the best they can with the funds they have and attract more resources, while executive organizations have to decide what and who to fund. Monitoring and Evaluation (M&E) has always been an element of the accountability of implementing organizations to their funders.
There has been a recent trend towards much greater rigor in evaluations to isolate causal impacts of projects and programs and more ‘evidence-based’ approaches to accountability and budget allocations. Here we extend the basic idea of rigorous impact evaluation—the use of a valid counterfactual to make judgments about causality—to emphasize that the techniques of impact evaluation can be directly useful to implementing organizations (as opposed to impact evaluation being seen by implementing organizations as only an external threat to their funding). We introduce structured experiential learning (which we add to M&E to get MeE) which allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real-time performance information with direct feedback into the decision loops of project design and implementation.
Our argument is that within-project variations in design can serve as their own counterfactual and this dramatically reduces the incremental cost of evaluation and increases the direct usefulness of evaluation to implementing agencies. The right combination of M, e, and E provides the right space for innovation and organizational capability building while at the same time providing accountability and an evidence base for funding agencies.
On Mon, Mar 3, 2014 at 9:01 AM, Peter Burgess wrote:
--
Steve Wright
Twitter: Conches
FB: facebook.com/conches
Blog: http://www.conches.org
__._,_.___
| The text being discussed is available at
| |
