Dataflow Architecture is Important

The available technology to assist with dataflow has improved immensely, but care still needs to be used in order that costs do not get out of control.

Data acquisition ... design to collect data locally

The best way to collect data is to collect using a locally based system. This has many advantages including the short linkage between the reality, the data that are collected, and local decisions that may be made using the data. There can be a well motivated team engaged with this work, and local costs may be considerably less than would be possible using a team comprised of external experts.

The use of satellite imagery and remote sensing to replace local data collection is counterproductive because much of the value of the data is derived from the use of the data in the local setting … but these technologies may be used effectively to supplement locally acquired data.

Academic research is not the right model

There is a place for academic research … and a role for rigorous scientific and statistical method … but most decision making should be bast on fast low cost dataflows that are right enough to get the decisions right practically all the time. This is not what academic researchers are able to do … and in the main, this is not what their they are working to do!

Use data many times The most cost effective data are data that are used in many different ways. There should ideally be one pool of data, and this one pool should be used in different ways for the specific analysis needed. Essentially analysis provides many different views of the data.

Use locally … simple analysis, practical use Local data may need some simple analysis to be useful for local decision making … but this should be quick and easy. If there is progress … good … if there is little or none then what was wrong with the analysis and what should be tried now.

But in this local analysis and local decision making there is a “risk” evaluation that may not be fully understood or articulated. Poor people do not have the resources to afford a mistake … they cannot “write it off” and move on the way a rich corporate group might do. The children do not go to school, or worse, they die.

In the context of TVM, local data are first used to help with local operational decisions. These are decisions that have a big impact on the performance of a community and frequently are the lacking in data that are relevant and timely.

The most important use of data is the use of data to manage local operations and activities. This is where performance improvement has the most impact and where good data may have achieve the most. With good local use of data, the cost of collecting data and the value of using data are within the same economic domain.

The following graphic is a simple representation of how data may be used to serve several different purposes effectively.

Local data collection ... local analysis ... local action is the cycle that improves performance most directly and most quickly.

Having the data also used at a “higher” level facilitates oversight and the sort of monitoring that can be used to identify the need for corrective action by the analysis of much larger sets of data. At a higher level there can be analysis that identifies “best practice” and issues that are impossible to identify with local analysis alone.

Local people collecting local information is a good way to achieve cost effective data collection. There is a need for adequate training and supervision, but that is true of any approach to data collection. The two advantages of local staff are: (1) modest remuneration requirements; and, (2) familiarity with the place and people.

Survey inaccuracy … amazingly wrong!

Some recent work supervised by Dr. Jonathan Morduch of NYU showed that interview data was hopelessly inaccurate from a first visit survey ... and only reached reasonable correctness after several weeks and multiple visits.

No one data collection approach is likely to be universally optimum. So much depends on the training and experience of the people in the community, and the practical issues of access to information technology and communications infrastructure. A hybrid system involving both manual forms and electronic systems will usually be the way forward. The cost effectiveness of writing in ink in a book should not be totally discounted!

Use same data for oversight and accountability

The same data that are useful to help make decisions at the local community level are also the data that may be used to do oversight. The data architecture allows for roll-up and making summary reports … and with summary reports it is possible to do oversight easily and accurately. Where needed the same data may be used to facilitate accountability. The data architecture used for TVM enables oversight and accountability without contributing to more and more data overload.

Then use data for academic study Some academic study needs a large amount of data, and the TVM data architecture makes it possible for a very large database to be built that allows for very large data mining projects to be designed and set in motion.

Example from the malaria health sub-sector

Detailed spatial information is needed to control malaria in a community … and these data in a consolidated form are suited to oversight and accountability at a higher level. The same data are also ideal for the large scale data mining needed for the early detection of pesticide and drug resistance.