Academia
Academia in Crisis

Why Academics Mustn't Control Research

Burgess COMMENTARY

Peter Burgess

Why Academics Mustn't Control Research Two problems plague academic research: nimbleness and quality. The upshot is that research is not all that relevant to the decisions practitioners need to make on an ongoing basis. I found myself considering the limits of academia at a conference at MIT. Alex Counts, CEO of Grameen Foundation challenged a panelist about calling RCTs - randomized contronl trials - the gold standard of research. Alex suggested I take a look at his blog post of last fall, which my inspired my response below. Academics made, I believe, a big contribution to microfinance by pointing out that the outsized effects so splashily advertised by suppliers were paler than imagined, and that in some instances, interventions caused harm. Microfinance needed to be taken down a peg and made real if it were to improve. The researchers helped to do that. However, the time has come for practitioners to take on the task of building internal research capability. The university with which I am associated is besieged by research requests of all kinds - marketing, operational, impact - and I often wonder why. There are two problems with delegating research to universities. First, research is crucial to the development of institutional knowledge. MFIs and other financial inclusion organizations (including those that promote savings groups) must continually gather intelligence in order to make informed tweaks to their value proposition. Research should be accompanied by constant adjustment, with an institution’s full grasp of the idea that what is perfect today will be imperfect tomorrow. No outsider, especially a large university complex, can do this nimbly or efficiently. Second, the quality of university research is in question. I have supervised many students who have interned at various esteemed university research organizations - the big brands. What actually goes on and counts as data is impressive, if not alarming. It only gets worse in the analytical phases, where data is plunged through various sieves until the desired findings are shaken out and held to the light. Studies are anything but rigorous. In fact, they are biased in favor of the outcome an external researcher wishes to find, usually influenced by a motivation to publish, to raise money, or, to build a brand. Academic research is anything but objective. Researchers often seek to discover: experimental proof of their own hypotheses (“aren’t I smart?”); disproof of someone else’s intervention (“X, Miracle or Myth?” - though X never claimed to be a miracle). Or, more recently, benign inconclusivity (“I need to say slightly positive things about X to keep my research funded”). To repeat, these biases reflect a researcher’s need to get published, to get funded, or, to build a brand. This should come as no surprise as the medical field has understood the problem of research quality for years (see works by Dr. John Ioannidis – e.g. An Epidemic of False Claims in Scientific American or this in Nature). Medicine is considered a softer science than physics, and economics far softer than medicine. So if medical academics have begun to question the work of peers, what does this say for the very soft science of economics? And the equally soft science of anthropology? It says that the studies are not to be trusted unless we know the individual researcher, have seen him or her in action in the field, and can ascertain his or her competence. There is nothing generically solid about economic or anthropological study. It’s an individual matter. If the purpose of research is to learn and make decisions, then financial inclusion entities must bring economic and anthropological talent (with full understanding of the limits of these disciplines) into the heart of their organizations, and to bolster such talent with indwelling organizational knowledge. Key of course is leadership committed to using research to drive internal change. Practitioner research would limit the hype that surrounds many academic studies, which make claims rarely verified by additional studies (in the same place) or by follow-on studies (of the same population and intervention). Think of all those “interesting findings” that were never re-proven over time. The essence of science is to prove and reprove hypotheses; verification is part of the scientific endeavor. You won’t find that in academia in the area of financial inclusion. To publish, academics must produce something flashy. What would universities then do were they to cease with their natural and unnatural experimenting? Universities might train practitioners about practical ways to collect and analyze data on important behavioral and cultural concepts. And it would be beneficial to all if universities continued to study the socio political-environment that surrounds and shapes financial inclusion. The trick will be for any research effort, whether practitioner-based, university-based, or third party-based, to include the dissemination of real information even when the findings disappoint. Article originally appeared on Savings Revolution (http://savings-revolution.org/).