Here are sections I cut from The Coming Prosperity that make reference to the aid debate. I cut them because I decided that, if my conclusion ends up being that the aid debate really doesn’t matter, why spend time talking about it? That’s still my view, but the aid debate doesn’t seem to be going away. Ref this, this, this … oh, and this (in sequence), all from the past few weeks.
The sections that follow are from a June 2010 draft of the book. I have highlighted a few sentences that I think are particularly pertinent to the most recent back-and-forth (including a Twitter exchange I had yesterday evening with @jeffdsachs regarding the annual letter from @billgates).
While the real action in the global economy is taking place in the “developing world” (which is to say, the places where most people live), business authors with a technology focus have until very recently tended to concentrate almost exclusively on trends in the U.S. and other highly developed countries. The leading lights on the topic of development in turn tend to be long on the description of threats and challenges and short on solutions. Jeffrey Sachs’s The End of Poverty is a case in point. The “economic possibilities of our time,” as Sachs sees them, are a failed formula from the fifties: rich countries and international organizations giving money to poor countries to deal with the problems created by the crush of their growing populations. Opportunities suggested for individual action are almost nonexistent. Sachs’s book achieved substantial success by riding a first wave of interest in and engagement with the importance of addressing global challenges. But, as Bill Easterly has so incisively argued, the lens that Sachs trains on the future is marred by failures of the past. Unfortunately for the quality of public discussion on the vitally important topic of global possibilities, when it comes to pathways for progress, Easterly is little better than Sachs. We learn from Easterly what doesn’t work in global development, but not what does.
* * * *
If there is a next big thing at the moment in the field of economics, it is the application of techniques from medical research—specifically, “randomized controlled trials,” or RCTs—to assess the effectiveness of development or other government-initiated projects. The general thrust of the work is as simple as it is brilliant: rather than employ complex, and often unreliable, statistical methods to tease out the extent to which a project or policy actually had a beneficial impact on intended beneficiaries, why not follow the tried and true methods employed by pharmaceutical researchers to assess the efficacy of medical treatments? The steps are: 1) randomly divide the experimental population into a treatment group that receives “benefits” from the program, and a control group that doesn’t; 2) assess outcomes for both groups; and 3) determine whether or not a significant difference exists between the two groups.
This approach, championed by MIT economist Esther Duflo (winner of the prestigious Bates Clark Medal, granted to the most promising economist under the age of forty), is a fabulous improvement on the status quo in development aid, which not infrequently involves assessing program effectiveness by simply checking whether or not the money was spent. Clearly, a world in which aid money is allocated to projects that actually benefit people is preferable to one in which money goes to whoever most effectively maneuvers to get the money to start with and then most reliably manages to spend it. In this way, the use of RCTs in development arguably advances the goal of aid effectiveness, famously championed by NYU economist William Easterly in a series of books and a popular blog titled Aid Watch whose slogan is “Just Asking that Aid Benefit the Poor.”
So, to be clear, RCTs quite credibly represent the “gold standard” in program assessment. If other methods are used, it is usually because RCTs are too expensive or otherwise impractical. Describe your favorite entrepreneurial initiative to an economist and the most probable skeptical response you’ll receive is: “Sounds wonderful, but where’s the evidence of effectiveness? In order to find out whether or not it worked, you really need to do a randomized controlled trial.”
So what is the problem with applying RCTs to development? The Achilles heel of RCTs is a little thing known to the statistically inclined as “external validity”—a phrase that translates informally to “Who cares?” (In the next chapter I’ll address the more general shortcomings of the “aid effectiveness” angle on global development priorities.)
The concept of external validity is straightforward. For any assessment, “internal validity” refers to the mechanism of conducting a clinical trial, and the reliability of results on the original setting. A professionally conducted RCT that yields a high level of statistical significance is said to be “internally valid.” However, it is fairly obvious that an intervention rigorously proven to work in one setting may or may not work in another setting. This second criterion—the extent to which results apply outside the original research setting—is known as “external validity.” External validity may be low because the populations in the original and the new research setting are not really comparable—for example, results of a clinical trial conducted on adults may not apply to children. But external validity may also be low because the environment in the new study setting is different in some fundamental way, not accounted for by the researcher, from the original study setting. Econometric studies that seek to draw conclusions about effectiveness from data that span large geographical areas or highly varied populations thus typically have lower levels of internal validity, but higher levels of external validity.
So, once again, the fundamental issue is not the purity of the methodology employed (as exciting as such methodological purity is to the technically inclined) but rather the inherent complexity of the world being studied.
For this precise reason, it turns out that those who most vociferously and naïvely advocate that we apply techniques from public health to economics (a group that does not include Esther Duflo) make a fundamental error. They fail to appreciate the fact that, when it comes to external validity, public health is the exception that proves the rule. Indeed, in aid-led development in general, of the few real historical successes, nearly all are in public health. Outside of public health, few of the large-scale, top-down development programs have in fact succeeded.
Why is this? Multiple conjectures are possible. But one persuasive one is this: when it comes to biophysical function, people are people. For this reason, a carefully developed medical protocol (read “recipes”) proven to be effective for one population is unlikely to work for another population. The smallpox vaccine tested on one population tended to work on other populations; this made it possible to eradicate smallpox. Oral rehydration therapy tested on one group of children tended to work of other groups of children; millions of children have been spared preventable deaths because the technique has been adopted on a global basis. Indeed, medical protocols have such a high level of external validity that, in the United States alone, tens if not hundreds of thousands of lives could be saved every year through a more determined focus on adherence to their particulars.[i]
These huge successes were achieved, and continue to be achievable, though bold action taken by public health officials. They are rightly celebrated and encouraged, but—outside of other public health applications—not easily replicated. Successes in medicine contrast sharply with failures in other domains. Decades of efforts to design and deploy improved cook-stoves—with the linked aims of reducing both deforestation and the illness and death due to indoor air pollution—have so far primarily yielded an accumulation of Western inventions maladapted to needs and realities in various parts of the world, along with locally developed innovations that cannot be expanded to meet the true scale of the challenge.[ii] For development programs in general, and RCTs in particular, public health is the exception that proves the rule.
What does work in areas outside of public health? How is it possible to design, test, and implement effective solutions in environments where complexity and volatility are dominant?
The general principle applies: Success requires adaptability as well as structure, flexibility as well as structure—a societal capacity to scale successful efforts combined with an engrained practice of entrepreneurial exploration. As the uniquely insightful Mancur Olson wrote in his classic Power and Prosperity:
Because uncertainties are so pervasive and unfathomable, the most dynamic and prosperous societies are those that try many, many things. They are societies with countless thousands of entrepreneurs who have relatively good access to credit and venture capital.[iii]
What works in development, according to Olsen, is experimentation. Why? Because we don’t know what works.
* * * *
To relieve the tedium created by repeated uses of the words “validity” and “economics,” let’s get back to food.
Now (bear with me for a moment here!) let’s say you’re back in the historical paradise of planning, namely the Soviet Union. Every day, you have to eat the same glorious institutional food provided in your communal cafeteria … You’re sick of it, but you can’t find a way out.
Then one day, a leader arrives, with a banner that reads “Just asking that the food not suck!” You cheer! You hoist your comrade on your shoulders! At last, you are fighting back against the system. The battle for better cafeteria food is on!!
But what is the opportunity that is missed here? What is the thing you really need, that you’re not going to get from the “Just Asking that Aid Benefit the P…”—I, mean, the “Just Asking that the Food not Suck” campaign?
What you’re not getting, and what you really need, is some new restaurants. Yes, that would be just the thing. Some options. You would like another place to go to eat.
There is a general rule here: What really drives change isn’t protest, but genuine competition driven by entrepreneurial entry. Think about food in airports 25 years ago, if you were alive then. All Sodexo monopoly. Uniformly terrible and expensive. Now, with entry and competition for licenses, the food in the airport is at least as good as what you get outside the airport.
Entry (or threat of entry) doesn’t have to be by entrepreneurs in order to induce beneficial change. In the United States, the most significant new entrant in the aid business in the last decade has been the Department of Defense. Why might it make sense to have soldiers—who, after all, have a comparative advantage in killing people, not aiding in their social advancement—lead development efforts? For the simple reason that the people who have had that job (notably the World Bank, the United Nations, and the U.S. Agency for International Development) have not been getting the job done; as I will discuss in detail later, ample evidence exists that aid directed to national governments does as at least as much harm to development—by distorting incentives and feeding corruption—as it does good.
But the potentially welcome role of the Department of Defense in disrupting old, and failed, practices in global development is just one instance of a more general phenomenon, which in Zen practice goes by the name “beginner’s mind.” As Penelope Chester, a young Canadian blogger and aid worker, wrote in a Spring 2009 response to a post by William Easterly:
I don’t know for sure, but I suspect that when a bunch of crazy French doctors decided to create Doctors without Borders during the Biafra war, everyone around them must have thought they were absolutely off their rockers.
What about Henri Dunant, the idealistic businessman whose disgust with the horrors of Napoleonic wars lead to the creation of the Red Cross? (Did you catch that? Henry Dunant was a businessman with no experience in anything remotely connected to humanitarian aid).
In the words of George Bernard Shaw, “Reasonable people adapt themselves to the world. Unreasonable people attempt to adapt the world to themselves. All progress, therefore, depends on unreasonable people.”
To close the loop on the above discussion of assessment methodology, it is worth noting that development driven by entrepreneurship (also known as “development”) is comprised of randomized out-of-control trials. That would be—yes!—the opposite of randomized controlled trials.
Again, why does this matter? Stop and think. In what U.S. industry do clinical trials dominate? That would be pharmaceuticals. And in what industry are markups higher, and barriers to entry greater, than they are in the pharmaceuticals industry? The answer to that question is, of course, no other industry. When it comes to persistent oligopoly, the pharmaceutical beats them all.
A very big part of the reason for this is that large-scale clinical trials are expensive. But you can’t sell a drug without them. (For mostly good reasons, I might add, in the case of medicines.) So even successful biotech companies have had great difficulty breaking into the business of conducting their own clinical trials; instead they often partner with “Big Pharma” on the last mile of drug development.
Now I’m not saying we should abolish the Food and Drug Administration. I’m just asking this question: Is the increasingly widespread use of RCTs a move in the direction of a Food and Drug Administration for development, if not in a hardwired, institutional sense then instead in the sense of customs, standards, and expectations? Will “higher standards of evidence” not only distort resource allocation (if outcomes are improperly defined) but also create barriers to entry? Won’t this favor incumbents, or outside consultants flown in to do the work? Might not all of these “secondary” effects more than outweigh any benefit gained from “better” standards of evaluation?
Furthermore, might we not do better by studying the work of those exceptional entrepreneurs who do a particularly remarkable job in creating social value (a term I will define in Chapter 10), and putting our resources into supporting the nascent efforts of others like them, using an approach to evaluation that is actually appropriate to entrepreneurship?
Instead of putting our faith in randomized controlled trials whose beneficial impacts are uncertain, shouldn’t we bet on the process of randomized out-of-control trials (a.k.a. entrepreneurship and innovation) that has been the very definition of development and growth pretty much everywhere in the world for five centuries?
A set of three principles applies:
- If solutions are known, people need money. When external validity is high and the potential for future gains is large, the most significant constraint is inadequacy of investments—easily proxied by cash dollars.
- If solutions are knowable, people need assessment. In settings where effective general solutions have not yet been discovered, but are in principle knowable, due to stability within, and similarly among, environments in which the particular problem presents itself.
- If solutions are evolving, people need entrepreneurship. Everywhere else—where complexity and volatility dominate and creative adaptation is requirement—progress will be attainable through entrepreneurship.
* * * *
Can such small-scale efforts [of entrepreneurs] really form the basis for the development of a nation? A better question is, on what else can the development of a nation possibly be based? As I will discuss in detail in chapter 9, in areas other than public health, foreign aid directed to national governments is not the answer. Indeed, those on opposite ends of ongoing debates over the effectiveness of foreign aid are in remarkable agreement when it comes to what actually drives the process of development: technology, entrepreneurship, and innovation. Columbia University’s Jeffrey Sachs states that “the single most important reason why prosperity spread, and why it continues to spread, is the transmission of technologies and the ideas underlying them.” NYU’s Bill Easterly argues that “historically, industrialization arose in initially poor countries which have since become rich, with the common theme a heavy reliance on both domestic and international market opportunities and decentralized private entrepreneurship.” The basic theme is the same: entrepreneurship and innovation, driven by new technologies, lead the creation of viable institutions—not the other way around.
As I suggested at the outset, the reality that entrepreneurs—whether farmers or founders of mobile phone companies—can thrive in environments rife with violence and corruption should come as no surprise. Citizens at large, and the entrepreneurs among them, tend not to wait for the optimal condition of a stable supportive state in order to function. On the contrary, by beginning where they are, they are likely to usher in the kind of changes in the state that they aspire to create. As Sir James Steuart wrote in 1767, “A modern economy . . . is the most effective bridle ever invented against the folly of despotism.” Granted, such a statement may seem almost bizarre, unless “modern economy” excludes not only the Soviet Union but also China’s massive, but rapidly receding, state-owned sector. Twentieth-century socialist models equating modernity with gigantism were as unimaginable in 1767 as they are antithetical to development progress today. Iqbal Quadir puts it thus: “The history of Western economic and political advancement illustrates that it is the economic strength of citizens—not governments—that gives rise to checks and balances.” The dispersion of economic power precedes dispersion of political power. As accountability grows, the prerequisites of prosperity strengthen.
[i] See Atul Gawande (2009), Checklist Manifesto: How to Get Things Right. New York: Metropolitan Books.
[ii] Burkhard Bilger (2009), Hearth Surgery: The Quest for a Stove That Can Save the World. The New Yorker. December 21.
[iii] Olson, op. cit., pp. 188-189.