A new approach to development aid
For many decades, humanitarian and development aid has been viewed in a very superficial way. Until ten years ago, when a young French researcher named Esther Duflo started a thorough investigation of the problem. She was idealist, tenacious, but also kind of discouraged at first. Because after 40 years of intensive development aid, and more than 3 billion dollars poured into the system, there was still no clear mechanism deploying the right amounts of money to the right places, as the right time, and for the right reasons. And neither did anyone have any idea if aid had any tangible long-term effect whatsoever.
In a historic report in 2010, Duflo drew two curves. One showed the amount of aid given to Africa for the last few decades. Undoubtedly, it was steadily climbing. The other one showed the GDP per capita of the continent. And today it remains similar to its 70s levels. How is that possible?
Duflo argues that without all that foreign aid, Africa could have been doing better at this point... or worse. Or the same. The point is, no one really knows if aid has had any profound effect. Her proposition is that these effects should be examined the way modern medicine examines disease treatment. With the methods used by pharmaceutical companies to establish the efficacy of a new medicine. The specialist jargon calls this "randomised impact evaluation". It means that you gather as many people as you can for your sample, and split them into two groups. One group gets the medicine, the other doesn't. Then you analyse if the active ingredient had worked. The randomness principle helps you filter the "statistical noise", and prevents bias. It establishes the connection between cause and effect.
Duflo was joined by economists from some of the best universities in the world, and they started analysing which measures within the development aid framework would bring real benefits. They created the Abdul Latif Jameel Poverty Action Lab (J-PAL). The results of their research come from the so called field research: going into poor neighbourhoods in towns and villages in the Third World. In Kenya for example, 75 schools with more than 30,000 pupils were split into three groups, to find out what stimulates the highest school attendance rates. The result was surprising: it wasn't free food or (as the World Bank prescribes) direct financial aid to the parents as a stimulus to send their children to school. It was anti-parasite medication (aka "deworming"). It turned out that most children who don't go to school have a problem with diseases caused by various parasites.
Not a single research had reached this conclusion up to that point. When families are interviewed about the reasons for school absence, they give all sorts of reasons, but they never say a word about parasites because they do not consider this an issue worth mentioning. Only the large-scale experiment allowed the researchers to find this subtle cause-effect relation. Eventually, the treatment of the parasite infections has led to a 25% increase in school attendance.
And it's not just students who miss school classes. In India, half of the classes are canceled because the teachers have to work two jobs and more to make ends meet; or because some are lazy; or because there is no one to punish them for not going to work, or stimulate them to go to work. Out of the many aid programs, the most successful one was a strange initiative that gave out photo cameras to the pupils. The teachers had to take photographs of their classes twice a day, and only those teachers who could present two pictures a day were given their salaries. Eventually the teacher absence rates dropped drastically, and the success rate of the pupils improved. At the cost of a few photo cameras.
Initially, all other measures that went through preliminary testing in India, also seemed reliable: better payment for the teachers, strict control from both the school authorities and the local community. If an aid organisation wanted to implement this method, it would be granted financing. But that was a terrible waste of resources, compared to the alternative option. Because very few organisations were testing their programs so precisely like the "randomness advocates" around Esther Duflo.
Duflo claims that her ultimate goal is to develop a culture of detailed experimentation that would lead to revolutionary changes in the aid policies of the 21st century. Some are seeing the birth of a new type of economy that would stop the current inertia in the aid system, and establish the "gold standard" of the J-PAL experts. Which is essentially an assessment of the possibilities through randomised research. The supporters of this method are earning more and more sympathies, while the classical aid schemes are losing popularity. Because those are perceived by many people as a set of alternating untested fashions: first there were the dams, then stimulating women, then "structural harmonisation", ecology and microcredits, and then the dams again.
So, it turns out no lessons have been drawn for the last 50 years? Or was the error in the way the researches were being carried out? Duflo is hoping that it was merely a lack of will and perseverence. A will to understand what functions well and what doesn't. And lack of ambition of those who are making a living from this sort of research. After all, who would want to over-analyse their own work?
The alternative sounds much more sinister, but more realistic: governments deliberately looking the other way, while billions of aid were floating into one direction, a large portion of it being diverted into the coffers of various dictators, warlords and "businessmen".
Another research has found that a surprisingly small amount of researches on the effectiveness of these traditional methods have reached negative conclusions. It's as if there is a tacit agreement within these circles that it is better to keep on with the ineffective programs, rather than having no programs at all. But on the other hand, we are talking of billions of dollars that are going to waste, where they could have made a real difference. And what's worse, this affects real humans with real lives and real destinies, in a very real way.
About a decade ago, when Esther Duflo decided to apply precise standards in the development aid sector, most economists were advising her to quit it. Doing scientific experiments outside the controlled laboratory conditions? Impossible! But today, major donors like Bill Gates are openly trusting her methods and seeking her advice. And she doesn't spare them the merciless, but also constructive analysis. Today, many academic experts are heeding the J-PAL analyses, and Duflo has been receiving the "genius grant" of the MacArthur Foundation since 2009, which is generally considered a tip for a Nobel prize.
Of course Duflo has earned far more detractors than awards. Because her research methodology inevitably leads to dismantling established models that are decades old, without necessarily proposing substitutes in their place. Her most recent "victim" were the micro-loans. 30 years ago they were popularised by Mumammad Junus, aiming to aid the poorest of the poor. He got a Nobel peace prize for his efforts. The expectation was that the small loans would not only stimulate entrepreneurship, but would also empower women, and improve children's health and their success at school.
But the J-PAL researchers tested this policy very thoroughly, and they reached some controversial results. Some are interpreting them as proof for the utter uselessness of microfinance. Because Junus' great expectations were proven unfounded.
Duflo herself interprets the results differently. She admits that the micro-credits do exactly what they are supposed to do: they provide crediting for the poorest. Which is a big achievement by itself. With the money, people can buy TVs, pay their debts and invest in some small business. But that is all. The credits achieve nothing more. Comparing these people with others who haven't been granted micro-credits, turns out they are not necessarily living in better housing conditions, their health is not better, they are not better educated and they do not have more equal rights.
This does not mean that micro-crediting is completely useless, and Duflo and her supporters do acknowledge this. But beyond that, they are against the far-fetched promises and expectations that are often to be heard among the supporters of this practice. Of course the latter (whose annual turnover has reached 60 billion dollars) are contesting the conclusions of the research. Duflo responds with her typical sober assessment: "We tried to help them. They don't want to be helped. Too bad".
At present, her team has taken on another "sacred cow" of the aid industry, the so called "community participation". The idea of this policy is that the bigger part of a community participates in a given project, the more successful it would be. Besides, the common notion is that this would empower the people and transfer participation in politics from the elites to the common folk. 9% of the World Bank investments go into such projects - billions of dollars are being transferred every year, based on an assumption.
Recently, a J-PAL expert analysed such a program in Sierra Leone, and she reached the conclusion that, while the development projects have indeed changed lots of things in these communities for the better, the way the decision-making is working hasn't changed one bit. The power of the elites remains intact, despite the participation of the communities. The conclusion is that this has been yet another generous promise that was only partially realised.
Granted, it must be a tough life being the black ram of your scientific branch, and it must be hard to live in the role of the scarecrow of the development aid sector, being constantly pointed fingers at, and having to parry this attack or the other. But Esther Duflo believes that ultimately, the choice is simple: between decades-old tradition that has been trying to cure a disease without knowing how its methods really work, and looking for a new approach that is randomised research.
What she means is that 50 years of useless debates have always had the same results. It has all boiled down to two opposing positions, now primarily expressed by Jeffrey Sachs and William Easterly. The differences between those two are probably the best illustration of the two principles that are at odds at the moment.
Simply put, Jeffrey Sachs, the world-famous economics professor of the University of Columbia, is pleading for more and more money. He believes the wealthy countries should be giving more, much more. That is the only way the scandalous economic discrepancies could be overcome. Sachs initiated the Millennium Villages Project, which literally drowns certain villages in Africa in money. Every single inhabitant gets additional financing. The total worth of the pilot stage for the whole continent was 120 million dollars. The next stage involves billions, and the aim is to prove the validity of the theory that lots and lots of aid can help a lot of people in a lot of ways.
Once the project was started, Sachs contacted Esther Duflo and asked her to do her precise examinations. She asked him to show her the preliminary data he had collected before the start of the project, how he had selected the villages, what was his info on the current dynamics of the effects, etc. His responses came nowhere near the critical line of the strict requirements of the J-PAL lab. At some point their correspondence died down. Duflo complained that 120 million dollars had just sunken into a program, whose effect no one will never know - if any. And no one knows whether the money wouldn't have made a real impact, had it been invested elsewhere, in a different way.
As for William Easterly, he is the most popular among the skeptics, and he sees the world in a very different way from Sachs. His argument is that we do not know which measuers stimulate development, we don't know which of our advices bring real benefit, and we are not even sure who these "We" are, of whom it is expected to "do something" for development. His conclusion: development assistance was a mistake in the first place.
Now, of course Duflo considers both views too extreme. Her approach is that every single project, every policy should be viewed separately. The "randomist" camp insists for moderation, first and foremost. A moderate, balanced approach is critical for rescuing the development aid system before it crumbles down under its own weight.
There are undoubtedly many useful and sensible projects, the problem is they have to be detected and encouraged. But the bottom line is that they could all add up to a system of aid that is much less wasteful, more efficient, and hits the nail on the head rather than hammering at random. Abhijit Banerjee, one of the J-PAL founders, goes as far as to provoke the public with a claim that at the moment a global aid system worth just a couple billion dollars annually would be more than enough to finance all the development programs that have proven efficiency, and stop wasting those enormous amounts of money. Of course he admits the amount could increase a bit, if the "randomists" manage to detect more functioning projects. But while they are looking for them around the world, with every new research of theirs, the number of their detractors is growing.
Duflo's people remain unperturbed. They have already carried out dozens of extensive researches throughout the Third World, in places like Zambia for example. Despite all the difficulties inherent to doing research in such a place like Lusaka (which is virtually a metropolis consisting of thousands of scattered, chaotic villages), the results have been more than promising. The method does seem to work well - which is not to say it should be universally applied everywhere. Duflo is adamant that each case has to be viewed separately, taking all local specifics in consideration. By the way Lusaka has been so extensively researched in the last few years that this has resulted in the local people starting to demand payment for any participation in a J-PAL interview from now on.
But the bottom line of these researches is clear. The method relies on a complex examination of all the possible links and connections between the elaborately intertwined types of motivations for development. So far financial aid (i.e. money) has been virtually the sole factor that has mattered, but the new method has reached some curious conclusions: proverty alone is not the only answer to all the questions.
Ultimately, this method is much more complex and precise than anything that has ever been done to this point, and it examines human behaviour in a way so profound that even its creators are not fully able to grasp yet. It asks questions that haven't been asked before in such a way: when and how do people decide to start saving, even the poor? Why don't villagers apply cheap technologies that could significantly increase their income? When do people decide to invest in disease prevention? There is still too little information on these fundamental questions.
Like I said, the uniqueness of this method hardly makes it universal for the entire world. Should the research in Zambia be automatically applied to India or Ecuador? Is it possible that these results could be generalised? Well, that is the main point of contention, and presumably the weakest spot of the method. Its critics have their problems with it, starting from the quiestionable applicability of one research outside the framework of its particular local/regional task. Secondly, there are concerns that the long-term effects could be impossible to foresee, making it impossible to build entire large-scale systems along the models dictated by a small-scale research. And thirdly, there have still been no long-term researches that would examine the effect of these interventions for a period longer than three years, and this hides a number of risks.
Duflo acknowledges that J-PAL still doesn't have all the answers, and in turn she adds some more criticism to her own methods. Important questions like the fight against crime, the fiscal policies of a country and others, cannot be aided by this method. But still, with more than 300 researches at this point, the general conclusion is that many of them are applicable universally. If the clear correlation between "deworming" and school attendance has already been established in Africa, there is no reason not to apply it to South America as well, and instead of doing the same research twice, all this effort could be used for doing something more meaningful that would have a real impact.
On the other hand, is it possible that programs with proven efficiency could bring undesirable results if they are applied to a larger scale? Of course that could happen too, and there is only one way to detect that undesirable effect: more research, based on the random principle. At ever increasing scales, encompassing ever more complex systems. Right now the economists are making first strolls in the realisation of such projects in India. The government is commissioning researches spanning thousands of households throughout hundreds of villages. And the results will come, sooner or later.
There is another problem with this type of statistical experiments: the ethical aspect of the issue. Each intervention by this methodology means that a certain measure that could have helped everybody in the community, would only be applied to one controlled group, while others would be deprived of it. This dilemma is familiar from the medical testing experiments. But after a thorough research, it soon becomes clear that the alternative almost never helps everybody, and most often doesn't help anybody at all. And, after all, whoever carries out a development project without preliminary research, cannot have any idea if it would be effective, and what its effects would exactly be (if any). If that is not an experiment, a shot in the dark, I don't know what is.
In fact, the J-PAL researchers should be more concerned about their supporters than their critics. Because there is a real risk that the new method could become the very next fashion in aid politics - after the dams, randomised researches could be the new kid on the block, only to give way to some other such arbitrary policy. Even though the method clearly does not provide answers to all questions.
Still, the World Bank is already taking the initiative and making researches of its own. The Bill Gates foundation often commissions random tests before supporting a particular initiative. The Spanish government has created a fund that sponsors only randomised researches. Would the involvement of the government structures lead to "diluting" the method? Is there a threat that the good results could come at the expense of not so precise testing, and ultimately, lead to the old routine ways of aid?
The problem with over-exploiting the method could wait for now. For the time being, it is important to expand the researches based on the random principle, and possibly encompass the entire world, including the developed world. The J-PAL lab has created a branch in Paris dealing with government programs for social aid. And no surprise, the very first tests have discredited the French government as incompetent and inefficient in this respect.
When former French president Nicolas Sarkozy was promoting his idea to put cops in schools to deal with low school attendance, the J-PAL research found a far more effective method: informing the parents and inviting them to school, making meetings between teachers and parents, and all in all, stimulating the parents to cooperate for solving the problem. Entirely voluntarily. The involvement of cops was proven utterly unnecessary, even harmful. Today, the school attendance rates where the method was applied, is similar to that of the elite high schools. Now speaking from personal experience, myself being the deputy principal of a working-class suburban elementary school in one of Africa's metropolises, I can say with certainty that this is one of those many occasions when the results of this method, after being tested locally, could be applied universally.
In a historic report in 2010, Duflo drew two curves. One showed the amount of aid given to Africa for the last few decades. Undoubtedly, it was steadily climbing. The other one showed the GDP per capita of the continent. And today it remains similar to its 70s levels. How is that possible?
Duflo argues that without all that foreign aid, Africa could have been doing better at this point... or worse. Or the same. The point is, no one really knows if aid has had any profound effect. Her proposition is that these effects should be examined the way modern medicine examines disease treatment. With the methods used by pharmaceutical companies to establish the efficacy of a new medicine. The specialist jargon calls this "randomised impact evaluation". It means that you gather as many people as you can for your sample, and split them into two groups. One group gets the medicine, the other doesn't. Then you analyse if the active ingredient had worked. The randomness principle helps you filter the "statistical noise", and prevents bias. It establishes the connection between cause and effect.
Duflo was joined by economists from some of the best universities in the world, and they started analysing which measures within the development aid framework would bring real benefits. They created the Abdul Latif Jameel Poverty Action Lab (J-PAL). The results of their research come from the so called field research: going into poor neighbourhoods in towns and villages in the Third World. In Kenya for example, 75 schools with more than 30,000 pupils were split into three groups, to find out what stimulates the highest school attendance rates. The result was surprising: it wasn't free food or (as the World Bank prescribes) direct financial aid to the parents as a stimulus to send their children to school. It was anti-parasite medication (aka "deworming"). It turned out that most children who don't go to school have a problem with diseases caused by various parasites.
Not a single research had reached this conclusion up to that point. When families are interviewed about the reasons for school absence, they give all sorts of reasons, but they never say a word about parasites because they do not consider this an issue worth mentioning. Only the large-scale experiment allowed the researchers to find this subtle cause-effect relation. Eventually, the treatment of the parasite infections has led to a 25% increase in school attendance.
And it's not just students who miss school classes. In India, half of the classes are canceled because the teachers have to work two jobs and more to make ends meet; or because some are lazy; or because there is no one to punish them for not going to work, or stimulate them to go to work. Out of the many aid programs, the most successful one was a strange initiative that gave out photo cameras to the pupils. The teachers had to take photographs of their classes twice a day, and only those teachers who could present two pictures a day were given their salaries. Eventually the teacher absence rates dropped drastically, and the success rate of the pupils improved. At the cost of a few photo cameras.
Initially, all other measures that went through preliminary testing in India, also seemed reliable: better payment for the teachers, strict control from both the school authorities and the local community. If an aid organisation wanted to implement this method, it would be granted financing. But that was a terrible waste of resources, compared to the alternative option. Because very few organisations were testing their programs so precisely like the "randomness advocates" around Esther Duflo.
Duflo claims that her ultimate goal is to develop a culture of detailed experimentation that would lead to revolutionary changes in the aid policies of the 21st century. Some are seeing the birth of a new type of economy that would stop the current inertia in the aid system, and establish the "gold standard" of the J-PAL experts. Which is essentially an assessment of the possibilities through randomised research. The supporters of this method are earning more and more sympathies, while the classical aid schemes are losing popularity. Because those are perceived by many people as a set of alternating untested fashions: first there were the dams, then stimulating women, then "structural harmonisation", ecology and microcredits, and then the dams again.
So, it turns out no lessons have been drawn for the last 50 years? Or was the error in the way the researches were being carried out? Duflo is hoping that it was merely a lack of will and perseverence. A will to understand what functions well and what doesn't. And lack of ambition of those who are making a living from this sort of research. After all, who would want to over-analyse their own work?
The alternative sounds much more sinister, but more realistic: governments deliberately looking the other way, while billions of aid were floating into one direction, a large portion of it being diverted into the coffers of various dictators, warlords and "businessmen".
Another research has found that a surprisingly small amount of researches on the effectiveness of these traditional methods have reached negative conclusions. It's as if there is a tacit agreement within these circles that it is better to keep on with the ineffective programs, rather than having no programs at all. But on the other hand, we are talking of billions of dollars that are going to waste, where they could have made a real difference. And what's worse, this affects real humans with real lives and real destinies, in a very real way.
About a decade ago, when Esther Duflo decided to apply precise standards in the development aid sector, most economists were advising her to quit it. Doing scientific experiments outside the controlled laboratory conditions? Impossible! But today, major donors like Bill Gates are openly trusting her methods and seeking her advice. And she doesn't spare them the merciless, but also constructive analysis. Today, many academic experts are heeding the J-PAL analyses, and Duflo has been receiving the "genius grant" of the MacArthur Foundation since 2009, which is generally considered a tip for a Nobel prize.
Of course Duflo has earned far more detractors than awards. Because her research methodology inevitably leads to dismantling established models that are decades old, without necessarily proposing substitutes in their place. Her most recent "victim" were the micro-loans. 30 years ago they were popularised by Mumammad Junus, aiming to aid the poorest of the poor. He got a Nobel peace prize for his efforts. The expectation was that the small loans would not only stimulate entrepreneurship, but would also empower women, and improve children's health and their success at school.
But the J-PAL researchers tested this policy very thoroughly, and they reached some controversial results. Some are interpreting them as proof for the utter uselessness of microfinance. Because Junus' great expectations were proven unfounded.
Duflo herself interprets the results differently. She admits that the micro-credits do exactly what they are supposed to do: they provide crediting for the poorest. Which is a big achievement by itself. With the money, people can buy TVs, pay their debts and invest in some small business. But that is all. The credits achieve nothing more. Comparing these people with others who haven't been granted micro-credits, turns out they are not necessarily living in better housing conditions, their health is not better, they are not better educated and they do not have more equal rights.
This does not mean that micro-crediting is completely useless, and Duflo and her supporters do acknowledge this. But beyond that, they are against the far-fetched promises and expectations that are often to be heard among the supporters of this practice. Of course the latter (whose annual turnover has reached 60 billion dollars) are contesting the conclusions of the research. Duflo responds with her typical sober assessment: "We tried to help them. They don't want to be helped. Too bad".
At present, her team has taken on another "sacred cow" of the aid industry, the so called "community participation". The idea of this policy is that the bigger part of a community participates in a given project, the more successful it would be. Besides, the common notion is that this would empower the people and transfer participation in politics from the elites to the common folk. 9% of the World Bank investments go into such projects - billions of dollars are being transferred every year, based on an assumption.
Recently, a J-PAL expert analysed such a program in Sierra Leone, and she reached the conclusion that, while the development projects have indeed changed lots of things in these communities for the better, the way the decision-making is working hasn't changed one bit. The power of the elites remains intact, despite the participation of the communities. The conclusion is that this has been yet another generous promise that was only partially realised.
Granted, it must be a tough life being the black ram of your scientific branch, and it must be hard to live in the role of the scarecrow of the development aid sector, being constantly pointed fingers at, and having to parry this attack or the other. But Esther Duflo believes that ultimately, the choice is simple: between decades-old tradition that has been trying to cure a disease without knowing how its methods really work, and looking for a new approach that is randomised research.
What she means is that 50 years of useless debates have always had the same results. It has all boiled down to two opposing positions, now primarily expressed by Jeffrey Sachs and William Easterly. The differences between those two are probably the best illustration of the two principles that are at odds at the moment.
Simply put, Jeffrey Sachs, the world-famous economics professor of the University of Columbia, is pleading for more and more money. He believes the wealthy countries should be giving more, much more. That is the only way the scandalous economic discrepancies could be overcome. Sachs initiated the Millennium Villages Project, which literally drowns certain villages in Africa in money. Every single inhabitant gets additional financing. The total worth of the pilot stage for the whole continent was 120 million dollars. The next stage involves billions, and the aim is to prove the validity of the theory that lots and lots of aid can help a lot of people in a lot of ways.
Once the project was started, Sachs contacted Esther Duflo and asked her to do her precise examinations. She asked him to show her the preliminary data he had collected before the start of the project, how he had selected the villages, what was his info on the current dynamics of the effects, etc. His responses came nowhere near the critical line of the strict requirements of the J-PAL lab. At some point their correspondence died down. Duflo complained that 120 million dollars had just sunken into a program, whose effect no one will never know - if any. And no one knows whether the money wouldn't have made a real impact, had it been invested elsewhere, in a different way.
As for William Easterly, he is the most popular among the skeptics, and he sees the world in a very different way from Sachs. His argument is that we do not know which measuers stimulate development, we don't know which of our advices bring real benefit, and we are not even sure who these "We" are, of whom it is expected to "do something" for development. His conclusion: development assistance was a mistake in the first place.
Now, of course Duflo considers both views too extreme. Her approach is that every single project, every policy should be viewed separately. The "randomist" camp insists for moderation, first and foremost. A moderate, balanced approach is critical for rescuing the development aid system before it crumbles down under its own weight.
There are undoubtedly many useful and sensible projects, the problem is they have to be detected and encouraged. But the bottom line is that they could all add up to a system of aid that is much less wasteful, more efficient, and hits the nail on the head rather than hammering at random. Abhijit Banerjee, one of the J-PAL founders, goes as far as to provoke the public with a claim that at the moment a global aid system worth just a couple billion dollars annually would be more than enough to finance all the development programs that have proven efficiency, and stop wasting those enormous amounts of money. Of course he admits the amount could increase a bit, if the "randomists" manage to detect more functioning projects. But while they are looking for them around the world, with every new research of theirs, the number of their detractors is growing.
Duflo's people remain unperturbed. They have already carried out dozens of extensive researches throughout the Third World, in places like Zambia for example. Despite all the difficulties inherent to doing research in such a place like Lusaka (which is virtually a metropolis consisting of thousands of scattered, chaotic villages), the results have been more than promising. The method does seem to work well - which is not to say it should be universally applied everywhere. Duflo is adamant that each case has to be viewed separately, taking all local specifics in consideration. By the way Lusaka has been so extensively researched in the last few years that this has resulted in the local people starting to demand payment for any participation in a J-PAL interview from now on.
But the bottom line of these researches is clear. The method relies on a complex examination of all the possible links and connections between the elaborately intertwined types of motivations for development. So far financial aid (i.e. money) has been virtually the sole factor that has mattered, but the new method has reached some curious conclusions: proverty alone is not the only answer to all the questions.
Ultimately, this method is much more complex and precise than anything that has ever been done to this point, and it examines human behaviour in a way so profound that even its creators are not fully able to grasp yet. It asks questions that haven't been asked before in such a way: when and how do people decide to start saving, even the poor? Why don't villagers apply cheap technologies that could significantly increase their income? When do people decide to invest in disease prevention? There is still too little information on these fundamental questions.
Like I said, the uniqueness of this method hardly makes it universal for the entire world. Should the research in Zambia be automatically applied to India or Ecuador? Is it possible that these results could be generalised? Well, that is the main point of contention, and presumably the weakest spot of the method. Its critics have their problems with it, starting from the quiestionable applicability of one research outside the framework of its particular local/regional task. Secondly, there are concerns that the long-term effects could be impossible to foresee, making it impossible to build entire large-scale systems along the models dictated by a small-scale research. And thirdly, there have still been no long-term researches that would examine the effect of these interventions for a period longer than three years, and this hides a number of risks.
Duflo acknowledges that J-PAL still doesn't have all the answers, and in turn she adds some more criticism to her own methods. Important questions like the fight against crime, the fiscal policies of a country and others, cannot be aided by this method. But still, with more than 300 researches at this point, the general conclusion is that many of them are applicable universally. If the clear correlation between "deworming" and school attendance has already been established in Africa, there is no reason not to apply it to South America as well, and instead of doing the same research twice, all this effort could be used for doing something more meaningful that would have a real impact.
On the other hand, is it possible that programs with proven efficiency could bring undesirable results if they are applied to a larger scale? Of course that could happen too, and there is only one way to detect that undesirable effect: more research, based on the random principle. At ever increasing scales, encompassing ever more complex systems. Right now the economists are making first strolls in the realisation of such projects in India. The government is commissioning researches spanning thousands of households throughout hundreds of villages. And the results will come, sooner or later.
There is another problem with this type of statistical experiments: the ethical aspect of the issue. Each intervention by this methodology means that a certain measure that could have helped everybody in the community, would only be applied to one controlled group, while others would be deprived of it. This dilemma is familiar from the medical testing experiments. But after a thorough research, it soon becomes clear that the alternative almost never helps everybody, and most often doesn't help anybody at all. And, after all, whoever carries out a development project without preliminary research, cannot have any idea if it would be effective, and what its effects would exactly be (if any). If that is not an experiment, a shot in the dark, I don't know what is.
In fact, the J-PAL researchers should be more concerned about their supporters than their critics. Because there is a real risk that the new method could become the very next fashion in aid politics - after the dams, randomised researches could be the new kid on the block, only to give way to some other such arbitrary policy. Even though the method clearly does not provide answers to all questions.
Still, the World Bank is already taking the initiative and making researches of its own. The Bill Gates foundation often commissions random tests before supporting a particular initiative. The Spanish government has created a fund that sponsors only randomised researches. Would the involvement of the government structures lead to "diluting" the method? Is there a threat that the good results could come at the expense of not so precise testing, and ultimately, lead to the old routine ways of aid?
The problem with over-exploiting the method could wait for now. For the time being, it is important to expand the researches based on the random principle, and possibly encompass the entire world, including the developed world. The J-PAL lab has created a branch in Paris dealing with government programs for social aid. And no surprise, the very first tests have discredited the French government as incompetent and inefficient in this respect.
When former French president Nicolas Sarkozy was promoting his idea to put cops in schools to deal with low school attendance, the J-PAL research found a far more effective method: informing the parents and inviting them to school, making meetings between teachers and parents, and all in all, stimulating the parents to cooperate for solving the problem. Entirely voluntarily. The involvement of cops was proven utterly unnecessary, even harmful. Today, the school attendance rates where the method was applied, is similar to that of the elite high schools. Now speaking from personal experience, myself being the deputy principal of a working-class suburban elementary school in one of Africa's metropolises, I can say with certainty that this is one of those many occasions when the results of this method, after being tested locally, could be applied universally.