[LJ Idol 9: Week 5] Building Better Science
I would guess that most of us have heard that the large, double-blind, randomized controlled trial is supposed to be the best kind of research - if one is particularly fanatical on this topic, it is the ONLY kind of research worth acknowledging. Never mind the practicalities - sometimes "blinding" research subjects to what they are receiving is not possible. Never mind the ethics - in a particularly infamous recent example near me, the mayor's office of a large city had proposed, in the name of research, randomly assigning homeless people seeking shelter to either receive or not receive shelter, in the supposed interests of scientific inquiry about how best to help the homeless! The more objective and controlled the experiment, the "better" it is.
The pursuit of neutrality and objectivity is, all too often, a pursuit of the status quo. It is a relatively straightforward path to become a recognized researcher by building on common assumptions or prior findings about how a process works, but not so simple to engage in research that stands to contradict a cherished assumption researchers hold about a discovery they think they made.
Consider the case of Rat Park. If you are not familiar with the story of Rat Park, I will summarize:
In 1977, a group of researchers at Simon Fraser University reviewed existing publications over the previous
twenty years or so on the topic of lab rats becoming so addicted to various drugs that they would choose drugs over food and even drug themselves to death in their laboratory cages. It occurred to them that something was missing - rats, naturally, are social animals. The lab cage or "Skinner box" situation was not a natural environment for these animals. As such, the researchers wondered how much could be learned about human or even animal response to the presence of addictive drugs in their natural environment by studying animals in a decidedly unnatural environment.
So, they decided to do an experiment. Or rather, a series of experiments. Some rats were assigned to standard laboratory cages, and others to "Rat Park" - a spacious enclosure with nature scenes painted on the walls, cedar for nesting in, interesting toys to play with, and (perhaps most importantly) other rats to interact with, including the possibility of producing baby ratlings. To quickly summarize the findings:
- Rats in standard cages were more likely to be interested in morphine-laced water than rats in Rat Park.
- Rats in Rat Park would, in fact, avoid morphine-laced water even when it was also heavily sweetened, in favor of plain water, until the amount of morphine in the water was decreased or a second drug that blocks the effects of morphine was added.
- Rats who had been given only morphine-laced water until they developed physical dependence on morphine, when given the opportunity to "kick the habit" and drink clean water, overwhelmingly chose to do so if they lived in Rat Park, but not if they lived in standard cages.
Of course, the big-deal journals didn't want to touch these results. One of the researchers later commented that the peer reviewers from Science and Nature who turned down the paper could not point to a specific methodological flaw but nonetheless strongly gave the message that something had to be wrong somewhere with the results. Smaller journals eventually did pick up and publish the results of the Rat Park experiments*, but the lab lost its funding soon after.
In recent years, Rat Park has received renewed attention for what it says about treating chemical dependency. I think it may be equally important for what it says about how we treat science. The idea that results that give a different explanation than the one with current popular scientific acceptance must be wrong somehow is alive and well, as is the drive to support and fund only those interventions that are "evidence-based".
To a point, this is probably a good thing. When we support evidence-based methods, we move away from things like Scared Straight, a program that takes teenagers who have been involved in minor acts of juvenile delinquency and brings them into prisons so inmates can lecture them away from a life of crime. "Everyone knows" this will work - except research shows that likelihood of reoffending increases with exposure to this type of program. Oops.
On the other hand, when we mechanically apply "evidence-based" interventions as a pre-packaged solution, there is the possibility that we are treating the wrong problem. I've written before about seeing this happen in the field - the young mother misdiagnosed as psychotic due to linguistic and cultural barriers between her and the diagnosing psychiatrist, the gay young man whose truancy had him labeled as "oppositional" and "defiant" when he was skipping school to avoid homophobic bullies, the teenagers supposedly "beyond lawful control" of their parents whose parents were actually asking them to do unlawful things.
You can't build a better treatment, or pick an existing treatment to use, when you don't understand the problem in the first place.
-------
* If you want to see the Rat Park research for yourself, check out Alexander, Coambs, and Hadaway 1978 and Hadaway, Alexander, Coambs, and Beyerstein 1979. (Links go to actual articles, on a server that provides the first few pages free. A good summary of the experiments, in comic form, is available here.)
The pursuit of neutrality and objectivity is, all too often, a pursuit of the status quo. It is a relatively straightforward path to become a recognized researcher by building on common assumptions or prior findings about how a process works, but not so simple to engage in research that stands to contradict a cherished assumption researchers hold about a discovery they think they made.
Consider the case of Rat Park. If you are not familiar with the story of Rat Park, I will summarize:
In 1977, a group of researchers at Simon Fraser University reviewed existing publications over the previous
twenty years or so on the topic of lab rats becoming so addicted to various drugs that they would choose drugs over food and even drug themselves to death in their laboratory cages. It occurred to them that something was missing - rats, naturally, are social animals. The lab cage or "Skinner box" situation was not a natural environment for these animals. As such, the researchers wondered how much could be learned about human or even animal response to the presence of addictive drugs in their natural environment by studying animals in a decidedly unnatural environment.
So, they decided to do an experiment. Or rather, a series of experiments. Some rats were assigned to standard laboratory cages, and others to "Rat Park" - a spacious enclosure with nature scenes painted on the walls, cedar for nesting in, interesting toys to play with, and (perhaps most importantly) other rats to interact with, including the possibility of producing baby ratlings. To quickly summarize the findings:
- Rats in standard cages were more likely to be interested in morphine-laced water than rats in Rat Park.
- Rats in Rat Park would, in fact, avoid morphine-laced water even when it was also heavily sweetened, in favor of plain water, until the amount of morphine in the water was decreased or a second drug that blocks the effects of morphine was added.
- Rats who had been given only morphine-laced water until they developed physical dependence on morphine, when given the opportunity to "kick the habit" and drink clean water, overwhelmingly chose to do so if they lived in Rat Park, but not if they lived in standard cages.
Of course, the big-deal journals didn't want to touch these results. One of the researchers later commented that the peer reviewers from Science and Nature who turned down the paper could not point to a specific methodological flaw but nonetheless strongly gave the message that something had to be wrong somewhere with the results. Smaller journals eventually did pick up and publish the results of the Rat Park experiments*, but the lab lost its funding soon after.
In recent years, Rat Park has received renewed attention for what it says about treating chemical dependency. I think it may be equally important for what it says about how we treat science. The idea that results that give a different explanation than the one with current popular scientific acceptance must be wrong somehow is alive and well, as is the drive to support and fund only those interventions that are "evidence-based".
To a point, this is probably a good thing. When we support evidence-based methods, we move away from things like Scared Straight, a program that takes teenagers who have been involved in minor acts of juvenile delinquency and brings them into prisons so inmates can lecture them away from a life of crime. "Everyone knows" this will work - except research shows that likelihood of reoffending increases with exposure to this type of program. Oops.
On the other hand, when we mechanically apply "evidence-based" interventions as a pre-packaged solution, there is the possibility that we are treating the wrong problem. I've written before about seeing this happen in the field - the young mother misdiagnosed as psychotic due to linguistic and cultural barriers between her and the diagnosing psychiatrist, the gay young man whose truancy had him labeled as "oppositional" and "defiant" when he was skipping school to avoid homophobic bullies, the teenagers supposedly "beyond lawful control" of their parents whose parents were actually asking them to do unlawful things.
You can't build a better treatment, or pick an existing treatment to use, when you don't understand the problem in the first place.
-------
* If you want to see the Rat Park research for yourself, check out Alexander, Coambs, and Hadaway 1978 and Hadaway, Alexander, Coambs, and Beyerstein 1979. (Links go to actual articles, on a server that provides the first few pages free. A good summary of the experiments, in comic form, is available here.)