Non-fandom post on reading scientific articles
There is this thing that has been on my mind, and I think tonight is the time to put in my 2 cents. One of you lovely ladies made a post a while back about the new fad of not vaccinating kids, and then someone on my FB (notably, one of the very few non-medical people I follow there) made a "public awareness" post with a link to a sad blog by a mother of an autistic kid, blaming vaccines, and now the thing is everywhere and is kinda making my blood boil.
I have no intention of getting into that argument. NONE. However, a post about how to read scientific articles critically might be a better contribution. It occurs to me that a lot of people outside the field simply don't know how to tell whether an article is worth the paper it's printed on, and it's difficult to make a decision for yourself if you can't judge how good your sources are. It's not common sense. I can criticize scientific articles by the virtue of having a science degree, and I didn't know how until way beyond the intro science classes. I also remember the first physiology article I tried to read in my life and the horrendous headache the thing gave me. So that's what I'm going to post about - reading a scientific article (any of them) critically.
1. If someone says that scientists "proved" something, it's the first big screaming sign that the person is an amateur. They don't know the first thing about scientific research, which is that you can never prove anything in natural sciences. You can show a strong correlation but never prove. For a trained scientist, "proof" is a dirty word, and if someone uses it to describe a research, that person is not a scientist.
2. Never trust the media and always go to the original source. The media has a long and colorful history of misinterpreting scientific data, either because they don't understand it or because they want to make it sound sensational. Example: I stumbled across an article in LA Times a few years back. "Vitamin D prevents bone cancer!!" Scientists totally proved it. When I went to the original article, it was for a study that looked at the effectiveness of vitamin D supplements in preventing degenerative bone disease in post-menopausal women. They had 60 subjects that they followed long-term. During the course of that research, they happened to notice that women who took Vit D had a lower incidence of bone cancer. They reported that as an interesting thing for future research. That was it. The article made no claims whatsoever about Vit D preventing cancer. That research was no aimed at cancer, which means that they did not screen their subjects for cancer risk factors. It was only done on women, which means that it cannot be generalized to men and women. The article, in fact, could make no statements about cancer because they were not studying cancer. So what they said was "Hey, we found this, might be worth looking at". There is a world of difference between that statement and "Vit D prevents bone cancer!!" that the media turned it into.
3. Any statement that is not supported by data has as much weight as a figure of speech. When an article says "There has been an increasing number of reports all over the world on the health dangers of childhood vaccines" and gives you no reference and no data, it's an equivalent of saying "This is the best sandwich in the world" and carries as much weight. This is important. If they are claiming an "increasing number of reports all over the world", where are they getting this information? Why can't I go to that same place and view all those reports for myself? If it's from a reputable source like WHO or CDC, why would the article neglect to list such strong factual support? So if they don't support their blah blah, it means that by "widespread reports" they mean the housewife blogs and the bag lady who ranted at the head researcher at the bus stop the other day.
4. Who are the subjects in this research? How were they recruited and how many are there? In the discussion of the no vaccination fad I mentioned earlier, someone linked an article that talked about HPV vaccine causing autoimmune disease and ovarian failure in women. That article had three subjects. I have no problem believing that the three women had all these things, and even that they got them as a result of a bad reaction to the HPV vaccine. It is still three subjects, among how many that received the vaccine. It would be a valid study to show that the HPV vaccine can have these rare adverse effects, but a study done on three subjects can never be applied to the general population. Incidentally, that was the same article that claimed some crap about "widespread reports". If this thing they speak of is so common, why can't they get more than three subjects? Plenty of legitimate studies are done on a very small number of subjects, and the limitation of those is that they can't be applied to the general population. By the same principle, a study looking at arthritis cannot make claims about cancer because their subject were not screened properly. A study looking at men only cannot make claims about men and women. Such a study can suggest that a correlation exist, but they'd need to be backed up.
5. Speaking of backing up. If you're reading up on something exotic, like the "playing dead" defense mechanism in crabs, you might get stuck with three articles by the same dude done on "playing dead" in crickets, and then you make do. However, if you're reading up on ovarian cancer, autism or the polio vaccine, there will be a shit ton of studies out there. If you're looking for research on the link between childhood vaccination and autism, and there is one article that looks like gold and makes the scariest claims but no one else can back it up, this article might very well be bullshit. If it's a big topic, no way only one person ever researched it. No backup from other sources likely means that no one else could verify it. I have personally falsified my data to pass a class. There was an infamous study that claimed marijuana use lead to progressive degeneration of brain cells, and it was later revealed that the researcher used very high doses of methamphetamines and not MJ. People lie and people make honest mistakes. If it's a big claim on a hot topic and no one can back it up, don't believe that claim, even if it's coming from a scientific article.
6. It is important who funds the study. I will have little faith in the study that claims that HPV vaccine is the hottest shit if it was funded by a pharmaceutical company that will draw huge profits from more people getting the vaccine. I want nothing to do with a study that shows how children raised by gay couples grow up to be criminals if the Baptist Church is behind it. A bias like that doesn't make the study invalid but it means that you should take it with a grain of salt.
There are bad articles that try to fuck with you, and there are good ones that get wildly misinterpreted by the general public. I have no problem with someone who looked at the data critically and made a conclusion. I do have a problem with a person that makes Facebook posts of doom and decides not to vaccinate their kids because some celebrity said so on Twitter, and because she knows someone down the street whose kid has autism and who KNOWS FOR ABSOLUTE SURE it was caused by the MMR series. Vaccines have existed since 1796, people. I find it deeply embarrassing that we're going to reintroduce polio in the 21st century because of a fucking fad. I'm sure a lot of people follow that trend not because they're dumb but because they don't know how to read scientific data, so there is my two cents. I'm all for informed decisions.
In other news, I learned how to put in sutures and it was awesome \o/ Also, just for fun, here is a cool Wiki article about Dr. James Barry, who was a surgeon in the British Army, and who was also quite possibly a woman living as a man in the times when medical schools only accepted men. That's some dedication :D
I have no intention of getting into that argument. NONE. However, a post about how to read scientific articles critically might be a better contribution. It occurs to me that a lot of people outside the field simply don't know how to tell whether an article is worth the paper it's printed on, and it's difficult to make a decision for yourself if you can't judge how good your sources are. It's not common sense. I can criticize scientific articles by the virtue of having a science degree, and I didn't know how until way beyond the intro science classes. I also remember the first physiology article I tried to read in my life and the horrendous headache the thing gave me. So that's what I'm going to post about - reading a scientific article (any of them) critically.
1. If someone says that scientists "proved" something, it's the first big screaming sign that the person is an amateur. They don't know the first thing about scientific research, which is that you can never prove anything in natural sciences. You can show a strong correlation but never prove. For a trained scientist, "proof" is a dirty word, and if someone uses it to describe a research, that person is not a scientist.
2. Never trust the media and always go to the original source. The media has a long and colorful history of misinterpreting scientific data, either because they don't understand it or because they want to make it sound sensational. Example: I stumbled across an article in LA Times a few years back. "Vitamin D prevents bone cancer!!" Scientists totally proved it. When I went to the original article, it was for a study that looked at the effectiveness of vitamin D supplements in preventing degenerative bone disease in post-menopausal women. They had 60 subjects that they followed long-term. During the course of that research, they happened to notice that women who took Vit D had a lower incidence of bone cancer. They reported that as an interesting thing for future research. That was it. The article made no claims whatsoever about Vit D preventing cancer. That research was no aimed at cancer, which means that they did not screen their subjects for cancer risk factors. It was only done on women, which means that it cannot be generalized to men and women. The article, in fact, could make no statements about cancer because they were not studying cancer. So what they said was "Hey, we found this, might be worth looking at". There is a world of difference between that statement and "Vit D prevents bone cancer!!" that the media turned it into.
3. Any statement that is not supported by data has as much weight as a figure of speech. When an article says "There has been an increasing number of reports all over the world on the health dangers of childhood vaccines" and gives you no reference and no data, it's an equivalent of saying "This is the best sandwich in the world" and carries as much weight. This is important. If they are claiming an "increasing number of reports all over the world", where are they getting this information? Why can't I go to that same place and view all those reports for myself? If it's from a reputable source like WHO or CDC, why would the article neglect to list such strong factual support? So if they don't support their blah blah, it means that by "widespread reports" they mean the housewife blogs and the bag lady who ranted at the head researcher at the bus stop the other day.
4. Who are the subjects in this research? How were they recruited and how many are there? In the discussion of the no vaccination fad I mentioned earlier, someone linked an article that talked about HPV vaccine causing autoimmune disease and ovarian failure in women. That article had three subjects. I have no problem believing that the three women had all these things, and even that they got them as a result of a bad reaction to the HPV vaccine. It is still three subjects, among how many that received the vaccine. It would be a valid study to show that the HPV vaccine can have these rare adverse effects, but a study done on three subjects can never be applied to the general population. Incidentally, that was the same article that claimed some crap about "widespread reports". If this thing they speak of is so common, why can't they get more than three subjects? Plenty of legitimate studies are done on a very small number of subjects, and the limitation of those is that they can't be applied to the general population. By the same principle, a study looking at arthritis cannot make claims about cancer because their subject were not screened properly. A study looking at men only cannot make claims about men and women. Such a study can suggest that a correlation exist, but they'd need to be backed up.
5. Speaking of backing up. If you're reading up on something exotic, like the "playing dead" defense mechanism in crabs, you might get stuck with three articles by the same dude done on "playing dead" in crickets, and then you make do. However, if you're reading up on ovarian cancer, autism or the polio vaccine, there will be a shit ton of studies out there. If you're looking for research on the link between childhood vaccination and autism, and there is one article that looks like gold and makes the scariest claims but no one else can back it up, this article might very well be bullshit. If it's a big topic, no way only one person ever researched it. No backup from other sources likely means that no one else could verify it. I have personally falsified my data to pass a class. There was an infamous study that claimed marijuana use lead to progressive degeneration of brain cells, and it was later revealed that the researcher used very high doses of methamphetamines and not MJ. People lie and people make honest mistakes. If it's a big claim on a hot topic and no one can back it up, don't believe that claim, even if it's coming from a scientific article.
6. It is important who funds the study. I will have little faith in the study that claims that HPV vaccine is the hottest shit if it was funded by a pharmaceutical company that will draw huge profits from more people getting the vaccine. I want nothing to do with a study that shows how children raised by gay couples grow up to be criminals if the Baptist Church is behind it. A bias like that doesn't make the study invalid but it means that you should take it with a grain of salt.
There are bad articles that try to fuck with you, and there are good ones that get wildly misinterpreted by the general public. I have no problem with someone who looked at the data critically and made a conclusion. I do have a problem with a person that makes Facebook posts of doom and decides not to vaccinate their kids because some celebrity said so on Twitter, and because she knows someone down the street whose kid has autism and who KNOWS FOR ABSOLUTE SURE it was caused by the MMR series. Vaccines have existed since 1796, people. I find it deeply embarrassing that we're going to reintroduce polio in the 21st century because of a fucking fad. I'm sure a lot of people follow that trend not because they're dumb but because they don't know how to read scientific data, so there is my two cents. I'm all for informed decisions.
In other news, I learned how to put in sutures and it was awesome \o/ Also, just for fun, here is a cool Wiki article about Dr. James Barry, who was a surgeon in the British Army, and who was also quite possibly a woman living as a man in the times when medical schools only accepted men. That's some dedication :D