In Which Your Obedient Serpent Defends the "Students"
There's been a bit of a kerfluffle about a recent study about students who fell for a hoax website about the Pacific Northwest Tree Octopus.
Frankly, the article linked above is a shoddy piece of science journalism. As eggshellhammer pointed out, it doesn't link to the original study. Even worse, in Your Obedient Serpent's eyes: it didn't specify the age level of the students. That's an important factor: a study about the critical thinking ability of kindergarten students has entirely different implications than the same study about a group of college undergraduates.
That in itself is an indication of a failure of critical thinking ability in would-be science journalists.
As it transpires, this study involved seventh-graders. The conclusion can thus be summarized as, "wow, you can con a 12-year-old into believing some crazy shit", which is hardly earth-shattering news. I'd say three-quarters of the contents of snopes.com is stuff that was repeated as gospel truth on the Bicentennial schoolyards of my twelfth year.
(I find the datum that students ignore search engines in favor of randomlytypinginaname.com to be much more startling, personally. Seriously, WTF?)
The other study mentioned in the University of Connecticut article suggests that this, in large measure, just reflects a need for improved emphasis on Internet search and access skills, and not some Terrible Crisis in Education. That's how the researchers seem to interpret it; the DANGER WILL ROBINSON! reactions were mostly imposed by the secondary sources. For my part, I was intrigued and, on some level, amused at the revelation that students who had difficulties with traditional literacy showed superior online reading facilities.
As for the details of the first study ... I'm going to be generous and completely ignore the implications of drawing broad conclusions from a sample group of twenty-five students in a single class. Let's assume, for the sake of argument, that this specific class is representative of the entire population of students in Connecticut. Let's take a look at two of the sited conclusions:
• All but one of the 25 rated the site as "very credible" ...
Let us, just for a moment, step out of the role of of the Know-It-All Grown-Up Who Knows This Site Is Patently Absurd Because There's No Such Thing. Let us remember that those reading this journal are likely to have at least five more years of formal education than the subjects of this study.
Yes, http://zapatopi.net/treeoctopus/ is "very credible".
"Credible" doesn't mean "true" or "accurate". It means "able to be believed", or "capable of persuading". The website has a professional presentation and a serious, convincing tone. The only obvious joke on the main page (aside from a deadpan link to sasquatch) is a reference to the organization "Greenpeas". The FAQ gets increasingly flippant and absurdist, but they avoid an overtly humorous tone for the main body.
Given that aquarium octopuses are well-known for getting out of their tanks and taking walks, and that there is at least one species of land-dwelling, arboreal hermit crabs, the idea of a "tree octopus" is just plausible enough to someone who knows just how weird and wacky life on Earth can get.
In science, "credibility" also means "reproducibility", and in this context, that extends to being able to find other corroborating sources.
This leads us to the second conclusion I want to examine:
• Most struggled when asked to produce proof - or even clues - that the web site was false ...
Hey, it's an exercise for the class! Let's check our own research and critical thinking abilities, shall we?
I'm curious to see what proofs (or even clues!) the folks reading this can come up with, above and beyond the flippant tone of the FAQ that I mentioned above. The Sasquatch link leads to an equally-deadpan page, of course.
Needless to say, "I just know there's no such thing" isn't a valid "proof"; in fact, it doesn't even rate as a "clue".
Answers will be graded!
Thanks ... and apologies ... to pseudomanitou for drawing my attention to this study and the reactions which followed. Please don't think I'm being an asshole for deconstructing this.Update: eggshellhammer contacted the original author and scored a link to the original document. Yes, the sample group was larger than 25.
Frankly, the article linked above is a shoddy piece of science journalism. As eggshellhammer pointed out, it doesn't link to the original study. Even worse, in Your Obedient Serpent's eyes: it didn't specify the age level of the students. That's an important factor: a study about the critical thinking ability of kindergarten students has entirely different implications than the same study about a group of college undergraduates.
That in itself is an indication of a failure of critical thinking ability in would-be science journalists.
As it transpires, this study involved seventh-graders. The conclusion can thus be summarized as, "wow, you can con a 12-year-old into believing some crazy shit", which is hardly earth-shattering news. I'd say three-quarters of the contents of snopes.com is stuff that was repeated as gospel truth on the Bicentennial schoolyards of my twelfth year.
(I find the datum that students ignore search engines in favor of randomlytypinginaname.com to be much more startling, personally. Seriously, WTF?)
The other study mentioned in the University of Connecticut article suggests that this, in large measure, just reflects a need for improved emphasis on Internet search and access skills, and not some Terrible Crisis in Education. That's how the researchers seem to interpret it; the DANGER WILL ROBINSON! reactions were mostly imposed by the secondary sources. For my part, I was intrigued and, on some level, amused at the revelation that students who had difficulties with traditional literacy showed superior online reading facilities.
As for the details of the first study ... I'm going to be generous and completely ignore the implications of drawing broad conclusions from a sample group of twenty-five students in a single class. Let's assume, for the sake of argument, that this specific class is representative of the entire population of students in Connecticut. Let's take a look at two of the sited conclusions:
• All but one of the 25 rated the site as "very credible" ...
Let us, just for a moment, step out of the role of of the Know-It-All Grown-Up Who Knows This Site Is Patently Absurd Because There's No Such Thing. Let us remember that those reading this journal are likely to have at least five more years of formal education than the subjects of this study.
Yes, http://zapatopi.net/treeoctopus/ is "very credible".
"Credible" doesn't mean "true" or "accurate". It means "able to be believed", or "capable of persuading". The website has a professional presentation and a serious, convincing tone. The only obvious joke on the main page (aside from a deadpan link to sasquatch) is a reference to the organization "Greenpeas". The FAQ gets increasingly flippant and absurdist, but they avoid an overtly humorous tone for the main body.
Given that aquarium octopuses are well-known for getting out of their tanks and taking walks, and that there is at least one species of land-dwelling, arboreal hermit crabs, the idea of a "tree octopus" is just plausible enough to someone who knows just how weird and wacky life on Earth can get.
In science, "credibility" also means "reproducibility", and in this context, that extends to being able to find other corroborating sources.
This leads us to the second conclusion I want to examine:
• Most struggled when asked to produce proof - or even clues - that the web site was false ...
Hey, it's an exercise for the class! Let's check our own research and critical thinking abilities, shall we?
I'm curious to see what proofs (or even clues!) the folks reading this can come up with, above and beyond the flippant tone of the FAQ that I mentioned above. The Sasquatch link leads to an equally-deadpan page, of course.
Needless to say, "I just know there's no such thing" isn't a valid "proof"; in fact, it doesn't even rate as a "clue".
Answers will be graded!
Thanks ... and apologies ... to pseudomanitou for drawing my attention to this study and the reactions which followed. Please don't think I'm being an asshole for deconstructing this.Update: eggshellhammer contacted the original author and scored a link to the original document. Yes, the sample group was larger than 25.