Warning: maaaaaaajor geekery ahead! A closer look at Doctor Who ratings

[kilodalton]

OK so I like math. Although I'm not a statistician, I am pretty good with math (which is lucky for anyone in the USA haha bc in a few short months I will be using said math to prepare intravenous drug compounds for hospitalized patients... tl;dr if I sucked at this, it would suck WAAAY worse to be you bwahahah XD)
ANYWAY! It should not have escaped

Comments

[hammard]

You should read Tom Spilsbury's post on this subject he did back during Season 6 where he analysed ALL the ratings and showed the change was very little:
http://tomspilsbury.moonfruit.com/#/home/4554491282/Let%27s-Kill-This-Myth/195123

Whilst we cannot analyse all the recent ratings in this way due to lack of data (Iplayer stats will not be available for a while) it is probable to be the same.

What the figures are possibly mapping is the move from TV to online watching (just as the strong downwards trend from S1 to S2 tracks people moving to repeat vieiwings

[eve11]

Another interesting story would be to start with eg, episode 4 in each series, and plot the quadratic curves point by point as you gain new points with each new episode, and see how much the prediction changes with the next new data point. The future predictive trend depends rather massively on where it estimates the inflection point.

[kilodalton]

Eh Spilsbury's post is kind of what I was saying about people who just look at the raw data and don't look for any statistical significance or trendlines. (And besides the trendlines, I went so far as to run a one-tailed T-test: Spilsbury is wrong. Season ONE is the outlier and is statistically significant, not season 4 -- see what I mean about misleading data on the surface?) Not that I'm surprised though - he makes his living off DWM, what the heck is he supposed to say lol

[eve11]

Can you explain to me what you mean by "Season 1 is statistically significant?" Do you mean the average ratings for season 1 are statistically significantly different from other seasons? You have 13 correlated data points in a temporal trend, and I'm willing to bet that whatever calculation was done in terms of a simple t-test for statistical significance is making assumptions that the data populations do not follow in this case.

[kilodalton]

Do you mean the average ratings for season 1 are statistically significantly different from other seasons?

Statistically significantly higher than s7. I didn't compare it to other seasons, I just wanted to see how other seasons compared to s7.

[eve11]

Higher how? Average? Looks not. More like pairwise difference episode by episode? s1 is the only one I see on the chart where s7 is lower on a point by point basis. If you don't account for episode by episode there doesn't seem to be a difference in the range. I would also worry about normality assumptions with this data and would do a bootstrap or permutation test before a t-test for significance.

[kilodalton]

More like pairwise difference episode by episode?

Yup.

I would also worry about normality assumptions with this data and would do a bootstrap or permutation test before a t-test for significance.

... True, but again - if it's good enough for the New England Journal of Medicine or Pharmacotherapy to print that X cancer treatment is better than Y cancer treatment because of a retrospective medical chart review with the statistics based on paired T tests, then it's good enough for me sitting at home bored on a Thursday night trying to figure out silly TV ratings XD

(Maybe they should hire more statisticians? Or maybe we should just try not to get cancer so our MDs don't give us treatment X based on imperfect stats XD)

[eve11]

Eh, Medical stats is a rather staid field. A lot of the basic stats they have are basic because they are derived from well-designed and highly controlled experiments, and because the researchers don't know any better. Sample size is also pretty important in using t-tests. I just don't know what it really means that s1 and s7 are pairwise separated, when they both also seem to be lying right smack in the middle of the point cloud. What is the interpretation of this difference in that case? There are only 7 data points per episode in the list: it might be more useful to look at something like relative ranking per episode number? That would include information from the obvious outliers but would mitigate their effect

[kilodalton]

I will definitely be updating this as the final BARB numbers keep rolling in - we'll see where it ends up!! =)

[eve11]

Well you have piqued my curiosity so I scraped the info from the wikipedia pages (air date, writer, director, rating, appreciation index), and for context I am also working on parsing the top 30 episodes per channel scraped from the BARB "top 30" site for all of the weeks Doctor Who aired. I just need to write a little python parser to get everything in a simple pipe delimited format, as it right now has the table headers and week names mixed in (plus when it has "other" categories, it changes the format, bleh).

Let me know if you want the data set when I am finished. It will have air dates, times, ratings, rankings on BBC1, rankings on the other big channels from the BARB top 30 tables, and I will put in labels of other covariates: 2-parters, and also I was curious about how well episodes with famous old school monsters do, comparatively. There will be two tables that can be cross classified as to the leading show, competing shows on different channels, and overall ratings trends per channel for each week.

[kilodalton]

Wow, sure thing, thanks!

[eve11]

Two files

[kilodalton]

Thanks for these, they'll be interesting to peruse! My updates to this will still be based on my methodology tho. Honestly, I think for our purposes it's enough -- and one of the most important things is making sure to keep it simple and understandable. Occam's Razor. Even with my data as it stands some folks had trouble following it - and I made a point to keep it VERY user-friendly. If I start chucking around too many terms and parabolas nobody is going to understand it lol.

[eve11]

Surely that's your prerogative. You can see on my lj why I think the global curves fit to this data is a bad idea. I drew pictures. I tend to try also to keep it simple, which is why I start with informative graphs & EDA and let those drive any hypotheses.

[kilodalton]

No I totally get it - but honestly I think it's good enough. And what you were saying about lack of statistical significance between s5-7 is what I found too using my methodology. I'm not arguing with yours at all, you're clearly good at what you do, I just think that mine suits the purpose as wel =)