Need stats stat.
Dear Lazyweb, Please help.
Anyone out there able to give some statistics advice?
We're trying to characterize a rare but highly annoying bug at work. With that and some other personal (game design) reasons I decided I needed to pick up some probability and statistics knowledge that I never got in school.
I've ordered some well reviewed but not traditional stats texts:
Statistics, Third Edition
The Nature of Statistics
and started working through an online course here
Any other recommendations for a way to pick this up?
We have a rare failure. The failure rate appears to change drastically with some specific changes to the test suite. I'd like to be able to compare test runs on specific test beds to test how conditions effect the fault and by how much and then to get a sense of how often it's actually occurring and how likely people are to be bit by it.
We can run a series of automated tests and eventually get a failure and can record how many trials it we did before the failure. A typical (and rough) set of results looks something like 500, 34000, 1500, 22000, 20000 trials before failure.
What I think I'm looking for is:
On each test what is the chance of a failure?
Where is the 50% chance of failure after N tests?
Home many test runs (of N tests) do we need to run on each test bed before the results we get are accurate enough to be useful?
I'm not even sure that those numbers are the best way to make these comparisons. If I'm off base I need to know that too.
Anyone out there able to give some statistics advice?
We're trying to characterize a rare but highly annoying bug at work. With that and some other personal (game design) reasons I decided I needed to pick up some probability and statistics knowledge that I never got in school.
I've ordered some well reviewed but not traditional stats texts:
Statistics, Third Edition
The Nature of Statistics
and started working through an online course here
Any other recommendations for a way to pick this up?
We have a rare failure. The failure rate appears to change drastically with some specific changes to the test suite. I'd like to be able to compare test runs on specific test beds to test how conditions effect the fault and by how much and then to get a sense of how often it's actually occurring and how likely people are to be bit by it.
We can run a series of automated tests and eventually get a failure and can record how many trials it we did before the failure. A typical (and rough) set of results looks something like 500, 34000, 1500, 22000, 20000 trials before failure.
What I think I'm looking for is:
On each test what is the chance of a failure?
Where is the 50% chance of failure after N tests?
Home many test runs (of N tests) do we need to run on each test bed before the results we get are accurate enough to be useful?
I'm not even sure that those numbers are the best way to make these comparisons. If I'm off base I need to know that too.