Testing self-driving cars
There's an interesting discussion here of self-driving car safety, and how to measure how effective it is.
And while it seemed reasonable enough, I realised that what I'd really like to see for self-driving cars is...a driving test. In fact, multiple driving tests of different kinds.
Whenever I write code it has unit tests to go with it. Basically _extra_ code which calls into my code and says "When I ask you to do something, do I get the expected response?" For example, "If I call the 'multiply' function with '6' and '9'" do I get back "54"?
Whenever I finish writing some new functionality (or fixing a bug, or anything else that's got me making code changes) all of these tests get run, and if that unit test comes back with "42" then a big checkbox goes red and I don't get to give my updated program to users until everything is green again.
Now, there are two problems with applying this to self-driving cars - one is that the inputs to a self driving car aren't as simple as a pair of numbers, and the other is that different cars are written in completely different ways, with different hardware hooked up to them.
However - I don't think this should make it impossible to write suites of tests for them. If the regulators came together and provided a standard virtual environment, and a set of test cases ("Empty road", "Road with 1 cyclist coming the other way", "Downtown Delhi during rush hour in the pouring rain"), then individual car manufacturers could write translators which allowed their software to interact with this virtual environment.
It wouldn't be perfect, because you'd be approximating the work that the cars sensors do. But for testing the decision-making ability of the cars it would be massively better than nothing. And the development of a shared set of virtual tests would be a boon to all the different self-driving car manufacturers. Heck, if you open-sourced the test suite/environment then anyone at all would be able to try and write self-driving technology which passed all of the tests.
Obviously, this wouldn't be enough to then sell them to the general public. You'd want to then test that the cars worked just as well in real life conditions as they did in virtual ones. But it seems like something worthwhile, which it would be worth regulators investing in.
Original post on Dreamwidth - there are
comments there.
And while it seemed reasonable enough, I realised that what I'd really like to see for self-driving cars is...a driving test. In fact, multiple driving tests of different kinds.
Whenever I write code it has unit tests to go with it. Basically _extra_ code which calls into my code and says "When I ask you to do something, do I get the expected response?" For example, "If I call the 'multiply' function with '6' and '9'" do I get back "54"?
Whenever I finish writing some new functionality (or fixing a bug, or anything else that's got me making code changes) all of these tests get run, and if that unit test comes back with "42" then a big checkbox goes red and I don't get to give my updated program to users until everything is green again.
Now, there are two problems with applying this to self-driving cars - one is that the inputs to a self driving car aren't as simple as a pair of numbers, and the other is that different cars are written in completely different ways, with different hardware hooked up to them.
However - I don't think this should make it impossible to write suites of tests for them. If the regulators came together and provided a standard virtual environment, and a set of test cases ("Empty road", "Road with 1 cyclist coming the other way", "Downtown Delhi during rush hour in the pouring rain"), then individual car manufacturers could write translators which allowed their software to interact with this virtual environment.
It wouldn't be perfect, because you'd be approximating the work that the cars sensors do. But for testing the decision-making ability of the cars it would be massively better than nothing. And the development of a shared set of virtual tests would be a boon to all the different self-driving car manufacturers. Heck, if you open-sourced the test suite/environment then anyone at all would be able to try and write self-driving technology which passed all of the tests.
Obviously, this wouldn't be enough to then sell them to the general public. You'd want to then test that the cars worked just as well in real life conditions as they did in virtual ones. But it seems like something worthwhile, which it would be worth regulators investing in.
Original post on Dreamwidth - there are
comments there.