Automated Speed Tests, take two!
I recently implemented some improvements in the A-Team's Automated Speed Tests as per some requests I got back when I first announced them in July. Not everything's done, but I think this is a good point to advertise what's been changed thus far.
Firstly, I ditched the awful BIRT reports in favour of a custom web app that is faster, easier to use, and more flexible. You can restrict the date range (default is the last four weeks) and switch between tests and machines. The graph is also more responsive when turning on and off particular browsers (just click on the name in the legend). All the same data is there, but it's less cluttered and, well, less ugly!
By the way, BIRT appears to have a security hole in that it will insert the value of some GET parameters directly into the page without sanitizing them! So beware of that if you want to use BIRT for some reason.
Secondly, more tests! The first is MazeSolver, one that Firefox is particularly bad at. The second is test262, a JavaScript conformance test that has unfortunately made the name "Speed Tests" a bit of a lie.
A couple interesting observations I've made recently:
Still more to come, including
Firstly, I ditched the awful BIRT reports in favour of a custom web app that is faster, easier to use, and more flexible. You can restrict the date range (default is the last four weeks) and switch between tests and machines. The graph is also more responsive when turning on and off particular browsers (just click on the name in the legend). All the same data is there, but it's less cluttered and, well, less ugly!
By the way, BIRT appears to have a security hole in that it will insert the value of some GET parameters directly into the page without sanitizing them! So beware of that if you want to use BIRT for some reason.
Secondly, more tests! The first is MazeSolver, one that Firefox is particularly bad at. The second is test262, a JavaScript conformance test that has unfortunately made the name "Speed Tests" a bit of a lie.
A couple interesting observations I've made recently:
- Nightly recently got better at Santa's Workshop. I ran the test myself to see, and Nightly maintains a higher number of elves for longer, but eventually it goes back down to one. So still a ways to go, but the median FPS is higher, at least. Nightly still also doesn't display all the colours properly.
- Seems that Nightly 10.0a1 has one less pass in test262 than 9.0a1.
Still more to come, including
- more tests!
- more browser strains!
- more platforms!