A technical note about a program I wrote last year called bors and some of its ancestry. This is excruciatingly boring unless you happen to build software for a living, in which case I recommend taking a minute to read it.
And the pull-request support in Travis-CI is so close, and yet so far... it tells you whether the PR passed when merged into master, at the time when the test was run. But if other PRs have been merged since then, then the result may be stale. One could trivially detect this, but who cares, right?
Though, looking at the bors-reviewed commits you linked, I will say that it is kind of nice that Travis goes ahead and tests everything immediately, so when reviewing you can refer to the test results (or skip reviewing things that are actually still broken). Best of both worlds would be to test immediately when the PR is submitted, and then when it passes review, check whether those test results are still valid and re-use them if so; or, if master has moved on, re-run.
(I guess this is probably more interesting for "slowish project that gets lots of PRs from random outsiders" than "fast-moving project under active development by small team".)
No, it's more that we weren't (and still aren't, on that test farm) fully isolating test environments, so "run random code someone opened a PR containing without any review" would be a nice way for a stranger to take over one of the buildslaves.
In 2014 I'd probably start with docker and do container-per-PR on a public cloud, which I think is now all the new CI stuff works.
You will be happy to learn that bors-on-travis is now a fully functioning configuration (though undocumented, sigh). If you set the key "use_github_commit_status_api" in the config file, then bors will take its cues about pass/fail status from travis -- on the temporary test-commit branch. Apparently the folks at basho are using it:
This means that all you have to do is configure travis to test the 'auto' branch as well as PRs, and you don't need a buildbot cluster at all. Just bors and travis. Assuming travis gives you sufficient coverage. It's openvz + osx, but you can cross to win32/win64 from linux and run tests under wine these days. So maybe it's enough.
I'm tempted to buy bors.io and set this up as a for-pay CI service. Or just tell the folks at travis to add it as an addon. It makes life so much nicer.
Reply
Reply
(I guess this is probably more interesting for "slowish project that gets lots of PRs from random outsiders" than "fast-moving project under active development by small team".)
Reply
In 2014 I'd probably start with docker and do container-per-PR on a public cloud, which I think is now all the new CI stuff works.
Reply
https://github.com/graydon/bors/commit/353d1ba46d1751b7ac020de4f479bbfc6e134a10
This means that all you have to do is configure travis to test the 'auto' branch as well as PRs, and you don't need a buildbot cluster at all. Just bors and travis. Assuming travis gives you sufficient coverage. It's openvz + osx, but you can cross to win32/win64 from linux and run tests under wine these days. So maybe it's enough.
I'm tempted to buy bors.io and set this up as a for-pay CI service. Or just tell the folks at travis to add it as an addon. It makes life so much nicer.
Reply
Leave a comment