The move to git
The time has come to bite the bullet and move Fedora's package source control on from CVS. CVS has done is well, and although it is a decaying source control, it handled our needs rather well for many years. However nothing is a constant, and over time more and more cracks have shown up in our source control. The time to move on is now, and I feel pretty confident in the plan we are exploring.
First though, why git? Git has skyrocketed in popularity over the past few years. More than I ever imagined, when I first toyed with the idea of doing the jump to git, back in the Fedora Core 6 days. Most notably the Gnome project has recently made the leap from SVN over to git (amusingly they started in CVS, we'll just be skipping that SVN step). Not only is git popular, it's really a better choice. Why git is better than x does a good job at hitting some of the highlights. Fedora is also a meritocracy, in that the people doing the work are generally the people making the decisions. Code talks. Since the work to do the migration is largely going to fall on me (because I've volunteered to do it) I get to make some of these decisions. Looking at the source control landscape over the past few years I feel confident that git is the way to go.
How are we going to do it? I've been thinking on this problem for years. There are two ideals that I really want to satisfy. Firstly, I want to make sure we stay true to our principles and differ from upstream as little as possible. To facilitate that I want to make it easy to discover our modifications and pass them along to upstream and/or other consumers of a given upstream. One way to force this is to continue working with a upstream archive (tarball release, maybe a snapshot) and our modifications as patches. Easy to see our changes, easy to pass them around. The other ideal is that working directly on source code in a scm repository is far better than trying to work with a tarball + patches. Trying to reconcile both of these ideals has been hard for me to manage, until the light went on the other day...
Git helps here in a few ways. Creating new repos easy, fast, and scriptable. Since it's a DSCM everything can be done locally on any filesystem. Commits to git can be exported in a text file format that is easily transportable and contains not just the diff, but context about the change such as author, reason, further log info, etc... These exported commits can be applied to a given repo easily and quickly, even if it's a brand new repo. So while we can maintain a forward moving repository of .spec file and patch files management, we can also on the fly create "throw away" repos of the upstream source code, plus our patches applied git style, leaving the developer with a repo that they can do development/patch management with and export the results back into our package source control, throwing stuff at upstream along the way. This really is the best I could come up with for the best of both worlds.
Details? See my slides.
Time line? I'm currently throwing our CVS archives at the cvs to git conversion tools, cvs2git (part of cvs2svn) and the git cvsimport utility. I think I have a good strategy for getting our release sub directories imported as real branches of a package module, where the master or unbranched content would be the rawhide content. Lots of work needed here to verify that the import is doing the right thing. But that's as far as I've gotten. I'm still in the idea phase, attempting to proof of concept my current idea and discovering where it falls down. I hope to get through this phase in the next few weeks so that we can get to the proposal phase where we get FESCo buy in. Implementation depends on the amount of work needed, but hopefully near Fedora 13 release.
Help wanted! I would love to have your help. Your thoughts on my plan, your experience in this area, your time to write scripts, your time to verify conversion output, your time to document progress in wiki pages, etc... As I work on the raw cvs into git conversion, we'll need a fpkg utility written that will take over from the Make system. Work on this tool can be very modular and well suited for many people working on it collaboratively (hey, great idea for a Fedora Activity Day!). Once we get a good format settled on for the packages and a good start on fpkg development, we'll need koji changes to be able to build from this style of repo. Good news, that code is pretty isolated and there already exists some git code. If you're interested in helping out, find me and I'll try to put you to work.
Keeping up with progress. I plan to talk about our progress in this task at our weekly Fedora Release Engineering meetings. Those meetings are summarized on fedora-devel-list so you can keep watch there. Eventually there will be a wiki page that will track progress, as well as a trac milestone and ticket set and all sorts of fun project management stuff like that as we progress.
First though, why git? Git has skyrocketed in popularity over the past few years. More than I ever imagined, when I first toyed with the idea of doing the jump to git, back in the Fedora Core 6 days. Most notably the Gnome project has recently made the leap from SVN over to git (amusingly they started in CVS, we'll just be skipping that SVN step). Not only is git popular, it's really a better choice. Why git is better than x does a good job at hitting some of the highlights. Fedora is also a meritocracy, in that the people doing the work are generally the people making the decisions. Code talks. Since the work to do the migration is largely going to fall on me (because I've volunteered to do it) I get to make some of these decisions. Looking at the source control landscape over the past few years I feel confident that git is the way to go.
How are we going to do it? I've been thinking on this problem for years. There are two ideals that I really want to satisfy. Firstly, I want to make sure we stay true to our principles and differ from upstream as little as possible. To facilitate that I want to make it easy to discover our modifications and pass them along to upstream and/or other consumers of a given upstream. One way to force this is to continue working with a upstream archive (tarball release, maybe a snapshot) and our modifications as patches. Easy to see our changes, easy to pass them around. The other ideal is that working directly on source code in a scm repository is far better than trying to work with a tarball + patches. Trying to reconcile both of these ideals has been hard for me to manage, until the light went on the other day...
Git helps here in a few ways. Creating new repos easy, fast, and scriptable. Since it's a DSCM everything can be done locally on any filesystem. Commits to git can be exported in a text file format that is easily transportable and contains not just the diff, but context about the change such as author, reason, further log info, etc... These exported commits can be applied to a given repo easily and quickly, even if it's a brand new repo. So while we can maintain a forward moving repository of .spec file and patch files management, we can also on the fly create "throw away" repos of the upstream source code, plus our patches applied git style, leaving the developer with a repo that they can do development/patch management with and export the results back into our package source control, throwing stuff at upstream along the way. This really is the best I could come up with for the best of both worlds.
Details? See my slides.
Time line? I'm currently throwing our CVS archives at the cvs to git conversion tools, cvs2git (part of cvs2svn) and the git cvsimport utility. I think I have a good strategy for getting our release sub directories imported as real branches of a package module, where the master or unbranched content would be the rawhide content. Lots of work needed here to verify that the import is doing the right thing. But that's as far as I've gotten. I'm still in the idea phase, attempting to proof of concept my current idea and discovering where it falls down. I hope to get through this phase in the next few weeks so that we can get to the proposal phase where we get FESCo buy in. Implementation depends on the amount of work needed, but hopefully near Fedora 13 release.
Help wanted! I would love to have your help. Your thoughts on my plan, your experience in this area, your time to write scripts, your time to verify conversion output, your time to document progress in wiki pages, etc... As I work on the raw cvs into git conversion, we'll need a fpkg utility written that will take over from the Make system. Work on this tool can be very modular and well suited for many people working on it collaboratively (hey, great idea for a Fedora Activity Day!). Once we get a good format settled on for the packages and a good start on fpkg development, we'll need koji changes to be able to build from this style of repo. Good news, that code is pretty isolated and there already exists some git code. If you're interested in helping out, find me and I'll try to put you to work.
Keeping up with progress. I plan to talk about our progress in this task at our weekly Fedora Release Engineering meetings. Those meetings are summarized on fedora-devel-list so you can keep watch there. Eventually there will be a wiki page that will track progress, as well as a trac milestone and ticket set and all sorts of fun project management stuff like that as we progress.