GitHub migration again -- The Plan

Stefan Negrea snegrea at redhat.com
Thu Feb 28 21:53:33 UTC 2013



----- Original Message -----
> From: "Lukas Krejci" <lkrejci at redhat.com>
> To: rhq-devel at lists.fedorahosted.org
> Sent: Wednesday, February 27, 2013 10:37:12 AM
> Subject: Re: GitHub migration again -- The Plan
> 
> On Wednesday, February 27, 2013 16:02:30 Thomas Segismont wrote:
> > Le 27/02/2013 15:08, Heiko W.Rupp a écrit :
> > > Hi,
> > > 
> > > I've updated the wiki page including "The Plan".
> > > 
> > > I would like to finish this Plan within the coming 7 days so that
> > > we will be able to switch over soon.
> > > 
> > > https://docs.jboss.org/author/display/RHQ/GitHub+migration#GitHubmigration
> > > -Processofthemigrationitself%28%22Theplan%22%29
> > > 
> > > Please comment. If you think it is ok, also say so please.
> > > 
> > >     Heiko
> > > 
> > > _______________________________________________
> > > rhq-devel mailing list
> > > rhq-devel at lists.fedorahosted.org
> > > https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
> > 
> > Hi Heiko,
> > 
> > I read the wiki page and everything git-related looks fine to me.
> > 
> > Still, if we choose to take the opportunity of the Github move to
> > split
> > the repo, I think we should also document the new build mechanism
> > and
> > the relationship between modules and core repository.
> > 
> 
> +1

Not only we would need to think about the new build mechanism but also have a plan in place and allocate time to update the release scripts. Over the past few years we improved from having a publishable release build in a 2-3 weeks time-frame down to about 8 hours. I think its important to keep the same level if not reduce the complexity and amount of time required for a release build.

> 
> > Splitting the repository will have important implications on CI
> > side and
> > we might also need a plan for Jenkins jobs.
> > 
> 
> This should be fairly simple, IMHO: each job we have will have to
> copied for
> each new repo.
> 
> So instead of having rhq-master-compile, we're going to have to have:
> rhq-core-master-compile
> rhq-helper-master-compile
> rhq-plugins-master-compile

If we do this, then we would need to re-work the maven repository isolation method that we currently have in place. Today every single job has it's own local maven repository. The separation was implemented because we have a total of 7 cascading Jenkins builds on the same branch (and most of the times two or three of them run concurrently). If these jobs were to use some shared repository they would step on each other's toes. We had huge stability issues in the past because two jobs were deleting/updating artifacts at the same time.

The other challenge that I see with splitting the current jobs into three jobs is the explosion on the number of Jenkins jobs. Today we have a chain of 7 jobs chained for master. Multiply that with 3 and we would need to monitor, update, and fix about 21 jobs. If we have to maintain builds for two branches that would be a total of 42 jobs. Think now about the trivial task of updating the maven version, it will become non-trivial and error prone.

>From my perspective the only way to monitor a chain of about 21 jobs is with the help of a build pipeline. Currently we do not have that capability implemented in Jenkins (... even though I requested it about 6 months ago). And Jenkins support is rudimentary at best when it comes to build pipelines anyway.


I know the hope was to split the current jobs into smaller, nimbler jobs. We can still probably achieve this, but we need to think it through more and 3x factor is not easy. The main limiting factors here are the build system and maven.


> 
> The good thing is that those jobs are going to be independent and
> will not
> require each other's build artifacts, because the policy we are
> adopting is
> that only published modules from the other repos are admissible as
> deps.

I am not sure how that will work if you have to fix a bug in a plugin that also requires a change in the core code. I had at least a few of those while working on the content system or the AS7 plugin.

What would the workflow be for a developer? What about creating a build with Jenkins? 

> 
> One thing that WILL have to change rather substantially is the API
> checking,
> but that's not directly related to Jenkins, it's a build issue. The
> API checks
> now assume a uniform version of all RHQ modules, which is no longer
> going to
> be true.
> 
> > Regards
> > Thomas
> > _______________________________________________
> > rhq-devel mailing list
> > rhq-devel at lists.fedorahosted.org
> > https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
> 


More information about the rhq-devel mailing list