On Thursday, February 28, 2013 16:53:33 Stefan Negrea wrote:
----- Original Message -----
> From: "Lukas Krejci" <lkrejci(a)redhat.com>
> To: rhq-devel(a)lists.fedorahosted.org
> Sent: Wednesday, February 27, 2013 10:37:12 AM
> Subject: Re: GitHub migration again -- The Plan
>
... snip ...
> > Splitting the repository will have important implications
on CI
> > side and
> > we might also need a plan for Jenkins jobs.
>
> This should be fairly simple, IMHO: each job we have will have to
> copied for
> each new repo.
>
> So instead of having rhq-master-compile, we're going to have to have:
> rhq-core-master-compile
> rhq-helper-master-compile
> rhq-plugins-master-compile
If we do this, then we would need to re-work the maven repository isolation
method that we currently have in place. Today every single job has it's own
local maven repository. The separation was implemented because we have a
total of 7 cascading Jenkins builds on the same branch (and most of the
times two or three of them run concurrently). If these jobs were to use
some shared repository they would step on each other's toes. We had huge
stability issues in the past because two jobs were deleting/updating
artifacts at the same time.
This is why I think it is important to be strict about only depending on
released and published core-artifacts in non-core modules. That way you don't
need a shared repo and can keep a repo-per-job as we do now.
The other challenge that I see with splitting the current jobs into
three
jobs is the explosion on the number of Jenkins jobs. Today we have a chain
of 7 jobs chained for master. Multiply that with 3 and we would need to
monitor, update, and fix about 21 jobs. If we have to maintain builds for
two branches that would be a total of 42 jobs. Think now about the trivial
task of updating the maven version, it will become non-trivial and error
prone.
From my perspective the only way to monitor a chain of about 21 jobs is with
the help of a build pipeline. Currently we do not have that capability
implemented in Jenkins (... even though I requested it about 6 months ago).
And Jenkins support is rudimentary at best when it comes to build pipelines
anyway.
I know the hope was to split the current jobs into smaller, nimbler jobs. We
can still probably achieve this, but we need to think it through more and
3x factor is not easy. The main limiting factors here are the build system
and maven.
I don't think we need to multiply all the jobs by 3. For example, we have a
job running the tests against an Oracle database. Running a plugins build
against an Oracle database just doesn't make sense, because the plugin tests
don't interact with the database (maybe barring the Oracle plugin's
integration tests if there are any - but itests are another whole story
orthogonal to this discussion). But generally you concern is very valid - the
number of builds will increase and we will have a harder job maintaining them.
I don't think that's necessarily a bad thing though. By the split, we're just
"formally" admitting that we maintain 3 separate projects - RHQ itself,
plugins for it and then a bunch of helpers that come in handy from time to
time.
> The good thing is that those jobs are going to be independent
and
> will not
> require each other's build artifacts, because the policy we are
> adopting is
> that only published modules from the other repos are admissible as
> deps.
I am not sure how that will work if you have to fix a bug in a plugin that
also requires a change in the core code. I had at least a few of those
while working on the content system or the AS7 plugin.
Heiko already mentioned it and I whole-heartedly agree with him. If you need a
fix in a plugin that also requires a change in core, do the change in core
first, publish it as a new incremental version and only then change the
plugin. Basically we need to start thinking about the core server as a
standalone project.
What would the workflow be for a developer? What about creating a
build with
Jenkins?
Locally, you can do both the changes together, but when committing to the
repo, you need to split the fix in two phases as outlined above.
While this is more cumbersome for the devs, I just can't see another way of
providing a disciplined and usable API for the plugin developers in the
community. If a 3rd party developer wants to develop a plugin for RHQ.Next,
they are essentially out of luck today because they need build against
snapshots which will force them to build the whole of RHQ from source just for
the sake of developing the plugin.
While this change will slow down the progress somewhat, I think it's worth it
- we will have more fine grained micro releases that will actually track the
progress in the core.
> One thing that WILL have to change rather substantially is the
API
> checking,
> but that's not directly related to Jenkins, it's a build issue. The
> API checks
> now assume a uniform version of all RHQ modules, which is no longer
> going to
> be true.
>
> > Regards
> > Thomas
> > _______________________________________________
> > rhq-devel mailing list
> > rhq-devel(a)lists.fedorahosted.org
> >
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel(a)lists.fedorahosted.org
>
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel