Tests playbooks accessing other files in dist-git repo
by Stef Walter
Jonathan Lebon got the first standard test (that I know of) into
dist-git. Woooo!
The rpm-ostree Ansible wrapper invokes tests are packaged in the source
RPM upstream.
In order to do this it needs to be able to assume that the tests are
invoked in an actual dist-git repo, and it can use the spec file to
access the sources.
I always assumed the spec [0] implied that the tests should be executed
in an actual dist-git repo checkout, but Jonathan pointed out that this
isn't completely clear.
I'd like to edit the spec to clear this up. I'd like to insert an item
after "5." in the list here:
https://fedoraproject.org/wiki/Changes/InvokingTests#Invocation
with:
6. The playbook must be executed within the dist-git repository
checkout and not copied elsewhere.
- The playbook may refer to any other file in the dist-git checkout
or its tests/ directory via a relative path.
Does that make sense?
Cheers,
Stef
6 years, 3 months
How to opt-in to gating by the CI pipeline
by Stef Walter
I believe we can have packages opt in on the CI pipeline like this.
Pingou, Ari and I discussed this a bit:
* A package with a tests/ directory according to the spec will
have its tests invoked according to the spec. If those tests fail
the package may be gated.
In other words the presence of tests in dist-git will be treated as an
acknowledgement from the dist-git repo that it is okay to gate the
package based on those tests.
As posted here last week, certain tests are appropriate for certain test
contexts. Certain tests are appropriate for containers, other for Atomic
Host systems, and others for classic installed systems.
At the current time the CI pipeline [0] only tests packages and tests
marked for Atomic Host, but in the future that scope can grow. This
initial scope would obviously also apply to gating.
Does that make sense? Any objections?
Cheers,
Stef
6 years, 3 months
Can't push to upstreamfirst.fic.org
by Jonathan Lebon
Hi all,
I seem unable to push commits to my repo:
[rpm-ostree master]$ git push
FATAL: W any rpm-ostree jlebon DENIED by fallthru
(or you mis-spelled the reponame)
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
[rpm-ostree master]$ cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
url = ssh://git@upstreamfirst.fedorainfracloud.org/rpm-ostree.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
There is another similar thread happening on devel[0] though
I'm posting here since that one seems to be about src.fp.o.
[0] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
6 years, 3 months
Re: [atomic-devel] Meeting item for 20170802: Atomic Host +
Modularity - Enabling CI/CD
by Dusty Mabe
On 08/03/2017 10:23 AM, Josh Boyer wrote:
> On Thu, Aug 3, 2017 at 10:16 AM, Ari LiVigni <ari(a)redhat.com> wrote:
>>
>>
>>>
>> I assume the CI/CD pipeline you refer to is the one upstream CI (my team) is
>> working on?
>> Otherwise this seems to be a duplication since we are already doing this
>> work.
>
> Dusty would have to answer that. I was just talking about CI/CD in
> general, but I think the specific pipeline Dusty is looking at is
> indeed the one your team is working on.
>
Indeed, assuming Ari is a part of the community behind the CI efforts in Fedora
as linked in the first email I sent: https://fedoraproject.org/wiki/CI
I think Stef is also a part of that effort and he has presented the pipeline to
the Atomic WG as something beneficial for Atomic Host, at least as an initial
smaller scope POC before introducing CI to the rest of Fedora's offerings.
Dusty
6 years, 4 months
Re: [atomic-devel] Meeting item for 20170802: Atomic Host +
Modularity - Enabling CI/CD
by Dusty Mabe
On 08/03/2017 08:32 AM, Josh Boyer wrote:
> On Wed, Aug 2, 2017 at 11:15 AM, Dusty Mabe <dusty(a)dustymabe.com> wrote:
>> Recently we talked about adding more CI/CD to the Atomic Host
>> processes. Since we release every two weeks we'd like to automatically
>> run tests on every change that goes in to Atomic Host and prevent
>> it from going into Atomic Host if it causes test failures.
>>
>> This was suggested to the Atomic Working Group from the new CI Working
>> Group [1] in the 2017-07-12 meeting [2]. Stef, from the CI working
>> group also sent a mail to the cloud list about it [3].
>>
>> Here are some of the things we want to be able to do:
>>
>> 1 Have a rigorous definition (including specific versions, buildroot) of
>> what goes into an Atomic Host, including dependencies.
>>
>> 2 Triggering the CI/CD pipeline based on a change to definition
>> of what goes into Atomic Host.
>>
>> 3 A way to revert a package in the Atomic Host compose to an earlier version.
>>
>> 4 A place to store higher level tests along with rigorous definition
>> of Atomic Host, including them being versioned.
>>
>> 5 Landing multiple changes that need to land together to pass testing.
>>
>>
>> Out of that list we think we require that:
>>
>> 1R The definition of what is composed into Atomic Host artifacts should
>> include specific versions of packages, and all dependencies included.
>
> Is this an output of or input to the compose? I ask because
> dependencies will change, so assuming they are static would be bad.
I believe we want this to be an input, but an input that can be updated by
bots. i.e. a new version of rpmxyz is available and a bot opens a PR to
add it to the definition of atomic host and tests run on that PR. This is
not set in stone, however. Definitely looking from input here, especially
from people who know more about modularity.
> They can also vary by architecture.
This is something that was previously glossed over in our discussions. Thanks
for bringing it up. Does modularity handle this at all today?
>
>> 2R The definition of what is composed in an Atomic Host should be stored
>> in a git repository so that changes can be detected easily. The
>> CI/CD pipeline can be triggered off of changes to this reposiroty.
>
> If the answer to the above question is "output of", then this item is
> essentially a package manifest for everything in the compose, correct?
Answer was 'input of'. We can debate on whether that is a good or a bad thing :)
>
> josh
>
>> 3R A mechanism to make a future composed Atomic Host artifact, contain
>> an earlier (in RPM NVR parlance) version of a package.
>>
>> 4R The high level functional Atomic Host tests should live in the same
>> git repository with the rigorous definition of what goes into an Atomic
>> Host.
>>
>> 5R A mechanism to tell the CI Pipeline that multiple dist-git repository
>> changes (i.e. multiple changing RPMs) should be built and tested together.
>>
>> We believe that moving to an Atomic Host defined by a module [4] can
>> satisfy our needs and enable the improved control of artifacts within
>> Atomic Host as well as the improved triggering and coupling with a
>> testing pipeline that will help us move forward. Since modularity is new
>> this will take some experimentation to prove out. Let's chat about this
>> in the Atomic Working Group meeting today at 1700UTC in #fedora-meeting-1.
>>
>> Dusty
>>
>>
>> [1] https://fedoraproject.org/wiki/CI
>> [2] https://meetbot.fedoraproject.org/fedora-meeting-1/2017-07-12/fedora_atom...
>> [3] https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.o...
>> [4] https://docs.pagure.org/modularity/
>>
> _______________________________________________
> Atomic Working Group mailing list -- atomic(a)lists.fedoraproject.org
> To unsubscribe send an email to atomic-leave(a)lists.fedoraproject.org
>
6 years, 4 months
About Bodhi integration with Pipeline (modules, koji, etc).
by Stef Walter
Hi bowlofeggs, I saw this on IRC after I had left for the evening. I
hope it's okay to answer on the mailing list.
These are good points, and solving them depends on work by many other
parties.
> bowlofeggs said:
> I can't stop thinking about something that i think doesn't make sense> to me about the current design around the CI pipeline: why are we
> connecting commit hash based tests to bodhi? by the time you've gotten
> to bodhi two problems leap to mind:>
> 0) the build being shipped isn't the same as the build tested in the CI
> pipeline (even if the sources are the same, the buildroot isn't
> guaranteed to have been)
Isn't Bodhi looking up the results for the exact NVR in Koji?
If so, then this is something that needs to be reconciled in the pipeline.
The pipeline folks are working towards registering the builds with Koji
and/or doing the build in Koji. That means exactly the RPM which is
built is the one which will be published by Bodhi.
Ditto for modules, MBS, and module based artifacts such as qcow2 Atomic
Host images.
> 1) commit results should be reported to the developer when > they make the commit (or when they propose the commit to be merged in
pagure)
Indeed, we should also report commit results there. That's future work
to be done. In addition, modules come into play here ... more on that below.
> <bowlofeggs> there's also a third problem: the CI pipeline didn't
> test the list of builds in the bodhi update, but it instead tested > individual builds - builds frequently pass or fail in combination with
others
Yup, definitely.
Keep in mind that even if they pass or fail is combination with those
specific others, there's still no guarantee that the end user doesn't
mix and match things in a way that causes the whole thing to fail
dramatically on their system.
This is where modules come into the picture. Modules complete this
story. A module represents all the software that is guaranteed to work
together, tested together, installed together.
The pipeline folks are looking to integrate with the modularity work in
Fedora so that we can test known sets of software together with known
buildroots, and gate them together.
In addition the Atomic Host folks are looking to define Atomic Host from
a module.
> <bowlofeggs> from my perspective, i think it makes more sense to run> bodhi gating tests in response to the creation of bodhi updates
> and it makes sense to write PR gating tests in response to the > creation of pagure PRs
> but, gating bodhi tests in response to the creation of pagure PRs
>causes problems, and i think also doesn't solve the intended problems
These would *definitely* be the same tests, but you're right they are
running in a different context.
So what we're trying to do is avoid having to boil the ocean before
getting started, and actually start to get feedback to people who are
porting the tests.
Once all the modularity work lands in Bodhi, pipeline, module level
tests, module definitions, arbitrary branching ... we will converge on a
workable and complete solution. However until then, would it work to
report results and gate on packages in Bodhi in a more limited but
workable fashion?
Does Bodhi have what it needs today to report results present in
ResultsDB for an update. If so, then overtime those results will get
better and better (see above, work on integrating with Koji, MBS,
modules etc.).
The most important missing ingredient today is missing display of
results and gating of the pipeline for packages that opt-into the system.
Without that we don't have forward progress on tests or a myriad of
other aspects of this work.
Cheers,
Stef
6 years, 4 months
Meeting item for 20170802: Atomic Host + Modularity - Enabling CI/CD
by Dusty Mabe
Recently we talked about adding more CI/CD to the Atomic Host
processes. Since we release every two weeks we'd like to automatically
run tests on every change that goes in to Atomic Host and prevent
it from going into Atomic Host if it causes test failures.
This was suggested to the Atomic Working Group from the new CI Working
Group [1] in the 2017-07-12 meeting [2]. Stef, from the CI working
group also sent a mail to the cloud list about it [3].
Here are some of the things we want to be able to do:
1 Have a rigorous definition (including specific versions, buildroot) of
what goes into an Atomic Host, including dependencies.
2 Triggering the CI/CD pipeline based on a change to definition
of what goes into Atomic Host.
3 A way to revert a package in the Atomic Host compose to an earlier version.
4 A place to store higher level tests along with rigorous definition
of Atomic Host, including them being versioned.
5 Landing multiple changes that need to land together to pass testing.
Out of that list we think we require that:
1R The definition of what is composed into Atomic Host artifacts should
include specific versions of packages, and all dependencies included.
2R The definition of what is composed in an Atomic Host should be stored
in a git repository so that changes can be detected easily. The
CI/CD pipeline can be triggered off of changes to this reposiroty.
3R A mechanism to make a future composed Atomic Host artifact, contain
an earlier (in RPM NVR parlance) version of a package.
4R The high level functional Atomic Host tests should live in the same
git repository with the rigorous definition of what goes into an Atomic
Host.
5R A mechanism to tell the CI Pipeline that multiple dist-git repository
changes (i.e. multiple changing RPMs) should be built and tested together.
We believe that moving to an Atomic Host defined by a module [4] can
satisfy our needs and enable the improved control of artifacts within
Atomic Host as well as the improved triggering and coupling with a
testing pipeline that will help us move forward. Since modularity is new
this will take some experimentation to prove out. Let's chat about this
in the Atomic Working Group meeting today at 1700UTC in #fedora-meeting-1.
Dusty
[1] https://fedoraproject.org/wiki/CI
[2] https://meetbot.fedoraproject.org/fedora-meeting-1/2017-07-12/fedora_atom...
[3] https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.o...
[4] https://docs.pagure.org/modularity/
6 years, 4 months