[SSSD] [RFC] Add basic support for CI test execution

Lukas Slebodnik lslebodn at redhat.com
Fri Jun 13 22:21:41 UTC 2014


On (13/06/14 23:02), Nikolai Kondrashov wrote:
>Sorry, I lost the whole day to this thread already and am quite tired, so my
>answers will be brief.
>
>On 06/13/2014 09:58 PM, Lukas Slebodnik wrote:
>>On (13/06/14 20:51), Nikolai Kondrashov wrote:
>>>On 06/13/2014 05:50 PM, Lukas Slebodnik wrote:
>>>>On (13/06/14 16:49), Nikolai Kondrashov wrote:
>>>>>On 06/13/2014 03:14 PM, Lukas Slebodnik wrote:
>>>>>>On (13/06/14 14:50), Nikolai Kondrashov wrote:
>>>>>
>>>>>CI runs configure anyway, so "make srpm" would be better as it's slightly more
>>>>>reliable compared to make_srpm.sh which duplicates substitution code.
>>>>>
>>>>You need to execute configrure for "make srpm"
>>>>thus you need to have installed all dependencies
>>>>and we are back to the problem with maintaining build depdendencies for fedora&rhel
>>>
>>>Sure, but we need to have the dependencies installed anyway (no matter how the
>>>information about them is retrieved) before we start the other tests.
>>You do not need to have installed all sssd build dependencies for creating
>>src.rpm with script contrib/fedora/make_srpm.sh. It is not really true
>>that you need to have installed all dependencies for all test.
>>Some static analysers don't depend on build dependencies. They use just
>>pattern matching. Neither does mock.
>
>Sorry, you lost me here. What are you suggesting exactly?
>
I meant that not all test require to have all build dependencies installed.

>>>>make_srpm.sh is reliable. It is used by Kevin Wright to rebuild srpm
>>>>on rhel6 and he caught few problems in last months.
>>>
>>>It is reliable towards repeated execution,
>>>but not so much towards repeated change of the other sources it depends on.
>>Sorry, I do not understand.
>>What kind of "repeated change of the other sources"
>
>The kind that happens during normal development.
>
It didn't help. I am still not sure what do you mean.

>>>>>Mock seems the right thing to do in general, but not on the CI server, since
>>>>>the latter already has builders with various distributions and installs
>>>>>dependencies itself. Using it there would duplicate the functionality and
>>>>>load to some extent and it wouldn't make sense to run it on each of the
>>>>>builders.
>>>>>
>>>>I proposed mock because it will not be difficult to integrate it. As you
>>>>could see, it is a simple command and configuration of mock is very easy.
>>>>With CI, you cannot guarantee minimal set of build dependencies.  On VM with
>>>>CI, leftovers from previous builds can be installed and it can hide some
>>>>problems.
>>>
>>>Yes, I understand that.
>>>
>>>>Or do you want to start new clean VM every time?
>>>>It would be overkill from my POV.
>>>
>>>As I understand it, the ultimate plan decided by the management is to use
>>>OpenStack or some such to provision the VMs. Still, this is not going to
>>>happen fast.
>>>
>>This discussion has nothing common with management.
>
>You mean we don't have to listen to our management?
I am sorry, but this is irrelevant. I wrote That I do not care about location
of VM. It is not related to the discussion about build dependencies.

>>It is a upstream discussion. I do not care about location of VM for CI.  It
>>can be in openstack, EC2 or azure. My question was whether you will create
>>clean VM for each test set in CI. If answer is yes than using mock will not
>>have big benefit.
>
>The answer is if management insists on using fresh VMs for every build, then
>that's what we'll have to do. I'm inclined to think that might be a good idea,
>if provisioning is fast, but am afraid that the framework for that would be
>too complex for our purposes, so am not hurrying to implement it yet.
>
It is not about decision from management. It is decision of developer
what is suitable for this purpose.

>>>Until then, I think a dedicated builder doing just mock builds for various
>>>distros would do. We can trigger it together with the distro-specific
>>>builders. I just don't want to start writing another script right now, I want
>>>to deliver this one first.
>>>
>>We do not need to ship rpms created by mock into yum repo. It would be used
>>just for testing build.
>
>I'm sorry I don't see where I suggested anything about a yum repo.
>
>>>>mock and rpm are fedora/rhel specific. I am not sure if it works with opensuse.
>>>>They have Open Build Service. On the other side, we needn't test each
>>>>distribution in CI from beginning. Fedora, rhel, debian are good start.
>>>
>>>Well, if it works on OpenSuse and we decide we need to test it, then we'll use
>>>it.
>>It is far future ant it isn't a priority.
>
>This is basically what I said.
>
>>>>>I would say that building directly on the builder distro will be good enough
>>>>>for a start. Later we can replace it with a separate script and a separate
>>>>>builder for running mock builds with various distros.
>>>>I am not against directly building sssd on distro. It is faster, because you
>>>>will need to compile sssd few times (scan-build, with gcc for valgrind and
>>>>others otions in future). Building with mock can be additional test and
>>>>it needn't be part of reduced test set.
>>>
>>>The thing is, I would like to have one script that could be ran both locally
>>>and on CI, with obvious benefits. However having mock there and then having
>>>every Red Hat distro builder run it doesn't make sense, as the mock
>>>environment and the results will be the same. That's what mock is for, as you
>>>know.
>>>
>>>So, I'd like to have a separate script for mock, which will run on a separate
>>>builder and will build packages for all the distros we need, once for each
>>>job.
>>>
>>In my opinion, it can be in the same script. You already proposed to execute
>>different tests with full version and reduced version. There can be another
>>3rd level, which will execute all test including test with mock.
>
>If we put mock on the lower levels, then all builders will have to execute it.
>If we put it on the top layer, then the dedicated builder will have to execute
>all other tests. Alternatively, we can just make one of the regular builders
>combine them, but that will be confusing and would make mock tests
>depend on the previous regular ones not aborting the script.
>
It does not bother me. I am fine with both. I just want to to have possibility
to test build in mock. I use it very often in my workflow and it helped me many
times to catch problems. It can be optional.

>I don't want to complicate the current script further by inventing something
>else above two sets of tests. If using Epoxy works out, it would allow
>executing arbitrary tests and we'll be able to put mock there. For now I think
>building directly on the distro will be good enough. We don't have to
>implement everything at once.
I am not sure what is Epoxy. My google foo didn't help me. I do not see
benefit in script which cannot be easily extensible.
(or it can be but you have a reason to do not extend it).
Your script can be used in temporary version of CI and probably is.
If we want to have script in upstream I will prefer to have easily extensible
version. If Epoxy is the right approach I will rather wait for next version.

LS



More information about the sssd-devel mailing list