[SSSD] [PATCH v5] Add basic support for CI test execution

Nikolai Kondrashov Nikolai.Kondrashov at redhat.com
Mon Sep 1 17:44:30 UTC 2014


On 09/01/2014 07:48 PM, Michal Židek wrote:
>> My response and plan here would be similar to the Valgrind's case. Only
>> here we may perhaps disable certain error classes for the start, or use
>> Clang augmentation in the code, making ignore the difficult issues, but in
>> the end we should arrive to the same binary status.
>>
>
> About Valgrind and Clang, I think you underestimate the amount of time
> that is in front of us before we get to the binary PASS/FAIL state.

Perhaps, but I'd like to try it. If it turns out so, with Valgrind we can
still use suppressions and I'll try finding something similar for Clang.

> Until then, these checks are completely useless and only take time (which is
> one of the reasons why I wanted to be able to disable/enable specific
> steps).

As automatic checks they are useless, but they're still providing an easy way
to run Valgrind and Clang to help people fix those issues.

Yes, I understand why you'd want to skip them and I expect there will be more
tests people would like to skip or run alone. Perhaps we can simply move both
Valgrind and Clang to the "rigorous" test set for now.

> The number would tell us if something changed from the past run or not.
> (and would make these steps less useless). It would be a valuable
> improvement IMO.

It could be, even though we won't be able to tell if one failure disappeared
and another one appeared. But I'd like to try for something better first.

> Also I do not think disabling specific category of test in static
> analyser is a good idea.

Yes, it might be dangerous. Perhaps a failure count will be safer than that.

BTW, did you notice how coverage percentage is verified? What do you think
about that?

>>> 3. More options:
>>> I would like to have possibility to disable/enable specific parts.
>>> The -e/-m/-r flags are IMO not sufficient (for example as long as we
>>> do not fix 1 and 2, I would like to disable/not enable these parts
>>> using some argument)
>>
>> I would like that as well. That's why I would like to try using Epoxy - my
>> Bash test framework (https://github.com/spbnick/epoxy), which allows
>> just that
>> and more.
>
> I am not familiar with epoxy, but Having better framework for shell script
> based tests would be a nice thing. But I would like to stick to programming
> languages our project already depends on (python/C/shell). I see that epoxy
> depends on Lua. Python framework would be better for us IMO.

The framework itself is written in Bash, only extended glob matching and some
log processing is done in Lua. Lua knowledge is not needed when writing test
suites. I don't think it will be very hard to rewrite that in Python, if
necessary.

Is it a problem of an extra dependency, or understanding source code?

If it's the former, then I think can fix it. If it's the latter, perhaps it
doesn't need to be fixed as Lua parts are non-core logic and are clearly
separated?

Nick



More information about the sssd-devel mailing list