On 06/11/2014 08:14 PM, Pavel Reichl wrote:
On Wed, 2014-06-11 at 19:34 +0300, Nikolai Kondrashov wrote:
> On 06/11/2014 05:54 PM, Pavel Reichl wrote:
>> On Wed, 2014-06-11 at 17:28 +0300, Nikolai Kondrashov wrote:
>>>
>>>>> Sure, there are various ways to make Valgrind happy, although none of
them are
>>>>> very easy. The important thing is that we would have a way to track
it now.
>>>>> This way, or another we should arrive to zero Valgrind errors.
>>>>>
>>>> Ignoring is not a good way for CI. We need to find a viable solution.
>>>
>>> I agree and I'm not saying we should ignore it always. This ignoring is
to
>>> enable CI earlier, so we can get at least *some* tests to run. And having CI
>>> running these tests (even though they're ignored) will make fixing the
>>> failures easier, than without the CI. If only for the ability to trigger
>>> background runs on various distros at once. Once we have the issues fixed we
>>> will enforce the check.
>>>
>>
>> Is it viable option to somehow "store" current problems found by
>> valgrind/clang/other tools and just report the new ones?
>>
>> Doing so would allow us to use all benefits of this tools without need
>> to promptly fix issues created over years and we could use this approach
>> to mask false positives.
>
> We can use Valgrind suppressions to achieve something close to this. Clang
> doesn't seem to have such facility, but it is probably possible to implement
> one which would process resulting XML files.
>
> However, I think that effort to do this is much greater than the effort
> required to bring errors/warnings to zero, using various exception mechanisms
> where necessary, and then simply keeping it there.
I haven't looked closely on this so I trust your opinion.
Thank you. I wouldn't trust myself on this much, though - it's no more than a
hunch at this moment :)
> One thing that would probably need this is code coverage
tracking. I'm
> considering storing at least the overall score somewhere and comparing each
> run to the previous commit, alerting on reduction. However, I wouldn't attempt
> to implement this right now, as it is mostly a convenience. Perhaps a few
> months later.
That sounds great. BTW I really like how convenient it's now to get
coverage results thanks to your script.
Actually, I think that a good approach would simply be to state the required
score in the CI script itself and fail if it's not achieved, then just update
it from time to time as a higher score is reached.
It would be easy to implement.
Nick