[SSSD] [PATCH 1/1] tests: Don't assume absolute test dir in test_io.c
Jakub Hrozek
jhrozek at redhat.com
Mon May 19 09:16:33 UTC 2014
On Mon, May 19, 2014 at 09:33:49AM +0200, Lukas Slebodnik wrote:
> On (17/05/14 01:14), Nikolai Kondrashov wrote:
> >On 05/16/2014 11:32 PM, Lukas Slebodnik wrote:
> >>On (16/04/14 21:30), Nikolai Kondrashov wrote:
> >>>In my case there is no beneficial performance difference when running with
> >>>valgrind. There is about 25% improvement using /dev/shm when running with
> >>>coverage enabled, but even in a VM it takes less than a minute on my machine.
> >>>
> >>I was waiting for pshing patch to the upstream.
> >>This is a reason of late reply.
> >>
> >>>I will use /dev/shm and will mess with moving test results around, when/if
> >>>I'll need to tune the performance, but for now I'd prefer to keep my CI code
> >>>simple and use relative directories.
> >>>
> >>I have totally different results. The most problematic is sysdb test,
> >>because there is a lof of IO operations.
> >
> It might look I was exaggerating in my previous mail.
>
> There are results of execution of all tests.
> * tests were precompiled (make tests)
> * I executed tests with "make -j4 check", because I have 4-core CPU
>
> ramdisk:
> time make -j4 check V=0
> // snip
> real 0m12.462s
> user 0m4.887s
> sys 0m0.812s
>
> HDD:
> time make -j4 check V=0
> // snip
> real 9m44.722s
> user 0m11.920s
> sys 0m8.038s
>
> "ramdisk" is 58x faster then HDD.
> 3 tests failed on HDD due to timeout.
> src/tests/sysdb-tests.c:1399:E:SYSDB Tests:test_sysdb_get_new_id:0: (after this
> point) Test timeout expired
>
> src/tests/sysdb_ssh-tests.c:202:E:SYSDB_SSH Tests:store_one_host_test:0: (after
> this point) Test timeout expired
>
> src/tests/simple_access-tests.c:122:S:user allow/deny:test_both_empty:0: (after
> this point) Test timeout expired
>
> The next time, I executed test withour running tests simultaneously: make check
> ramdisk:
> time make check V=0
> // snip
> real 0m25.722s
> user 0m4.573s
> sys 0m0.785s
>
> HDD:
> time make check V=0
> // snip
> real 11m3.424s
> user 0m9.509s
> sys 0m7.085s
>
> "ramdisk" is 26x faster then HDD.
>
> >I admit I didn't test this thoroughly and might have hit a corner case and
> >perhaps I was biased. I'll retry my tests properly once I get to the point of
> >optimizing performance.
> Running tests in ramdisk is necessary for developers to pass tests
> in reasonable time. It is not optimization, it is well known fact
> and I will appreciate if continous integration is aware of this fact.
I wonder if we should raise the CK_TIMEOUT variable (or its equivalent)
in the tests? I'm not sure if requiring the tests to be ran in ramdisk
is the right thing to do, mostly because --with-test-dir only defaults
to shm in our specfile, not the source.
So I think we should either set the --with-test-dir value to ramdisk by
default (we already set it this way for distcheck) or raise the
CK_TIMEOUT.
More information about the sssd-devel
mailing list