On 12/19/2014 01:34 AM, Lukas Slebodnik wrote:
On (17/12/14 20:06), Nikolai Kondrashov wrote:
> To make that work we need the LDAP server to work with arbitrary configuration
> and data locations (in our case under the build tree). OpenLDAP can do that,
> with locations specified at startup time. Not sure about 389-ds.
OpenLDAP is good enough for cwrap test. 389-ds can be used for real
integration testing. Do you have a plan to include kdc to cwrap tests?
It can be implemented in 2nd round of cwrap tests.
I have the intention, but I don't have a plan how to do it yet :)
Everything sounds good but I was more interested about details
cwrap tests. Will it be just one test or more tests?
Because make uses "Parallel Test Harness"  and there can be problem with
shared ldap server.
FWIW, we can arrange several OpenLDAP instances, relatively easily. What will
be more of a problem is the hard-coded (for now) sssd configuration, data and
socket locations. For that reason we can only have one Automake-level test
suite. However, we can probably employ pytest-level parallelization for some
tests which happen to be independent.
How will be done setup/teardown of OpenLDAP/SSSD (pytest?)
For the start OpenLDAP setup/teardown will be done in two shell scripts, which
will be invoked from the proper pytest harness setup/teardown. The reason is
the OpenLDAP setup is lengthy, not entirely clear yet, and involves shuffling
files around and invoking and possibly interpreting program I/O, which shell
excels at. Once we have figured out all the pieces, we can re-interpret it in
Python for a bit of added reliability. For now shell will let us start faster.
> We also need all sssd components involved in the tests (at least
> and sssd) to be able to work with arbitrary configuration, data and socket
> locations (under the build tree, again). At the moment we can change these
> only at the build time, via "configure" invocation.
> We can probably make the "check" target depend on an additional build with
> changed prefix, but it feels too much of a hassle and a slowdown for the RPM
> build. Making "contrib/ci/run" do the special-prefix build instead seems
> easier and simpler.
The initial configure could be executed with custom prefix in CI
but it would not work for developer use case. It might not be a solution.
Even that extra configure has some overhead it seems to be the fastest &
easiest way how to implement cwrap test. It can be improved in future.
On my machine, it took 20 seconds (configure && make install).
I think we can live with this and my expectation is that tests will take
If we want to "limit" overhead for regular usage then we can separate
unit tests and cwrap test. Unit tests will be executed by default with
"make check" and cwrap test only in case of enabled special variable
"make check CWRAP=yes" (I dind't test it) or we can use different make
target "make cwrap-check"
Some ideas can be modified. It depends on answer about organization of setup,
Thinking about this a little, what if we have the integration tests be a part
of "make check" but skip themselves (return 77 to Automake), if they don't
have write permissions for all the locations they need?
We can generate a Python or a shell module during configure with all the
relevant paths substituted (if that's not done yet), or have some of the built
C programs (sssd, utilities?) output them, perhaps with some options. Then,
when the cwrap test suite is run by Automake, we can check for permissions and
exit with 77, if we don't get them. We'll need to translate pytest exit
codes to what Automake expects anyway, BTW.
Next, we can have contrib/ci/run create the relevant temporary directories and
use special prefixes for both its debug and coverage builds (i.e. not have a
separate build for this), which would allow the cwrap tests to run.
Building manually with writeable prefix, or running "make check" as root,
would probably work as well.
Then, when (and if) we get startup-time / environment variable specification
of all these locations, we can simply remove that condition from the test
suite, remove prefix specification from contrib/ci/run, and move temporary
directory creation to the suite itself.
What do you think?