On Tue, Nov 1, 2016 at 10:36 AM, Jakub Hrozek
<jhrozek(a)redhat.com> wrote:
> On Tue, Nov 01, 2016 at 10:31:20AM +0200, Nikolai Kondrashov wrote:
>> Hi Jakub,
>>
>> On 10/27/2016 05:20 PM, Jakub Hrozek wrote:
>> > I'm currently working on integration tests for the 'files'
provider and
>> > during this work I started to feel we are pushing the boundaries around
our
>> > test infrastructure already quite a bit. When SSSD talks over network to
>> > a server, then we're more or less okay, but for some parts of SSSD,
like
>> > the files provider, we have to mock a lot of pieces and the end result
>> > is that we are testing something that resembles the target system, but
>> > probably has its own bugs. Additionally, we can't run some tests at
all
>> > (anything against IPA) and I suspect we'll be running into this sort
of
>> > a problem even more in the future.
>> >
>> > So I'm interested in hearing what are other's thoughts on exploring
how
>> > to run some of our tests in a privileged environment, either in a VM or
>> > in a container?
>> >
>> > Our current tests have the big advantage of being able to provision a
>> > test locally in a screen session, but maybe something similar would be
>> > possible by e.g. running a screen in a container and attaching to its
>> > tty.. And for "simple" tests like LDAP provider we could keep the
current
>> > infrastructure around.
>>
>> I agree that the current test setup is limiting, can be difficult to work
>> with, and bring its own issues. However I believe we haven't employed it
>> fully yet, and I'm not sure how spending time on another test system
compares
>> to just writing more tests (e.g. for Samba and just general tests).
>
> Yes, many tests can still continue exploring the wrapped scenario. I
> think I will be perfectly happy writing tests for the KCM service using a
> cwrapped KDC. If your tests talks to a server, then as long as the server
> is running and you're talking to it over socket_wrapper, you're fine.
>
> But to give a concrete example, here are the issues I ran into with the
> files provider tests:
> - in production, the files provider dlopens libnss_files.so and
> calls its functions. I had to wrap a different module for tests
> that calls nss_wrapped functions instead to reach nss_wrapper's
> passwd and groups. This is just additional work, but I wouldn't have
> to do it in a VM/container.
> - more importantly, one part of testing the files provider
> is detecting the changes the admin does to /etc/passwd and
> /etc/groups. In tests, I wrote a simple module that modifies
> nss_wrappers' passwd and group files, but then I need to take care
> so that my module changes the files using the same modifications
> (create a new file and replace the old one with it, not modify
> the old file in-place) otherwise the files provider was receiving
> different inotify callbacks. In a container or a VM, I would just
> call the shadow-utils binaries.
>
> Other examples of things we can't test easily with wrappers are D-Bus
> system bus interfaces (these wouldn't be too hard to wrap, though,
> 'just' additional work) and anything that involves an IPA server.
>
>>
>> Regardless, I was casually thinking about this for a while and have this to
>> say. I think being able to run tests locally, on your machine, easily is very
>> important. It is also important to let outside developers run those tests with
>> minimal setup.
>
> Yes, I really like being able to spin up a screen session and work on my
> test there. But what I'm worried about is that (see the examples about
> files provider) by writings wrappers and helpers for my tests so that
> they run in a cwrapped environment is actually testing something
> different than what the real system would do.
>
>>
>> The Docker container registry helps with that a lot: we can just publish the
>> containers we built and pull/refresh them automatically with each run, and so
>> could outside developers. At least as far as I understand it. I don't think
>> there is a similar service for VM images, and I don't think we want to
build
>> our own.
>
> There is:
>
https://atlas.hashicorp.com/boxes/search
> but of course, this would require the system running the test to have
> vagrant installed.
>
> Looking at how Cockpit runs their test, they simply publish qcow2 disk
> images on Internet and fetch them from there..
>
>>
>> Even though Docker is still limiting, it is less limiting than the current
>> setup.
>
> (Replying to Michal also here..)
>
> In my view, whether we use containers or VMs should be a detail as long
> as the test is concerned, the test would just execute and output its
> results "somewhere". Of course, the test runner must set up the
> environment for the test and that's where we either provision a VM or a
> container.
>
> Containers might be more dense (less resource-hungry) and it's quite
> easy to bind-mount directories to share data like log files or the test
> output. On the other hand, I know that running some services, like IPA,
> in a container is not totally straightforward.
This is a genuine question ... can we have Windows container?
The question is because running AD tests would be more then welcome
and in case we cannot have a Windows container, choosing a provision
VM the more logical way to go.
I think we should experiment with samba4 at least for AD provider tests
and Nick already did some work a couple of months ago to create a
proof-of-concept of using samba4 for tests with cwrap.
Until the MIT Kerberos work is completed, we can't establish a trust
scenario between IPA and samba4 I think, but the MIT work is hopefully
going to be completed in the near future.